500 server error on cache flush [twig specifically]

I am getting a 500 error, The request timed out, The web server failed to respond within the specified time.

This only happens if I flush all cache or flush twig cache, but not for css/js cache flush or other cache flush. Any idea how I can find out the cause of the error.

Thank you