WordPress Session Scaling

I have about 30k to 40k sessions per day on wordpress website, I have caching enabled, wp-optimize plugin I have php-fpm enabled my pool options is below: max-request = 200 process idle timout = 30 max-children = 40

my realtime stats for sessions on site goes up to 300 sometimes. i exprience sometimes downtime on my dedicated server.

Where am i going wrong?

my server config:

16processors each: cache: 16384 Intel(R) Xeon(R) D-2141I CPU @ 2.20GHz 2199.998 MHz

Memory: 4975220k/17825792k available (7784k kernel code, 1049112k absent, 532940k reserved, 5957k data, 1980k init)