CPU Sizing for Concurrent Client Connections to EAServer
How many simultaneous client connections do you think is reasonable for a server?
This document helps you understand how to answer that question, by examining response times in a sample application scenario.
ScenarioTest_Servlet1 - with a known response time of 1.5 seconds -- is deployed to EAServer on ServerMachine1.
ServerMachine1 is an Windows or Solaris box, with 2 CPUs.
Each minute, there are 600 concurrent client requests for Test_Servlet1.
ResultServerMachine1 begins thrashing, EAServer generates messages that the server is busy, and/or may hang.
ExplanationServerMachine1 is undersized.
When there are 600 requests per minute, that averages out to 10 requests per
Since there are 2 CPUs, each CPU receives half of the requests; 5 requests per second.
In order to process 5 requests per second, the request would have to complete in 0.2 second.
Since the response time for Test_Servlet1 is actually 1.2 seconds, the request cannot be completed.
Therefore, the requests wait in line for a small slice of CPU time, then go to the end of the line to wait for another slice, and repeat this until it received enough slices to be able to complete. This causes a backlog, and thrashing begins.
For example, imagine a CPU with backlog of even 50 requests for Test_Servlet1 that are waiting to complete. The first request in line gets one fiftieth of a second, then goes to the end of the line to wait for another fiftieth of a second, and this continues. The request may never get to finish when more and more requests are coming in and waiting in line.
To make it worse, many of the end-users start hitting the browser Refresh button
since a response did not come back quickly enough.
(The presence of many disconnect errors in the EAServer log file can be a telltale sign that this is happening.)
Although a Unix machine would show 100% CPU usage in this type of scenario, Windows may only show 60-70% CPU usage.
To identify the problem, use a CPU monitoring tool (such as PSTAT on Windows) to view the number of threads waiting, and their average execution time.
Upgrade the server.
As an average guideline, assuming the application code is designed and tuned properly:
- Solaris: consider allocating 100 concurrent requests per CPU
- Windows: allocate 75 concurrent requests per CPU, for the first two CPUs. After 2 CPUs, testing has shown that performance is not linear, so you will need to stress test your application to determine appropriate sizing.
For more specific tuning suggestions, see the TechWave 2003 presentation Application Server Benchmarks and Sizing.
Although it may reduce scalability, you can also prevent a backlog of requests by limiting the maximum number of connections to EAServer.