We discussed the methodology in the article, in this one we are testing HTTPS in more realistic scenarios. For testing, a Letβs Encrypt certificate was obtained, and Brotli compression was enabled at level 11.
This time we will attempt to replicate the scenario of deploying a server on a VDS or as a virtual machine on a host with a standard processor. The limit was set to:
- 25% β which translates to a frequency of ~ 1350MHz
- 35% - 1890MHz
- 41% β 2214MHz
- 65% β 3510MHz
The number of simultaneous connections decreased from 500 to 1, 3, 5, 7, and 9,
Results:
Delays:
TTFB was specifically isolated as a separate test because HTTPD Tools creates a new user for each individual request. This test is still quite detached from reality, as users will click on a couple of pages, and in reality, TTFP will play a significant role.
The first, or actually the very first request after the initial start of the IIS virtual machine averages about 120 ms.
All subsequent requests show TTFP at 1.5 ms. Apache and Nginx lag behind in this regard. The author personally considers this test the most indicative and would choose a winner based solely on it.
The result is not surprising, as IIS caches already compressed static content and does not recompress it each time it is accessed.
Time spent on one client
For performance evaluation, testing with one simultaneous connection is sufficient.
For example, IIS completed testing with 5000 users in 40 seconds, which is 123 requests per second.
The graphs below show the time until the complete transfer of site content. This represents the share of requests completed in a certain timeframe. In our case, 80% of all requests were processed in 8 ms on IIS and 4.5 ms on Apache and Nginx, while 98% of all requests were completed within an 8 millisecond interval on Apache and Nginx.
Time taken to process 5000 requests:
Time taken to process 5000 requests:
If you have a virtual machine with a 3.5GHz CPU and 8 cores, choose what you want. All web servers are very similar in this testing. We will discuss which web server to choose for each host below.
When it comes to a slightly more realistic situation, all web servers are neck and neck.
Throughput:
The delay chart based on the number of concurrent connections. The flatter and lower, the better. The last 2% were removed from the charts as they would make them unreadable.
Now let's consider the case where the server is hosted on virtual hosting. We'll take 4 cores at 2.2GHz and one core at 1.8GHz.
How scalability works
If you've ever seen how the IV characteristics of vacuum triodes, pentodes, and the like look, these charts will be familiar to you. This is precisely what we are trying to capture β saturation. The limit at which no matter how many cores you throw at it, the performance increase will not be noticeable.
Previously, the entire challenge was to process 98% of requests with the lowest delay across all requests, keeping the curve as flat as possible. Now, by constructing a different curve, we will find the optimal operating point for each of the servers.
To do this, we will take the Requests per second (RPR) metric. The horizontal axis represents frequency, and the vertical axis represents the number of requests processed per second, with lines indicating the number of cores.
It shows the correlation of how well Nginx processes requests one after another. 8 cores in this testing perform better.
This graph clearly shows how much better (not by much) Nginx performs on a single core. If you have Nginx, consider reducing the number of cores to one if you are only hosting static content.
Although IIS has the lowest TTFB according to DevTools in Chrome, it manages to lose out to both Nginx and Apache in a serious stress test from the Apache Foundation.
The entire curvature of the graphs is consistent.
A sort of conclusion:
Yes, Apache performs worse on 1 and 8 cores, while on 4 it works slightly better.
Yes, Nginx on 8 cores handles requests better one after another, performs worse on 1 and 4 cores, and struggles when connections are numerous.
Yes, IIS prefers 4 cores for multithreading loads and favors 8 cores for single-threaded tasks. Ultimately, IIS turned out to be slightly faster than all on 8 cores under high load, although all servers performed closely.
This is not a measurement error; the margin of error is no more than Β±1ms in delays and Β±2-3 requests per second for RPR.
The results showing 8 cores underperforming are not surprising; many cores and SMT/Hyperthreading significantly degrade performance if we have time constraints within which to complete the entire pipeline.
Source: habr.com
