10,000 simultaneous page views on a €4 server

This weekend I thought it would be useful to see how effective low cost solutions can be at scaling up. I found that even if you don't have a huge budget you can still have a site that can handle respectable volumes of traffic.

Of course 10,000 simultaneous requests would correspond to a much higher number of visitors on your site. When I'm generating traffic I'm not trying to simulate how long a person views a single page before requesting the next one. On most sites at any given time most visitors will be looking at content rather than requesting new content.

To show how to get 10,000 concurrent requests on a cheap server I'm going to use a very cheap VM and hit it with requests from Loader.io. The exact specifications of the VM are on the companies website (here).

It's a very basic single core, 1 GB RAM virtual machine on shared tenancy. It costs me four Euro a month and it runs my book website and BOINC software in its spare time. So as far as web-servers go this is a very humble one and it's even being forced to donate its spare compute cycles to help cure cancer and find evidence of neutron stars.

It's running Nginx as a web-server with a tuned instance of PHP7.1-FPM serving up PHP pages. Nginx serves static assets itself and sets all of the appropriate headers that let devices know they can cache them.

For this particular site the homepage is static and doesn't change for different users, so at the top I'm using PHP to set cache control headers, like this:

<?php
  header('Cache-Control: max-age=604800, public');
  if (isset($_SERVER['HTTP_IF_NONE_MATCH']) && $_SERVER['HTTP_IF_NONE_MATCH'] == sha1('version 1.0')) {
      header('HTTP/1.1 304 Not Modified');
      exit;
  }
  header('ETag: ' . sha1('version 1.0') );
?>

This indicates to Nginx and other intermediary caches that it's safe to cache the content and give it to any user who asks. I set an ETag that is based on a static version number because the only time I would want to process the file is if the version changes.

In my Nginx config I'm setting up fastcgi_cache so that I respect the cache control header and only send requests to PHP if my cache is old. With this setup Nginx will be serving the majority of requests directly and passing very little (if anything) to PHP.

Nginx is a very capable server and on my limited and overworked VM is able to serve up about 1,000 concurrent page requests before it starts having difficulty and giving timeouts. This is far in excess of the traffic on that site to be, so there really is no need to scale other than to see how far I can go.

I decided to increase how aggressively Cloudflare caches my site. It's a static page so there's no good reason not to cache it as much as possible on the edge locations right?

To do this I go into Cloudflare and set up a special page rule that will override the default way that Cloudflare caches stuff. This rule will force it to cache everything on my site. I can only do this because the site is static and doesn't change for different users.

Next I open up Loader and set up a test. I'm going to ramp up from 1000 users all the way to 10,000 concurrent connections. The Loader graph shows increasing the number of visitors did very little to the response time. 

Even with 10,000 users hitting the site I'm responding in under 800 milliseconds. Well of course actually I'm not - the CDN is returning all of the data so my webserver isn't being hit at all.

Of course this is a special case and not every page is static and able to be shared by all of your users. More advanced rules would let you set more intelligent timeouts and identify which parts of your system are able to be shared between users. This is a very basic website, however, so we can get away with a lot less.