Surviving the Reddit-Hug-of-Death
Due to a recent policy shift at one of the major Internet image sharing websites there became a demand for a more transparent, less totalitarian, platform the users on Reddit could use. Luckily, I'd been developing imgn.io for a few weeks and it already had a slew of major bugs reported by its small community. However, being able to meet the demands of a fleeing user base within 30 minutes was something I'm glad I'd also been preparing for!
When your website is linked on Reddit, Facebook, Twitter, or any other major social media platform you run the risk of attracting a lot of users very fast. This is known, comically, as "The Hug of Death." So many people are interested, so quickly, that they zap your bandwidth and crash the infrastructure. They like you so much they kill the service. Luckily, not every post is going to blow up the servers, but once in a blue moon you might have the hottest content on-line for a couple of hours, do you have a plan to deal with this?
During development I dreamed of such a catastrophe, so I reserved a few safeguards to meet such a demand. However, there are more generic solutions simple enough to consider by anyone, today, before they implement more robust features moving forward.
1. Have a beefy entry server that handles the majority of the incoming connections.
Whether the website is a lone server or part of a small cluster, it's important to have the beefiest server handling the incoming connections. It's also important to have the most robust software at this critical junction. I usually choose nginx as my public facing HTTP server, acting as a reverse proxy for the underlying logic. It's lightweight, super fast, and incredibly battle tested. A single nginx process can handle tens of thousands of (real world) requests per second, fully saturating the available bandwidth. In order for this to work, however, you first have to...
2. Write scalable web server software that can be spun up and down on demand.
It's great that nginx is handling the bulk of your connections, but unless it's serving flat files from the disk you're going to need to use its reverse proxy to allow other servers to generate the content. These applications will build the HTML, handle user sessions, and manage database connections to keep new data pumping through nginx as fast as possible. You may only need one or two of these servers day to day, but once social media comes in for a squeeze you should be able to scale into the dozens effortlessly.
However, no matter how many back end workers you have going, you'll still overload the database and exhaust the bandwidth unless you...
3. Cache. Cache. Cache.
Cache from the database. Cache in the web servers. Cache in nginx. Cache on the client's browser. Consider using a commercial CDN, like CloudFlare, to globally cache the outgoing data as much as possible.
Caching presents a lot of interesting scenarios given cases where a lot of dynamic content is being uniquely generated per user, but in most of these Death-Hug cases that won't be a problem. There's probably ONE page, whatever was linked on social media, that will get 99% of the traffic, and it's going to be exactly the same every time for every person. If the final output is sitting in cache, or the generated templates are sitting in their cache, or even the database queries are primed then you're going to have a good time and breeze through your success. The less work your servers HAVE to do, the more they're CAPABLE of doing; such as non-cached pages that were found by engaged users after being directed from something else.
4. Pray and Enjoy!
Watch the stats, check the performance, grab a drink and hope nothing important breaks.
By having a scalable, cached, website ready to go...imgn.io was able to handle 5-100 requests per second on the same infrastructure that normally hums around 5-100 per hour. At this very moment we're handling hundreds of concurrent users on $20/mo hardware.
-Blake VandeMerwe