Posted on 09 December 2009.
Roger created and runs an educational site about the science of the so-called “golden hour” – the time around sunrise and sunset that has the most fantastic light for taking photographs. Although there were lots of websites giving tables of sunrise and sunset times, I wanted to create something more visual that showed the golden-hour visually and allowed you to explore how it changes with latitude and time of year.
It was an interesting project as it ties together various libraries such as the poly9 3D flash globe and openlayers mapping alongside astronomical algorithms and geolocation work. I was quite pleased with the site, put it online and then went back to work …
For a few months, there wasn’t really much interest in the site until all of a sudden in July, my traffic spiked quite significantly.
July Traffic Spike
Nothing too dramatic, a peak of 2,000 page views per day, so in the real world not really a “peak” at all, however I was intrigued to discover that the traffic came from a mention in a Canon newsletter. With enough traffic now to make it worthwhile, I spent some time adding in some advertising to the site, some Amazon widgets to sell Canon camera gear, and again, just left it and went back to work, this time pleased that other people were enjoying the site.
My headache however arrived a month later when all of a sudden I received a terms of service violation notice from my hosting company with a note that I had exceeded their CPU allowance and had disabled the site. Nice!
It became clear that something had triggered a much larger traffic spike and to get back online I needed to act fast.
Amazon EC2 to the rescue!
It just so happens that my work at Mindsizzlers has been exploring and evaluating cloud computing and I had been building applications in Amazon’s compute cloud for several months. This meant I had an account ready to go and was able to spin up a suitable server instance, upload the appropriate cgi scripts other resources that the golden hour required and have an alternative server up and running, literally within the space of 15 minutes or so.
My other good fortune was that I’m in the habit of using short DNS time-to-live values on many sites through DNS made easy (daft name, great service). This meant I could change the A-records for the golden hour and get the traffic flowing across pretty quickly. In the meantime I was able to put a redirect in at the old host to bounce the traffic into the cloud until things calmed down.
Incidentally, this was the traffic spike. It’s the first one in the graphic.
First spike took the site down, the second I was prepared for.
The spike was due to a mention on www.lifehacker.com and generated an intense burst of visitors that I had absolutely no warning about.
Without Amazon’s computing cloud, I’d have been sunk. My hosting company were fast to turn the site off, but then unacceptably slow at responding to any follow up. It took over 4 hours to restore even limited access and in fact it turned out that I had a widget that wasn’t delivering cached output which was the real cause of the problem in the first place.
With Amazon and EC2, I was able to get a new server on-line in minutes and the best part is I only needed to pay by the hour I was using it. Actually it’s even better than that. One USP about the Amazon cloud is that I’m not tied into having a particular operating system ( think Azure ) or writing my application in a particular language ( think Google Appengine ). This really is a killer feature of Amazon’s cloud, I can pick from hundreds (if not thousands) of machine images or build my own from a basic starting point, but at all times I am in control, I can pick the operating system, how much memory I need, what sort of CPU power I want – it really is the perfect instant on-line server shop and what makes it even more amazing is that it comes from a company that started life as a book seller.
Anyway, as things calmed down, what I did was have round robin DNS so that half the visitors are served by my original hosting company, and half are served by a cloud instance that I use for other projects too. This was very handy a few weeks later because the more observant of you will have noticed a second traffic spike on the chart – this time it came from being voted “cool site of the day” by America’s Digital Goddess (no, really, I’m not making this up). Two great things about that. Firstly they were kind enough to warn me a day or so in advance – thanks guys, and secondly with the round robin DNS in place, and the cloud humming away nicely, the servers barely broke into a sweat.