How Stephen Hawking Taught Us an Important Lesson About Preparing for Traffic Spikes
December 06, 2017

Archana Kesavan
ThousandEyes

Share this

The recent outage of the University of Cambridge website hosting Stephen Hawking's doctoral thesis is a prime example of what happens when niche websites become exposed to mainstream levels of traffic.

The widespread fame of the author as one of the figureheads of science generated a level of interest the university's web team was not prepared to handle, resulting in a familiar story: Website goes live; minutes or hours later, it crashes due to the large influx of traffic.

While it is obvious that the University of Cambridge didn't expect the level of traffic they saw, there are steps organizations and enterprises of all sizes can take to prevent this kind of digital downtime.

On Oct. 23, Hawking's Ph.D thesis went live, but by Oct. 24, the website had crashed. The release of the paper was timed with Open Access Week 2017, a worldwide event aimed at promoting free and open access to scholarly research. Though the scholarly research was made available through the university, within 24 hours of its release, no one could access it.

According to a Cambridge spokesperson, the website received nearly 60,000 download requests in less than 24 hours, causing a shutdown of the page, slower runtimes, and inaccessible content for users.

While this could be the first time a doctoral thesis invoked such widespread interest, this kind of problem, due to overloaded networks has unfolded before. In this case, it seems that the sudden increase in the number of visitors saturated the infrastructure that hosts and delivers this research. This happens when the amount of processing power required to determine what the searcher is looking for and where to send it exceeds the ability of the machines (routers, switches and servers) on the network to respond.

Organizations like Cambridge University often have limited processing power on their networks either because they build their own data centers, reducing their flexibility to respond to spikes in traffic. While each individual request may only take a fraction of each machine's resources, when several come in at once, it can slow connections, create congestion or even absolute failure.


Figure 1: Global locations unable to access the Cambridge University website, with errors in the connect and receive stages.


Figure 2: Traffic from all over the world terminates within the Cambridge infrastructure, as indicated by the spike in packet loss

For a web property like the Cambridge library, this is a temporary surge in traffic -- but not all websites are this lucky. The lesson is that if an organization isn't prepared, this is how a problem would manifest itself. Pre-planning for a spike would include increasing capacity on existing infrastructure. Leveraging a CDN can also help distribute the load across servers/geographies.

As you make important decisions about your company's website, there are many factors you'll want to consider, especially if you're expecting a surge (like on Black Friday or Cyber Monday). For sites that have spiky, but predictable traffic, here are a few options to help them stay online:

■ Use a CDN to serve up traffic round-the clock. This costs more but will have the best customer experience.

■ Flip on a CDN service well before known traffic peaks. If Cambridge had done this prior to releasing Hawking's thesis, they could have stayed afloat during the massive download requests.

■ Diversify with multiple data centers and upstream ISPs. If your organization has only one data center and one upstream ISP — if the ISP or their single data center goes down, your service goes with it.

■ Within the data center, load balanced network paths and web servers can also help reduce performance impacts.

The University of Cambridge may not plan to release another legendary scientist's thesis again anytime soon, but when it comes to web performance, you can have a guaranteed return if you properly prepare for your network's next big event.

Archana Kesavan is Sr. Network Analyst at ThousandEyes
Share this

The Latest

February 14, 2019

Part 3 of our three-part blog series on the shortcomings of traditional APM solutions for monitoring microservices based applications explains how the alerting and troubleshooting capabilities of traditional APM do not address the evolving requirements of monitoring microservices based applications ...

February 13, 2019

In a digital world where the speed of innovation matters, are you anchored down by legacy APM agents? ...

February 12, 2019

In a digital world where customer experience defines your business, is your APM solution doing its job? This may seem like a strange question to open a technical blog on Application Performance Management (APM), but it's not. With customer experience today largely driven by software, we think there's no more important question to ask ...

February 11, 2019

According to the NetEnrich 2019 Cloud Adoption survey, 68% of enterprise IT departments are using public cloud infrastructure today, and 27% of respondents said that doing so is part of their near-term plan ...

February 08, 2019

Organizations and their IT teams are not in sync when pursuing their digital transformation strategies, according to a new report released today by The Economist Intelligence Unit ...

February 07, 2019

Having the right tools and good visibility are critical to understanding what's going on in your network and applications. However, as networks become more complex and hybrid in nature, organizations can no longer afford to be reactive and rely only on portable diagnostic tools. They need real-time, comprehensive visibility ...

February 06, 2019

When building out new services, SaaS providers need to keep in mind a set of best practices and "habits of success," which cover their organization's culture, relationships with third-party providers and customers, and overall strategic decisions and operational know-how. If you're a SaaS application provider, here are five considerations you need to keep in mind ...

February 05, 2019

In the coming weeks, EMA will be gathering data on what we believe is a unique research topic — approaching DevOps initiatives from the perspectives of all key constituents. We're doing this to try to break through some of the "false walls" created by more niche, market-defined insights, or some of our industry hyperbole. Here are some of the directions we're pursuing ...

February 01, 2019

An application on your network is running slow. Before you even understand what the problem is, the network is blamed for the issue. This puts network teams in a dangerous position — guilty until proven innocent. Even when network teams are sure an issue doesn't stem from a network problem, they are still forced to prove it, spending sometimes significant amounts of time going through troubleshooting processes, looking for a problem that doesn't exist ...

January 31, 2019

Tap and SPAN. It's the same thing, right? That answer would be wrong. Some network engineers may not know the difference, but there are definitely clear and distinct differences between these two types of devices. Understanding these differences will help you elevate your game when it comes to network performance monitoring and application performance monitoring ...