How Stephen Hawking Taught Us an Important Lesson About Preparing for Traffic Spikes
December 06, 2017

Archana Kesavan
ThousandEyes

Share this

The recent outage of the University of Cambridge website hosting Stephen Hawking's doctoral thesis is a prime example of what happens when niche websites become exposed to mainstream levels of traffic.

The widespread fame of the author as one of the figureheads of science generated a level of interest the university's web team was not prepared to handle, resulting in a familiar story: Website goes live; minutes or hours later, it crashes due to the large influx of traffic.

While it is obvious that the University of Cambridge didn't expect the level of traffic they saw, there are steps organizations and enterprises of all sizes can take to prevent this kind of digital downtime.

On Oct. 23, Hawking's Ph.D thesis went live, but by Oct. 24, the website had crashed. The release of the paper was timed with Open Access Week 2017, a worldwide event aimed at promoting free and open access to scholarly research. Though the scholarly research was made available through the university, within 24 hours of its release, no one could access it.

According to a Cambridge spokesperson, the website received nearly 60,000 download requests in less than 24 hours, causing a shutdown of the page, slower runtimes, and inaccessible content for users.

While this could be the first time a doctoral thesis invoked such widespread interest, this kind of problem, due to overloaded networks has unfolded before. In this case, it seems that the sudden increase in the number of visitors saturated the infrastructure that hosts and delivers this research. This happens when the amount of processing power required to determine what the searcher is looking for and where to send it exceeds the ability of the machines (routers, switches and servers) on the network to respond.

Organizations like Cambridge University often have limited processing power on their networks either because they build their own data centers, reducing their flexibility to respond to spikes in traffic. While each individual request may only take a fraction of each machine's resources, when several come in at once, it can slow connections, create congestion or even absolute failure.


Figure 1: Global locations unable to access the Cambridge University website, with errors in the connect and receive stages.


Figure 2: Traffic from all over the world terminates within the Cambridge infrastructure, as indicated by the spike in packet loss

For a web property like the Cambridge library, this is a temporary surge in traffic -- but not all websites are this lucky. The lesson is that if an organization isn't prepared, this is how a problem would manifest itself. Pre-planning for a spike would include increasing capacity on existing infrastructure. Leveraging a CDN can also help distribute the load across servers/geographies.

As you make important decisions about your company's website, there are many factors you'll want to consider, especially if you're expecting a surge (like on Black Friday or Cyber Monday). For sites that have spiky, but predictable traffic, here are a few options to help them stay online:

■ Use a CDN to serve up traffic round-the clock. This costs more but will have the best customer experience.

■ Flip on a CDN service well before known traffic peaks. If Cambridge had done this prior to releasing Hawking's thesis, they could have stayed afloat during the massive download requests.

■ Diversify with multiple data centers and upstream ISPs. If your organization has only one data center and one upstream ISP — if the ISP or their single data center goes down, your service goes with it.

■ Within the data center, load balanced network paths and web servers can also help reduce performance impacts.

The University of Cambridge may not plan to release another legendary scientist's thesis again anytime soon, but when it comes to web performance, you can have a guaranteed return if you properly prepare for your network's next big event.

Archana Kesavan is Sr. Network Analyst at ThousandEyes
Share this

The Latest

June 23, 2022

Digital businesses don't invest in monitoring for monitoring's sake. They do it to make the business run better. Every dollar spent on observability — every hour your team spends using monitoring tools or responding to what they reveal — should tie back directly to business outcomes: conversions, revenues, brand equity. If they don't? You might be missing the forest for the trees ...

June 22, 2022

Every day, companies are missing customer experience (CX) "red flags" because they don't have the tools to observe CX processes or metrics. Even basic errors or defects in automated customer interactions are left undetected for days, weeks or months, leading to widespread customer dissatisfaction. In fact, poor CX and digital technology investments are costing enterprises billions of dollars in lost potential revenue ...

June 21, 2022

Organizations are moving to microservices and cloud native architectures at an increasing pace. The primary incentive for these transformation projects is typically to increase the agility and velocity of software release and product innovation. These dynamic systems, however, are far more complex to manage and monitor, and they generate far higher data volumes ...

June 16, 2022

Global IT teams adapted to remote work in 2021, resolving employee tickets 23% faster than the year before as overall resolution time for IT tickets went down by 7 hours, according to the Freshservice Service Management Benchmark Report from Freshworks ...

June 15, 2022

Once upon a time data lived in the data center. Now data lives everywhere. All this signals the need for a new approach to data management, a next-gen solution ...

June 14, 2022

Findings from the 2022 State of Edge Messaging Report from Ably and Coleman Parkes Research show that most organizations (65%) that have built edge messaging capabilities in house have experienced an outage or significant downtime in the last 12-18 months. Most of the current in-house real-time messaging services aren't cutting it ...

June 13, 2022
Today's users want a complete digital experience when dealing with a software product or system. They are not content with the page load speeds or features alone but want the software to perform optimally in an omnichannel environment comprising multiple platforms, browsers, devices, and networks. This calls into question the role of load testing services to check whether the given software under testing can perform optimally when subjected to peak load ...
June 09, 2022

Networks need to be up and running for businesses to continue operating and sustaining customer-facing services. Streamlining and automating network administration tasks enable routine business processes to continue without disruption, eliminating any network downtime caused by human error or other system flaws ...

June 08, 2022

Enterprises have had access to various Project and Portfolio Management (PPM) tools for quite a few years, to guide in their project selection and execution lifecycle. Yet, in spite of the digital evolution of management software, many organizations still fail to construct an effective PPM plan or utilize cutting-edge management tools ...

June 07, 2022

It has become increasingly difficult for DevOps and SRE teams to minimize the impact of issues and ensure high-quality end-user experiences. In this blog, I'm going to propose a new approach to support real-time use cases — edge observability — that enables you to detect issues as they occur and resolve them in minutes ...