All Clouds Are Not Created Equal
February 27, 2020
Share this

The Cloud Performance Benchmark from ThousandEyes compares global network performance and connectivity differences between the five major public cloud providers — Amazon Web Services (AWS), Google Cloud Platform (GCP), Microsoft Azure, Alibaba Cloud and IBM Cloud — proving that, when it comes to performance, not all clouds are created equal.

"When businesses need to decide which cloud provider best meets their needs, one metric that's notably missing from their assessments has been performance data, mainly because it's never been available or has, at best, been myopic," said Archana Kesavan, research author and Director of Product Marketing at ThousandEyes. "The second annual Cloud Performance Benchmark gives businesses that comparative data. Understanding cloud performance is essential for planning and for ongoing measurement so you can be assured that you're providing customers and employees with the best possible performance."

Results were derived from an analysis of over 320 million data points collected from 98 global metro locations over the course of 30 days, using both end-to-end performance and in-depth path measurement techniques.

Notable findings include:

RELIANCE ON PUBLIC CLOUD

Some cloud providers rely heavily on the public Internet to transport traffic instead of their backbones, which can impact performance predictability. While Google Cloud and Azure rely heavily on their private backbone networks to transport their customer traffic, protecting it from performance variations associated with delivering over the public Internet, AWS and Alibaba Cloud rely heavily on the public Internet for the majority of transport, resulting in greater operational risk that can impact performance predictability. IBM takes a hybrid approach that varies regionally.

Regional performance differences

Latin America and Asia have the highest performance variations across all clouds, whereas in North America, cloud performance is generally comparable. Regional performance differences can make a significant impact in terms of performance gains or losses.

AWS Global Accelerator

AWS Global Accelerator doesn't always out-perform the Internet. AWS Global Accelerator was introduced in November 2018 to let customers use the AWS private backbone network for a fee — rather than use the public Internet, which is AWS' default behavior.

While there are many examples of performance gains in various regions around the world, the Global Accelerator is not a one-size-fits-all solution, as there are several examples where the Internet actually performs faster and more reliably than Global Accelerator, or, the results are negligible.

Broadband ISP

Broadband ISP choice makes a difference in cloud performance. Businesses looking to get every performance edge possible should consider which broadband ISP provider they select, depending on which cloud they most heavily rely upon. There are performance gains and losses depending on which broadband provider businesses use to connect to each cloud.

The Great Firewall of China

All cloud providers pay a performance toll when crossing the Great Firewall of China. Despite Alibaba's origins in China, it experiences packet loss when crossing through China's Great Firewall just as all the other cloud providers do, showing it does not get any preferential treatment.

Enterprises serving customers in China, but hesitant to pick a hosting region in China due to stringent regulations on traffic and data privacy, might consider Hong Kong as a viable option — Alibaba Cloud traffic experienced the least packet loss from Hong Kong to China, followed by Azure and IBM.

Benchmark Findings By Cloud Provider

The inaugural Cloud Performance Benchmark published in November of 2018 benchmarked the three biggest cloud providers (by market share) AWS, Azure and GCP. Alibaba Cloud and IBM Cloud are new this year, meaning they have no prior comparison points. There are notable differences between each cloud in terms of overall performance and connectivity architectures.

Key findings, by cloud provider include:

AWS generally demonstrates low latency and made some minor improvements over the last year. Its performance predictability metrics, however, noticeably improved, with its most dramatic change in Asia, which showed a 42 percent reduction in variability over last year. However, when compared to Azure and GCP, it still has lower performance predictability due to its extensive reliance on the Internet rather than leveraging its own backbone for delivery.

Azure continues its strong network performance based on aggressive use of its own backbone to carry user traffic to cloud hosting regions. Standout changes from 2018 to 2019 include on average a 50 percent improvement in performance predictability in Sydney, whereas in India, there was a 31 percent decrease in performance predictability. Despite a slight decrease year over year, Azure continues to lead in performance predictability in Asia when compared to the other cloud providers.

Google Cloud continues to favor use of its own backbone for user to cloud hosting region traffic delivery and has strong performance in most regions, but it still has some significant global gaps that haven't been resolved since last year's report. Traffic from Europe and Africa takes 2.5-3 times longer to get to India, going around the rest of the world instead of taking a direct route. Google Cloud also decreased visibility into their internal network, making it harder for its users to understand its network paths and performance.

Alibaba Cloud offers comparable performance to other cloud providers. It closely resembles AWS in terms of connectivity patterns, region locations, and even region naming constructs. Like AWS, Alibaba Cloud leans heavily on the Internet rather than their own private network backbone for the majority of user traffic transport. Uniquely, inter-region traffic between Alibaba Cloud regions is not contained within their own cloud infrastructure as it is for the other four, instead it exits Alibaba Cloud, traverses the Internet and then makes its way back into Alibaba Cloud.

IBM Cloud performance is comparable to the major players. IBM takes a hybrid approach to traffic delivery, fluctuating between using its own private backbone and the public Internet, depending on which regions user traffic is accessing.

"Historically, most IT professionals select cloud providers based on factors such as price or proximity to users, but often overlook the underlying network architecture of the cloud providers, which can have a significant impact on performance," said Zeus Kerravala, principal analyst, ZK Research. "The 2019 ThousandEyes Cloud Performance Benchmark shows just how widely performance can vary from the top cloud service providers and reinforces the point that businesses should remain diligent about collecting network intelligence and measure their own performance to ensure they are getting what they expect from their cloud providers."

Share this

The Latest

June 25, 2020

I've had the opportunity to work with a number of organizations embarking on their AIOps journey. I always advise them to start by evaluating their needs and the possibilities AIOps can bring to them through five different levels of AIOps maturity. This is a strategic approach that allows enterprises to achieve complete automation for long-term success ...

June 24, 2020

Sumo Logic recently commissioned an independent market research study to understand the industry momentum behind continuous intelligence — and the necessity for digital organizations to embrace a cloud-native, real-time continuous intelligence platform to support the speed and agility of business for faster decision-making, optimizing security, driving new innovation and delivering world-class customer experiences. Some of the key findings include ...

June 23, 2020

When it comes to viruses, it's typically those of the computer/digital variety that IT is concerned about. But with the ongoing pandemic, IT operations teams are on the hook to maintain business functions in the midst of rapid and massive change. One of the biggest challenges for businesses is the shift to remote work at scale. Ensuring that they can continue to provide products and services — and satisfy their customers — against this backdrop is challenging for many ...

June 22, 2020

Teams tasked with developing and delivering software are under pressure to balance the business imperative for speed with high customer expectations for quality. In the course of trying to achieve this balance, engineering organizations rely on a variety of tools, techniques and processes. The 2020 State of Software Quality report provides a snapshot of the key challenges organizations encounter when it comes to delivering quality software at speed, as well as how they are approaching these hurdles. This blog introduces its key findings ...

June 18, 2020

For IT teams, run-the-business, commodity areas such as employee help desks, device support and communication platforms are regularly placed in the crosshairs for cost takeout, but these areas are also highly visible to employees. Organizations can improve employee satisfaction and business performance by building unified functions that are measured by employee experience rather than price. This approach will ultimately fund transformation, as well as increase productivity and innovation ...