Performance Testing and APM: Why You Need Both
October 23, 2015

Paola Moretto
Nouvola

Share this

Performance Testing (aka synthetic testing) and APM (Application Performance Management and monitoring solutions) are often regarded as two competing solutions, as if one were an alternative to the other. They are actually perfect complementary tools to achieve optimal performance for your application. Whether you develop web applications, SaaS products or mobile apps, you‘ll find both approaches to be absolutely necessary in your software operations. Here, I explain why. (I’ll use “monitoring solutions” as a synonym for APM solutions.)

Let’s talk about performance for a moment. Performance can refer to end-user performance or backend performance.

End-user performance is a measurement of the user experience as relates to speed, responsiveness and scalability. Page load time is one of the typical metrics for the user experience.

Backend performance, as the name suggests, refers to the performance of backend resources used by the application. This is something end users may not be aware of, but which is important to the way the app performs.

Let’s look at APM or monitoring solutions first. These tools are normally used to collect data from a live environment. Monitoring solutions help gather important information on the application backend metrics, server behaviors, slow components and transactions. Several application monitoring solutions track database and browser performance as well.

Monitoring is Not Enough

There are many kinds of monitoring solutions that operate at different levels of the stack, with different timing granularity, with the opportunity to define custom metrics. You have server monitoring, infrastructure monitoring, user monitoring, high-frequency metrics, and others. What is clear with monitoring solutions is that one is not enough. You probably need a portfolio of tools to have a clear understanding of what is happening with your application. Most tools have alerts systems that provide real-time information on page load and are designed to notify you in real time of events so you are immediately informed about deteriorating performance.

It doesn’t matter how comprehensive your monitoring architecture is, it doesn’t provide the full picture. First of all, live traffic is noisy. You have no control over what your users are doing and how many users you have at any given point. If you have a performance issue, that makes it really hard to troubleshoot. Was that expected behavior? Did they just hit a corner case previously unanticipated? Is a combination of the live workloads that crashed the system? So even though the technology is designed to enable real-time troubleshooting, the reality is that since it’s not a controlled environment, you might not be able to identify root cause performance issues in a timely manner.

Second and most important, the information produced by monitoring is delivered after the fact. Monitoring is like calling AAA after an accident. It’s a great service to have, but it’s much better to prevent the accident in the first place.

This explains why you need to add performance testing to the mix. While monitoring can inform you about performance after the fact, performance testing can help you prevent bad things from happening.

While monitoring is usually done on your live/production environment, performance testing usually utilizes synthetic traffic on a pre-production/staging environment. Having a pre-production environment as close as possible to your production environment will help you derive the most meaningful results.

In performance testing, users are simulated but the traffic is absolutely real. You can apply different types of loads to discover the breaking point of your application before it goes live. You can use performance testing to test with traffic that is higher than anything your actual application has seen – yet – so that you can prepare for peak of traffic.

Performance testing can also help you identify performance degradations that might have resulted from code changes, infrastructure changes or third party changes. It basically answers the question: “Can you trust this build to deliver the same user experience your users are counting on?”

In performance testing, you have total control over the amount of traffic and the workloads your users execute. That makes it a lot easier to troubleshoot.

And, if you are using Docker or containers-based architecture, you could also test performance improvements, under different configurations and platforms easily.

With performance testing, you can also measure end-to-end performance – a good indication of user experience -- which gives visibility into the entire application delivery chain, enabling greater transparency and targeted troubleshooting.

You wouldn’t build a bridge and then send hundreds of thousands of cars over it without first doing structural tests for structural problems. But you also wouldn’t open a bridge to traffic without continually monitoring how it is holding up under all the traffic. You need to do both, whether you’re talking about bridges or apps. The difference is, while a bridge is static and needs to be tested only once or at periodic intervals, software today is highly dynamic and needs to be tested on a daily basis as part of your regular flow.

Used together, performance testing and monitoring make a great team, so to speak. Use both to make sure you deploy a reliable product.

Paola Moretto is Founder and CEO of Nouvola.

Share this

The Latest

November 12, 2019

We're in the middle of a technology and connectivity revolution, giving us access to infinite digital tools and technologies. Is this multitude of technology solutions empowering us to do our best work, or getting in our way? ...

November 07, 2019

Microservices have become the go-to architectural standard in modern distributed systems. While there are plenty of tools and techniques to architect, manage, and automate the deployment of such distributed systems, issues during troubleshooting still happen at the individual service level, thereby prolonging the time taken to resolve an outage ...

November 06, 2019

A recent APMdigest blog by Jean Tunis provided an excellent background on Application Performance Monitoring (APM) and what it does. A further topic that I wanted to touch on though is the need for good quality data. If you are to get the most out of your APM solution possible, you will need to feed it with the best quality data ...

November 05, 2019

Humans and manual processes can no longer keep pace with network innovation, evolution, complexity, and change. That's why we're hearing more about self-driving networks, self-healing networks, intent-based networking, and other concepts. These approaches collectively belong to a growing focus area called AIOps, which aims to apply automation, AI and ML to support modern network operations ...

November 04, 2019

IT outages happen to companies across the globe, regardless of location, annual revenue or size. Even the most mammoth companies are at risk of downtime. Increasingly over the past few years, high-profile IT outages — defined as when the services or systems a business provides suddenly become unavailable — have ended up splashed across national news headlines ...

October 31, 2019

APM tools are ideal for an application owner or a line of business owner to track the performance of their key applications. But these tools have broader applicability to different stakeholders in an organization. In this blog, we will review the teams and functional departments that can make use of an APM tool and how they could put it to work ...

October 30, 2019

Enterprises depending exclusively on legacy monitoring tools are falling behind in business agility and operational efficiency, according to a new study, Prevalence of Legacy Tools Paralyzes Enterprises' Ability to Innovate conducted by Forrester Consulting ...

October 29, 2019

Hyperconverged infrastructure is sometimes referred to as a "data center in a box" because, after the initial cabling and minimal networking configuration, it has all of the features and functionality of the traditional 3-2-1 virtualization architecture (except that single point of failure) ...

October 28, 2019

Hyperconvergence is a term that is gaining rapid interest across the manufacturing industry due to the undeniable benefits it has delivered to IT professionals seeking to modernize their data center, or as is a popular buzzword today ― "transform." Today, in particular, the manufacturing industry is looking to hyperconvergence for the potential benefits it can provide to its emerging and growing use of IoT and its growing need for edge computing systems ...

October 24, 2019

More than 92 percent of US respondents agree that Artificial Intelligence (AI) and Machine Learning (ML) will become important for how they run their digital systems ...