Performance Testing and APM: Why You Need Both
October 23, 2015

Paola Moretto

Share this

Performance Testing (aka synthetic testing) and APM (Application Performance Management and monitoring solutions) are often regarded as two competing solutions, as if one were an alternative to the other. They are actually perfect complementary tools to achieve optimal performance for your application. Whether you develop web applications, SaaS products or mobile apps, you‘ll find both approaches to be absolutely necessary in your software operations. Here, I explain why. (I’ll use “monitoring solutions” as a synonym for APM solutions.)

Let’s talk about performance for a moment. Performance can refer to end-user performance or backend performance.

End-user performance is a measurement of the user experience as relates to speed, responsiveness and scalability. Page load time is one of the typical metrics for the user experience.

Backend performance, as the name suggests, refers to the performance of backend resources used by the application. This is something end users may not be aware of, but which is important to the way the app performs.

Let’s look at APM or monitoring solutions first. These tools are normally used to collect data from a live environment. Monitoring solutions help gather important information on the application backend metrics, server behaviors, slow components and transactions. Several application monitoring solutions track database and browser performance as well.

Monitoring is Not Enough

There are many kinds of monitoring solutions that operate at different levels of the stack, with different timing granularity, with the opportunity to define custom metrics. You have server monitoring, infrastructure monitoring, user monitoring, high-frequency metrics, and others. What is clear with monitoring solutions is that one is not enough. You probably need a portfolio of tools to have a clear understanding of what is happening with your application. Most tools have alerts systems that provide real-time information on page load and are designed to notify you in real time of events so you are immediately informed about deteriorating performance.

It doesn’t matter how comprehensive your monitoring architecture is, it doesn’t provide the full picture. First of all, live traffic is noisy. You have no control over what your users are doing and how many users you have at any given point. If you have a performance issue, that makes it really hard to troubleshoot. Was that expected behavior? Did they just hit a corner case previously unanticipated? Is a combination of the live workloads that crashed the system? So even though the technology is designed to enable real-time troubleshooting, the reality is that since it’s not a controlled environment, you might not be able to identify root cause performance issues in a timely manner.

Second and most important, the information produced by monitoring is delivered after the fact. Monitoring is like calling AAA after an accident. It’s a great service to have, but it’s much better to prevent the accident in the first place.

This explains why you need to add performance testing to the mix. While monitoring can inform you about performance after the fact, performance testing can help you prevent bad things from happening.

While monitoring is usually done on your live/production environment, performance testing usually utilizes synthetic traffic on a pre-production/staging environment. Having a pre-production environment as close as possible to your production environment will help you derive the most meaningful results.

In performance testing, users are simulated but the traffic is absolutely real. You can apply different types of loads to discover the breaking point of your application before it goes live. You can use performance testing to test with traffic that is higher than anything your actual application has seen – yet – so that you can prepare for peak of traffic.

Performance testing can also help you identify performance degradations that might have resulted from code changes, infrastructure changes or third party changes. It basically answers the question: “Can you trust this build to deliver the same user experience your users are counting on?”

In performance testing, you have total control over the amount of traffic and the workloads your users execute. That makes it a lot easier to troubleshoot.

And, if you are using Docker or containers-based architecture, you could also test performance improvements, under different configurations and platforms easily.

With performance testing, you can also measure end-to-end performance – a good indication of user experience -- which gives visibility into the entire application delivery chain, enabling greater transparency and targeted troubleshooting.

You wouldn’t build a bridge and then send hundreds of thousands of cars over it without first doing structural tests for structural problems. But you also wouldn’t open a bridge to traffic without continually monitoring how it is holding up under all the traffic. You need to do both, whether you’re talking about bridges or apps. The difference is, while a bridge is static and needs to be tested only once or at periodic intervals, software today is highly dynamic and needs to be tested on a daily basis as part of your regular flow.

Used together, performance testing and monitoring make a great team, so to speak. Use both to make sure you deploy a reliable product.

Paola Moretto is Founder and CEO of Nouvola.

Share this

The Latest

March 16, 2018

The State of the Mainframe report from Syncsort revealed an increased focus on traditional data infrastructure optimization to control costs and help fund strategic organizational projects like AI, machine learning and predictive analytics in addition to widespread concern about meeting security and compliance requirements ...

March 15, 2018

The 2018 Software Fail Watch report from Tricentis investigated 606 failures that affected over 3.6 billion people and caused $1.7 trillion in lost revenue ...

March 14, 2018

Gartner predicts there will be nearly 21 billion connected “things” in use worldwide by 2020 – impressive numbers that should catch the attention of every CIO. IT leaders in nearly every vertical market will soon be inundated with the management of both the data from these devices as well as the management of the devices themselves, each of which require the same lifecycle management as any other IT equipment. This can be an overwhelming realization for CIOs who don’t have an adequate configuration management strategy for their current IT environments, the foundation upon which all future digital strategies – Internet-connected or otherwise – will be built ...

March 13, 2018

Many network operations teams question if they need to TAP their networks; perhaps they aren't familiar with test access points (TAPs), or they think there isn't an application that makes sense for them. Over the past decade, industry best-practice revealed that all network infrastructure should utilize a network TAP as the foundation for complete visibility. The following are the seven most popular applications for TAPs ...

March 12, 2018

Organizations are eager to adopt cloud based architectures in an effort to support their digital transformation efforts, drive efficiencies and strengthen customer satisfaction, according to a new online cloud usage survey conducted by Denodo ...

March 09, 2018

Globally, cloud data center traffic will represent 95 percent of total data center traffic by 2021, compared to 88 percent in 2016, according to the Cisco Global Cloud Index (2016-2021) ...

March 08, 2018

Enterprise cloud spending will grow rapidly over the next year, and yet 35 percent of cloud spend is wasted, according to The RightScale 2018 State of the Cloud Survey ...

March 07, 2018

What often goes overlooked in our always-on digital culture are the people at the other end of each of these services tasked with their 24/7 management. If something goes wrong, users are quick to complain or switch to a competitor as IT practitioners on the backend race to rectify the situation. A recent PagerDuty State of IT Work-Life Balance Report revealed that IT professionals are struggling with the pressures associated with the management of these digital offerings ...

March 06, 2018

Businesses everywhere continually strive for greater efficiency. By way of illustration, more than a third of IT professionals cite "moving faster" as their top goal for 2018, and improving the efficiency of operations was one of the top three stated business objectives for organizations considering digital transformation initiatives ...

March 05, 2018

One of the current challenges for IT teams is the movement of the network to the cloud, and the lack of visibility that comes with that shift. While there has been a lot of hype around the benefits of cloud computing, very little is being said about the inherent drawbacks ...