Performance Testing and APM: Why You Need Both
October 23, 2015

Paola Moretto
Nouvola

Share this

Performance Testing (aka synthetic testing) and APM (Application Performance Management and monitoring solutions) are often regarded as two competing solutions, as if one were an alternative to the other. They are actually perfect complementary tools to achieve optimal performance for your application. Whether you develop web applications, SaaS products or mobile apps, you‘ll find both approaches to be absolutely necessary in your software operations. Here, I explain why. (I’ll use “monitoring solutions” as a synonym for APM solutions.)

Let’s talk about performance for a moment. Performance can refer to end-user performance or backend performance.

End-user performance is a measurement of the user experience as relates to speed, responsiveness and scalability. Page load time is one of the typical metrics for the user experience.

Backend performance, as the name suggests, refers to the performance of backend resources used by the application. This is something end users may not be aware of, but which is important to the way the app performs.

Let’s look at APM or monitoring solutions first. These tools are normally used to collect data from a live environment. Monitoring solutions help gather important information on the application backend metrics, server behaviors, slow components and transactions. Several application monitoring solutions track database and browser performance as well.

Monitoring is Not Enough

There are many kinds of monitoring solutions that operate at different levels of the stack, with different timing granularity, with the opportunity to define custom metrics. You have server monitoring, infrastructure monitoring, user monitoring, high-frequency metrics, and others. What is clear with monitoring solutions is that one is not enough. You probably need a portfolio of tools to have a clear understanding of what is happening with your application. Most tools have alerts systems that provide real-time information on page load and are designed to notify you in real time of events so you are immediately informed about deteriorating performance.

It doesn’t matter how comprehensive your monitoring architecture is, it doesn’t provide the full picture. First of all, live traffic is noisy. You have no control over what your users are doing and how many users you have at any given point. If you have a performance issue, that makes it really hard to troubleshoot. Was that expected behavior? Did they just hit a corner case previously unanticipated? Is a combination of the live workloads that crashed the system? So even though the technology is designed to enable real-time troubleshooting, the reality is that since it’s not a controlled environment, you might not be able to identify root cause performance issues in a timely manner.

Second and most important, the information produced by monitoring is delivered after the fact. Monitoring is like calling AAA after an accident. It’s a great service to have, but it’s much better to prevent the accident in the first place.

This explains why you need to add performance testing to the mix. While monitoring can inform you about performance after the fact, performance testing can help you prevent bad things from happening.

While monitoring is usually done on your live/production environment, performance testing usually utilizes synthetic traffic on a pre-production/staging environment. Having a pre-production environment as close as possible to your production environment will help you derive the most meaningful results.

In performance testing, users are simulated but the traffic is absolutely real. You can apply different types of loads to discover the breaking point of your application before it goes live. You can use performance testing to test with traffic that is higher than anything your actual application has seen – yet – so that you can prepare for peak of traffic.

Performance testing can also help you identify performance degradations that might have resulted from code changes, infrastructure changes or third party changes. It basically answers the question: “Can you trust this build to deliver the same user experience your users are counting on?”

In performance testing, you have total control over the amount of traffic and the workloads your users execute. That makes it a lot easier to troubleshoot.

And, if you are using Docker or containers-based architecture, you could also test performance improvements, under different configurations and platforms easily.

With performance testing, you can also measure end-to-end performance – a good indication of user experience -- which gives visibility into the entire application delivery chain, enabling greater transparency and targeted troubleshooting.

You wouldn’t build a bridge and then send hundreds of thousands of cars over it without first doing structural tests for structural problems. But you also wouldn’t open a bridge to traffic without continually monitoring how it is holding up under all the traffic. You need to do both, whether you’re talking about bridges or apps. The difference is, while a bridge is static and needs to be tested only once or at periodic intervals, software today is highly dynamic and needs to be tested on a daily basis as part of your regular flow.

Used together, performance testing and monitoring make a great team, so to speak. Use both to make sure you deploy a reliable product.

Paola Moretto is Founder and CEO of Nouvola.

Share this

The Latest

May 21, 2020

As cloud computing continues to grow, tech pros say they are increasingly prioritizing areas like hybrid infrastructure management, application performance management (APM), and security management to optimize delivery for the organizations they serve, according to ...

May 20, 2020

Businesses see digital experience as a growing priority and a key to their success, with execution requiring a more integrated approach across development, IT and business users, according to Digital Experiences: Where the Industry Stands ...

May 19, 2020

Fully 90% of those who use observability tooling say those tools are important to their team's software development success, including 39% who say observability tools are very important ...

May 18, 2020

As our production application systems continuously increase in complexity, the challenges of understanding, debugging, and improving them keep growing by orders of magnitude. The practice of Observability addresses both the social and the technological challenges of wrangling complexity and working toward achieving production excellence. New research shows how observable systems and practices are changing the APM landscape ...

May 14, 2020
Digital technologies have enveloped our lives like never before. Be it on the personal or professional front, we have become dependent on the accurate functioning of digital devices and the software running them. The performance of the software is critical in running the components and levers of the new digital ecosystem. And to ensure our digital ecosystem delivers the required outcomes, a robust performance testing strategy should be instituted ...
May 13, 2020

The enforced change to working from home (WFH) has had a massive impact on businesses, not just in the way they manage their employees and IT systems. As the COVID-19 pandemic progresses, enterprise IT teams are looking to answer key questions such as: Which applications have become more critical for working from home? ...

May 12, 2020

In ancient times — February 2020 — EMA research found that more than 50% of IT leaders surveyed were considering new ITSM platforms in the near future. The future arrived with a bang as IT organizations turbo-pivoted to deliver and support unprecedented levels and types of services to a global workplace suddenly working from home ...

May 11, 2020

The Internet of Things (IoT) is changing the world. From augmented reality advanced analytics to new consumer solutions, IoT and the cloud are together redefining both how we work and how we engage with our audiences. They are changing how we live, as well ...

May 07, 2020

Despite IT professionals' confidence in their ability to support today's much greater dependence on digital services, there is a rise in application performance errors reported by more than half of consumers, according to the Impact of COVID-19 on Digital Transformation survey from xMatters ...

May 06, 2020

The new normal includes not only periodic recurrences of Covid-19 outbreaks but also the periodic emergence of new global pandemics. This means putting in place at least three layers of digital business continuity practice ...