Performance Testing and APM: Why You Need Both
October 23, 2015

Paola Moretto
Nouvola

Share this

Performance Testing (aka synthetic testing) and APM (Application Performance Management and monitoring solutions) are often regarded as two competing solutions, as if one were an alternative to the other. They are actually perfect complementary tools to achieve optimal performance for your application. Whether you develop web applications, SaaS products or mobile apps, you‘ll find both approaches to be absolutely necessary in your software operations. Here, I explain why. (I’ll use “monitoring solutions” as a synonym for APM solutions.)

Let’s talk about performance for a moment. Performance can refer to end-user performance or backend performance.

End-user performance is a measurement of the user experience as relates to speed, responsiveness and scalability. Page load time is one of the typical metrics for the user experience.

Backend performance, as the name suggests, refers to the performance of backend resources used by the application. This is something end users may not be aware of, but which is important to the way the app performs.

Let’s look at APM or monitoring solutions first. These tools are normally used to collect data from a live environment. Monitoring solutions help gather important information on the application backend metrics, server behaviors, slow components and transactions. Several application monitoring solutions track database and browser performance as well.

Monitoring is Not Enough

There are many kinds of monitoring solutions that operate at different levels of the stack, with different timing granularity, with the opportunity to define custom metrics. You have server monitoring, infrastructure monitoring, user monitoring, high-frequency metrics, and others. What is clear with monitoring solutions is that one is not enough. You probably need a portfolio of tools to have a clear understanding of what is happening with your application. Most tools have alerts systems that provide real-time information on page load and are designed to notify you in real time of events so you are immediately informed about deteriorating performance.

It doesn’t matter how comprehensive your monitoring architecture is, it doesn’t provide the full picture. First of all, live traffic is noisy. You have no control over what your users are doing and how many users you have at any given point. If you have a performance issue, that makes it really hard to troubleshoot. Was that expected behavior? Did they just hit a corner case previously unanticipated? Is a combination of the live workloads that crashed the system? So even though the technology is designed to enable real-time troubleshooting, the reality is that since it’s not a controlled environment, you might not be able to identify root cause performance issues in a timely manner.

Second and most important, the information produced by monitoring is delivered after the fact. Monitoring is like calling AAA after an accident. It’s a great service to have, but it’s much better to prevent the accident in the first place.

This explains why you need to add performance testing to the mix. While monitoring can inform you about performance after the fact, performance testing can help you prevent bad things from happening.

While monitoring is usually done on your live/production environment, performance testing usually utilizes synthetic traffic on a pre-production/staging environment. Having a pre-production environment as close as possible to your production environment will help you derive the most meaningful results.

In performance testing, users are simulated but the traffic is absolutely real. You can apply different types of loads to discover the breaking point of your application before it goes live. You can use performance testing to test with traffic that is higher than anything your actual application has seen – yet – so that you can prepare for peak of traffic.

Performance testing can also help you identify performance degradations that might have resulted from code changes, infrastructure changes or third party changes. It basically answers the question: “Can you trust this build to deliver the same user experience your users are counting on?”

In performance testing, you have total control over the amount of traffic and the workloads your users execute. That makes it a lot easier to troubleshoot.

And, if you are using Docker or containers-based architecture, you could also test performance improvements, under different configurations and platforms easily.

With performance testing, you can also measure end-to-end performance – a good indication of user experience -- which gives visibility into the entire application delivery chain, enabling greater transparency and targeted troubleshooting.

You wouldn’t build a bridge and then send hundreds of thousands of cars over it without first doing structural tests for structural problems. But you also wouldn’t open a bridge to traffic without continually monitoring how it is holding up under all the traffic. You need to do both, whether you’re talking about bridges or apps. The difference is, while a bridge is static and needs to be tested only once or at periodic intervals, software today is highly dynamic and needs to be tested on a daily basis as part of your regular flow.

Used together, performance testing and monitoring make a great team, so to speak. Use both to make sure you deploy a reliable product.

Paola Moretto is Founder and CEO of Nouvola.

Share this

The Latest

July 22, 2019

Many organizations are unsure where to begin with AIOps, but should seriously consider adopting an AIOps strategy and solution. To get started, it's important to identify the key capabilities of AIOps that are needed to realize maximum value from your investments ...

July 18, 2019

Organizations that are working with artificial intelligence (AI) or machine learning (ML) have, on average, four AI/ML projects in place, according to a recent survey by Gartner, Inc. Of all respondents, 59% said they have AI deployed today ...

July 17, 2019

The 11th anniversary of the Apple App Store frames a momentous time period in how we interact with each other and the services upon which we have come to rely. Even so, we continue to have our in-app mobile experiences marred by poor performance and instability. Apple has done little to help, and other tools provide little to no visibility and benchmarks on which to prioritize our efforts outside of crashes ...

July 16, 2019

Confidence in artificial intelligence (AI) and its ability to enhance network operations is high, but only if the issue of bias is tackled. Service providers (68%) are most concerned about the bias impact of "bad or incomplete data sets," since effective AI requires clean, high quality, unbiased data, according to a new survey of communication service providers ...

July 15, 2019

Every internet connected network needs a visibility platform for traffic monitoring, information security and infrastructure security. To accomplish this, most enterprise networks utilize from four to seven specialized tools on network links in order to monitor, capture and analyze traffic. Connecting tools to live links with TAPs allow network managers to safely see, analyze and protect traffic without compromising network reliability. However, like most networking equipment it's critical that installation and configuration are done properly ...

July 11, 2019

The Democratic presidential debates are likely to have many people switching back-and-forth between live streams over the coming months. This is going to be especially true in the days before and after each debate, which will mean many office networks are likely to see a greater share of their total capacity going to streaming news services than ever before ...

July 10, 2019

Monitoring of heating, ventilation and air conditioning (HVAC) infrastructures has become a key concern over the last several years. Modern versions of these systems need continual monitoring to stay energy efficient and deliver satisfactory comfort to building occupants. This is because there are a large number of environmental sensors and motorized control systems within HVAC systems. Proper monitoring helps maintain a consistent temperature to reduce energy and maintenance costs for this type of infrastructure ...

July 09, 2019

Shoppers won’t wait for retailers, according to a new research report titled, 2019 Retailer Website Performance Evaluation: Are Retail Websites Meeting Shopper Expectations? from Yottaa ...

June 27, 2019

Customer satisfaction and retention were the top concerns for a majority (58%) of IT leaders when suffering downtime or outages, according to a survey of top IT leaders conducted by AIOps Exchange. The effect of service interruptions on customers outweighed other concerns such as loss of revenue, brand reputation, negative press coverage, or the impact on IT Ops teams.

June 26, 2019

It is inevitable that employee productivity and the quality of customer experiences suffer as a consequence of the poor performance of O365. The quick detection and rapid resolution of problems associated with O365 are top of mind for any organization to keep its business humming ...