It's Time to Modernize Pre-Deployment Testing
August 31, 2021

Jeff Atkins
Spirent

Share this

Here's how it happens: You're deploying a new technology, thinking everything's going smoothly, when the alerts start coming in. Your rollout has hit a snag. Whole groups of users are complaining about poor performance on their devices. Some can't access applications at all. You've now blown your service-level agreement (SLA). You might have just introduced a new security vulnerability. In the worst case, your big expensive product launch has missed the mark altogether.

"How did this happen?" you're asking yourself. "Didn't we test everything before we deployed?"

Yes, you did. But you made a critical though common mistake: your tests assumed ideal network conditions. And as you just learned firsthand, the idealized environment in your testing models and the way things work in the real world are two very different things.

Hopefully, this hypothetical doesn't sound too familiar. But if you're relying on traditional testing workflows and you've managed to avoid these kinds of outcomes so far, count your blessings. Because you're taking a big risk with every new launch.

There's a better way to test new enterprise technologies so they get deployed on time, under budget, with the performance you expect. To do it though, you need to get better at predicting the future. That starts with painting a more accurate picture of the present.

Navigating Complexity

Modern IT organizations already deal with more devices, more connections, and complexity than ever before. But even if you get a handle on today's technology landscape, new innovations emerge all the time. Next-generation Ethernet technologies, 5G networks, SD-WAN, Wi-Fi 6, and others can all bring important benefits to your users — benefits your competitors may already be realizing, that you can't afford to ignore. Yet, each new deployment carries significant unpredictability and risk.

All of this means it's more critical than ever to thoroughly test and validate new technology before you deploy. But all the testing in the world can't help you if you're not testing the right things. And the fact is, next-generation enterprise technologies are evolving too quickly for legacy testing approaches to keep up.

In too many cases, enterprises still test new applications and infrastructure by connecting devices directly to datacenters or clouds, with little or no traffic on the network. That kind of testing can tell you how the technology works under ideal conditions, but how often can you expect ideal conditions in the real world?

How will the technology perform on a congested or impaired network?

What kinds of problems will have the biggest impact on user experience?

Too often, those questions get answered only after deployment, when users complain. At which point customer satisfaction has already taken a hit, you may have missed an SLA, and you're looking at a time-consuming, expensive repair process.

Even more concerning, security often gets less attention than performance in pre-deployment validation. Many enterprises still rely on basic tools and firmware checks, or even just assurances from vendors, that software is safe to deploy. Which means there's a good chance you'll only learn about a vulnerability after it's been exploited, and your systems are already compromised.

A Smarter Approach

Fortunately, it's possible to predict and avoid most of these issues. To do it though, we need to recognize that testing models that worked a decade ago won't cut it anymore. We need to reimagine pre-deployment testing for today's more complex, dynamic, and distributed world.

Whatever your updated testing methodology looks like, it should include the following core practices:

Performance validation: Your vendors aren't lying when they claim to hit certain benchmarks, but you can't assume you'll achieve comparable performance in your own environment—especially if you'll be operating under an SLA. You should be measuring everything from voice quality to packet jitter. By validating real-world performance across more granular metrics, you can better evaluate any new solutions you're considering. At the same time, you identify everything you'll need to understand the user experience and troubleshoot problems post-deployment.

Network emulation: If you're going to deploy with confidence, you want to get your test beds as close as possible to real-world conditions. That includes mimicking networks, devices, and users under heavy traffic loads.

Network impairment: Network faults and service degradations are an unavoidable (if hopefully infrequent) reality. So, wouldn't you prefer to know how a new technology will respond under those conditions ahead of time? By running controlled network impairment scenarios alongside emulation, you'll know exactly how problems will affect your users, so you can better prepare. Even more important, you can set realistic expectations with customers and achievable SLAs.

Security assessments: Don't bet your security on third-party assurances or basic firmware checks. Take the time to thoroughly test for vulnerabilities, simulate known attacks, and evaluate weaknesses in the end-to-end network.

Testbed automation: To keep pace with rapidly changing networks and clouds, you should look to automate as much of the testing process as possible. The less you rely on slow, manual testing methodologies, the more quickly and cost-effectively you'll be able to simulate new scenarios as your environment evolves.

Proactive Testing Makes All the Difference

So, what happens when you put these principles into practice — when you modernize your testing to reflect a more realistic picture of your technology landscape?

First, you save time and money by identifying problems before deploying instead of after. It's a lot harder and more expensive to fix issues with a new technology when diverse users and systems already rely on it, and SLAs are already violated.

Second, you protect your users and your business by detecting and mitigating security vulnerabilities before malicious actors can exploit them. Finally, you improve your organization's ability to take advantage of new technology. By automating the testing process, you can continually bring in new testing practices and collect more valuable insights without slowing down innovation.

By overhauling your testing strategy based on realism and automation, you can put your organization in the best position to capitalize on new technologies when they emerge. You can reduce the risk of disruptive (and expensive) problems cropping up out of the blue. And, you can make ongoing innovation a core strength of your IT organization — and a key competitive advantage for your business.

Jeff Atkins is Director of Solutions Marketing at Spirent
Share this

The Latest

March 04, 2024

This year's Super Bowl drew in viewership of nearly 124 million viewers and made history as the most-watched live broadcast event since the 1969 moon landing. To support this spike in viewership, streaming companies like YouTube TV, Hulu and Paramount+ began preparing their IT infrastructure months in advance to ensure an exceptional viewer experience without outages or major interruptions. New Relic conducted a survey to understand the importance of a seamless viewing experience and the impact of outages during major streaming events such as the Super Bowl ...

March 01, 2024

As organizations continue to navigate the complexities of the digital era, which has been marked by exponential advancements in AI and technology, the strategic deployment of modern, practical applications has become indispensable for sustaining competitive advantage and realizing business goals. The Info-Tech Research Group report, Applications Priorities 2024, explores the following five initiatives for emerging and leading-edge technologies and practices that can enable IT and applications leaders to optimize their application portfolio and improve on capabilities needed to meet the ambitions of their organizations ...

February 29, 2024

Despite the growth in popularity of artificial intelligence (AI) and ML across a number of industries, there is still a huge amount of unrealized potential, with many businesses playing catch-up and still planning how ML solutions can best facilitate processes. Further progression could be limited without investment in specialized technical teams to drive development and integration ...

February 28, 2024

With over 200 streaming services to choose from, including multiple platforms featuring similar types of entertainment, users have little incentive to remain loyal to any given platform if it exhibits performance issues. Big names in streaming like Hulu, Amazon Prime and HBO Max invest thousands of hours into engineering observability and closed-loop monitoring to combat infrastructure and application issues, but smaller platforms struggle to remain competitive without access to the same resources ...

February 27, 2024

Generative AI has recently experienced unprecedented dramatic growth, making it one of the most exciting transformations the tech industry has seen in some time. However, this growth also poses a challenge for tech leaders who will be expected to deliver on the promise of new technology. In 2024, delivering tangible outcomes that meet the potential of AI, and setting up incubator projects for the future will be key tasks ...

February 26, 2024

SAP is a tool for automating business processes. Managing SAP solutions, especially with the shift to the cloud-based S/4HANA platform, can be intricate. To explore the concerns of SAP users during operational transformations and automation, a survey was conducted in mid-2023 by Digitate and Americas' SAP Users' Group ...

February 22, 2024

Some companies are just starting to dip their toes into developing AI capabilities, while (few) others can claim they have built a truly AI-first product. Regardless of where a company is on the AI journey, leaders must understand what it means to build every aspect of their product with AI in mind ...

February 21, 2024

Generative AI will usher in advantages within various industries. However, the technology is still nascent, and according to the recent Dynatrace survey there are many challenges and risks that organizations need to overcome to use this technology effectively ...

February 20, 2024

In today's digital era, monitoring and observability are indispensable in software and application development. Their efficacy lies in empowering developers to swiftly identify and address issues, enhance performance, and deliver flawless user experiences. Achieving these objectives requires meticulous planning, strategic implementation, and consistent ongoing maintenance. In this blog, we're sharing our five best practices to fortify your approach to application performance monitoring (APM) and observability ...

February 16, 2024

In MEAN TIME TO INSIGHT Episode 3, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at Enterprise Management Associates (EMA) discusses network security with Chris Steffen, VP of Research Covering Information Security, Risk, and Compliance Management at EMA ...