The Application Apocalypse - Rethink Performance or Prepare to Meet Thy Doom
September 01, 2015

Pete Waterhouse
CA Technologies

Share this

With the inevitable zombie apocalypse, having the right strategies to combat the plague will be essential. Turns out that trouble-shooting application performance isn’t much different. As any good zombie fighter will tell you, in a pandemic that threatens to consume all humanity, it’ll be important to find the first person infected – called “patient zero”. Knowing that sucker's history can help determine how and when the infection started, and with a bit of luck, a way to stop it.

You might scoff, but there are many parallels between this and the way we manage application performance. Ok, perhaps not on a World War Z scale, but still troublesome enough to bite your business where it hurts most.

Heroic Best Effort Hurts People and the Business

In the aforementioned movie, the hero risked everything to lead a worldwide search for the source of the plague. In IT we have our own heroes -- administrators who spend lots of time scrambling from console to console hoping to find the root cause of the latest event storm. Guys who puts everything aside, getting called in after hours to try and figure out why customers suddenly can’t check-out goods with that new mobile app – yes, the one those wretched code junkies just threw over the wall.

It’s not that our heroes doesn’t have the tools for the job. On the contrary, they’re armed with enough products, dashboards and diagnostic wizardry to slay an army of performance problems. They are our champions, veritable experts in configuring systems, the uber nerds who can sift through logs and data; the undisputed king gods of monitoring the monitors.

Yet problems still persist, customer experience degrades and business suffers, Oh, and our valued APM admin just missed his wedding anniversary after an all-weekend triage binge – been there, done that.

So how do we end this vicious cycle?

It requires new approaches to monitoring; rethinking the traditional ways we go about finding our own “patient zeros” – changes - and quickly understanding how they impact performance. When IT can detect changes quickly they no longer need a “shoot them in the head” approach to managing performance – that might work with one or two zombies, but it no longer scales in business.

Constant Change is a Both a Blessing and a Curse

Forget the hype – the real reason businesses must employ mobile and cloud is the opportunity to engage with customers at scale. This, however, precipitates the need for faster development of high-quality software. Software that changes not for change sake, but in order to quickly test assumptions, build new markets, and create and solve problems customers didn’t know they had. Yet these changes are a curse on IT operations.

Call me captain obvious, but the vast majority of application performance problems are caused by some kind of change. Again, it’s not that we don’t have enough tools. We have truck load full of products to detect changes in code, infrastructure and configuration screw ups. The problem is we lack context – or the link and association between performance behavior and changes. And because changes now happen more frequently (and cumulatively), it’s left in the hands of heroes to painstakingly sift through the wreckage to determine the root cause. That requires expertise yes, but more often gut feel and dumb luck. Even a vision impaired squirrel will sometimes find a nut, right?

Lessons Learned from Air Crash Investigation

If you want to see the best example of effective root cause analysis watch the TV show Air Emergency. You’ll see the importance of cockpit voice and flight data recorders. Not only do they gather all the data leading up to an incident, they’re also the focal point around which experts in many fields (across metallurgy, weather, instrumentation, even psychology) coalesce to gain a true picture of what caused the problem. Without this there’ll blame games, conjecture – even whacky conspiracy theories.

In IT we need similar capabilities that understand changes over time and provide the ability to dive into any changes that precede a performance issue. If you will, a system that provides an integrated change/performance timeline which lets our own teams of experts see the chronological order of change events and navigates the event series to the impact of change related to application elements. This is invaluable in a production context, but also in pre-production. Because by identifying the beginning of a performance issues (perhaps before a new code release) and understanding change related elements, teams can quickly gain agreement on root cause. Then through DevOps style feedback and collaboration they can work to prevent issues reoccurring.

Modern APM solutions are now employing these approaches, so cross-functional teams can quickly pinpoint what and where changes occurred and correlate to any resulting performance impact. They find that “patient zero” quickly and often without involving any other team member.

And without this type of functionality? Well, there’ll be blame games, conjecture (“I think its Frank’s lousy code”), and yes, maybe even conspiracy theories.

The modern digital business understands that loyalty is only ever earned by providing a rich, high quality customer experience. Supporting this, our IT staff don’t need more tools, just better methods that apply common sense thinking to address problems that continue to plague operations, hurt staff and bite the business.

Now if you’ll excuse me, I must get back to my command center, turn on my monitors – and wait for the zombie hoards.

Pete Waterhouse is Senior Strategist at CA Technologies
Share this

The Latest

December 05, 2023

Industry experts offer thoughtful, insightful, and often controversial predictions on how APM, AIOps, Observability, OpenTelemetry and related technologies will evolve and impact business in 2024. Part 2 covers more on Observability ...

December 04, 2023

The Holiday Season means it is time for APMdigest's annual list of Application Performance Management (APM) predictions, covering IT performance topics. Industry experts — from analysts and consultants to the top vendors — offer thoughtful, insightful, and often controversial predictions on how APM, observability, AIOps and related technologies will evolve and impact business in 2024. Part 1 covers APM and Observability ...

November 30, 2023

To help you stay on top of the ever-evolving tech scene, Automox IT experts shake the proverbial magic eight ball and share their predictions about tech trends in the coming year. From M&A frenzies to sustainable tech and automation, these forecasts paint an exciting picture of the future ...

November 29, 2023
The past few years have presented numerous challenges for businesses: a pandemic, rising interest rates, supply chain disruptions, and geopolitical conflict that sent shockwaves across the global economy. But change may finally be on the horizon. According to a recent report by Endava ... a majority of executives confirmed they are feeling optimistic about the current business climate, and as a result, are forecasting larger IT budgets, increased technology funding and rollout, and prioritized innovation in the coming year ...
November 28, 2023

Incident management processes are not keeping pace with the demands of modern operations teams, failing to meet the needs of SREs as well as platform and ops teams. Results from the State of DevOps Automation and AI Survey, commissioned by Transposit, point to an incident management paradox. Despite nearly 60% of ITOps and DevOps professionals reporting they have a defined incident management process that's fully documented in one place and over 70% saying they have a level of automation that meets their needs, teams are unable to quickly resolve incidents ...

November 27, 2023

Today, in the world of enterprise technology, the challenges posed by legacy Virtual Desktop Infrastructure (VDI) systems have long been a source of concern for IT departments. In many instances, this promising solution has become an organizational burden, hindering progress, depleting resources, and taking a psychological and operational toll on employees ...

November 22, 2023

Within retail organizations across the world, IT teams will be bracing themselves for a hectic holiday season ... While this is an exciting opportunity for retailers to boost sales, it also intensifies severe risk. Any application performance slipup will cause consumers to turn their back on brands, possibly forever. Online shoppers will be completely unforgiving to any retailer who doesn't deliver a seamless digital experience ...

November 21, 2023

Black Friday is a time when consumers can cash in on some of the biggest deals retailers offer all year long ... Nearly two-thirds of consumers utilize a retailer's web and mobile app for holiday shopping, raising the stakes for competitors to provide the best online experience to retain customer loyalty. Perforce's 2023 Black Friday survey sheds light on consumers' expectations this time of year and how developers can properly prepare their applications for increased online traffic ...

November 20, 2023

This holiday shopping season, the stakes for online retailers couldn't be higher ... Even an hour or two of downtime for a digital storefront during this critical period can cost millions in lost revenue and has the potential to damage brand credibility. Savvy retailers are increasingly investing in observability to help ensure a seamless, omnichannel customer experience. Just ahead of the holiday season, New Relic released its State of Observability for Retail report, which offers insight and analysis on the adoption and business value of observability for the global retail/consumer industry ...

November 16, 2023

As organizations struggle to find and retain the talent they need to manage complex cloud implementations, many are leaning toward hybrid cloud as a solution ... While it's true that using the cloud is not a "one size fits all" proposition, it is clear that both large and small companies prefer a hybrid cloud model ...