How AIOps Defuses the Impact of Change
July 12, 2021

Phil Tee

Share this

When you see distressing internet outages occur like the recent Fastly incident that threw a slew of websites offline, I am never surprised by how widespread the problem was, but paradoxically that it wasn't worse.

The infrastructure behind our digital world is mind-numbingly complex. The movement to cloud computing has added even more layers to the interconnectedness. So when a simple software update goes awry, despite the best efforts of quality control, the ripple effects can go far and wide. The digital economy in the US alone accounts for at least $1,849 billion annually, according to a 2020 report by the Bureau of Economic Analysis. So every moment offline matters.

Prompt troubleshooting is a herculean task — impossible, really, for the human mind alone. There's just too much information to sift through to quickly identify how a single change event precipitated such a widespread crash. IT teams must rely on artificial intelligence, machine learning and algorithms to find and repair the root cause of the problem.

The Perils of "Change"

What seems near effortless online to most of us — ordering food, a Zoom call, reading this article — is a staggeringly Byzantine interconnected flow of data packets, routers, modems, internet service providers, gateways, network exchanges, servers and applications. The interdependencies are at such a level that any meaningful amount of mappability is out of reach. For a human mind, you're talking about understanding more interdependencies than particles in the observable universe — a stunning amount of complexity.

Amid that landscape is the need to update software, whether to refresh the operating system, add features or bolster security. And from time to time, someone performs a routine update that has an unintended and unforeseen consequence. Identifying a problem before an outage occurs is largely a fool's errand because the scale of the situation is just too great. The key is to find the problem before a widespread outage occurs. In such an interconnected digital world, errors tend to cascade and propagate. Catching them early is paramount.

One simple update that goes awry could cripple e-commerce if widespread system outages lingered. The potential risk is profound. History has shown when unintended consequences snowball. Mexico reeled in the 1990s from the devaluation of the peso. The United States stumbled in the 2000s when collateralized debt obligations tied to the mortgage industry prompted a financial crisis.

To be clear, the Fastly incident wasn't a global crisis. The Fastly team responded remarkably well. But the outage underscored how trouble quickly can spread in the interconnected digital world. What's absolutely necessary is to pinpoint the problem immediately.

How Intelligent Observability Defuses the Threat

This is where intelligent observability comes in to analyze the impact of change. AIOps with observability work together to quickly spot the patterns and interconnections in the application data to identify the root cause of a problem before it cascades further and causes a widespread outage.

Every change, every software update, has some kind of record associated with it. So theoretically, when something goes wrong, a site reliability engineer or other IT expert would get an alert in which they could simply trace the issue back to the record of the change that triggered the issue. But in practice, the situation is very complicated. Thousands of other data points were created before and after this specific change occurred, so the challenge to identifying the root cause of the problem is linking the right data to the relevant change.

AIOps finds the right data. It applies algorithms to observability data such as metrics, logs and traces to identify anomalies, determine event significance, surface meaningful alerts and correlate data to provide valuable context. Observability makes the job easier by engineering the application infrastructure to make all of the data more observable. AIOps surfaces the right data amid an ocean of data so your IT teams can quickly spot and repair the problem.

Every change, every software update, leaves a clue behind. The problem is there are thousands and thousands of potential suspects. Intelligent observability can quickly solve the "whatdunnit" before any outage becomes much worse.

Phil Tee is CEO of Moogsoft
Share this

The Latest

October 21, 2021

Scaling DevOps and SRE practices is critical to accelerating the release of high-quality digital services. However, siloed teams, manual approaches, and increasingly complex tooling slow innovation and make teams more reactive than proactive, impeding their ability to drive value for the business, according to a new report from Dynatrace, Deep Cloud Observability and Advanced AIOps are Key to Scaling DevOps Practices ...

October 20, 2021

Over three quarters (79%) of database professionals are now using either a paid-for or in-house monitoring tool, according to a new survey from Redgate Software ...

October 19, 2021

Gartner announced the top strategic technology trends that organizations need to explore in 2022. With CEOs and Boards striving to find growth through direct digital connections with customers, CIOs' priorities must reflect the same business imperatives, which run through each of Gartner's top strategic tech trends for 2022 ...

October 18, 2021

Distributed tracing has been growing in popularity as a primary tool for investigating performance issues in microservices systems. Our recent DevOps Pulse survey shows a 38% increase year-over-year in organizations' tracing use. Furthermore, 64% of those respondents who are not yet using tracing indicated plans to adopt it in the next two years ...

October 14, 2021

Businesses are embracing artificial intelligence (AI) technologies to improve network performance and security, according to a new State of AIOps Study, conducted by ZK Research and Masergy ...

October 13, 2021

What may have appeared to be a stopgap solution in the spring of 2020 is now clearly our new workplace reality: It's impossible to walk back so many of the developments in workflow we've seen since then. The question is no longer when we'll all get back to the office, but how the companies that are lagging in their technological ability to facilitate remote work can catch up ...

October 12, 2021

The pandemic accelerated organizations' journey to the cloud to enable agile, on-demand, flexible access to resources, helping them align with a digital business's dynamic needs. We heard from many of our customers at the start of lockdown last year, saying they had to shift to a remote work environment, seemingly overnight, and this effort was heavily cloud-reliant. However, blindly forging ahead can backfire ...

October 07, 2021

SmartBear recently released the results of its 2021 State of Software Quality | Testing survey. I doubt you'll be surprised to hear that a "lack of time" was reported as the number one challenge to doing more testing, especially as release frequencies continue to increase. However, it was disheartening to see that a lack of time was also the number one response when we asked people to identify the biggest blocker to professional development ...

October 06, 2021

The role of the CIO is evolving with an increased focus on unlocking customer connections through service innovation, according to the 2021 Global CIO Survey. The study reveals the shift in the role of the CIO with the majority of CIO respondents stating innovation, operational efficiency, and customer experience as their top priorities ...

October 05, 2021

The perception of IT support has dramatically improved thanks to the successful response of service desks to the pandemic, lockdowns and working from home, according to new research from the Service Desk Institute (SDI), sponsored by Sunrise Software ...