When you see distressing internet outages occur like the recent Fastly incident that threw a slew of websites offline, I am never surprised by how widespread the problem was, but paradoxically that it wasn't worse.
The infrastructure behind our digital world is mind-numbingly complex. The movement to cloud computing has added even more layers to the interconnectedness. So when a simple software update goes awry, despite the best efforts of quality control, the ripple effects can go far and wide. The digital economy in the US alone accounts for at least $1,849 billion annually, according to a 2020 report by the Bureau of Economic Analysis. So every moment offline matters.
Prompt troubleshooting is a herculean task — impossible, really, for the human mind alone. There's just too much information to sift through to quickly identify how a single change event precipitated such a widespread crash. IT teams must rely on artificial intelligence, machine learning and algorithms to find and repair the root cause of the problem.
The Perils of "Change"
What seems near effortless online to most of us — ordering food, a Zoom call, reading this article — is a staggeringly Byzantine interconnected flow of data packets, routers, modems, internet service providers, gateways, network exchanges, servers and applications. The interdependencies are at such a level that any meaningful amount of mappability is out of reach. For a human mind, you're talking about understanding more interdependencies than particles in the observable universe — a stunning amount of complexity.
Amid that landscape is the need to update software, whether to refresh the operating system, add features or bolster security. And from time to time, someone performs a routine update that has an unintended and unforeseen consequence. Identifying a problem before an outage occurs is largely a fool's errand because the scale of the situation is just too great. The key is to find the problem before a widespread outage occurs. In such an interconnected digital world, errors tend to cascade and propagate. Catching them early is paramount.
One simple update that goes awry could cripple e-commerce if widespread system outages lingered. The potential risk is profound. History has shown when unintended consequences snowball. Mexico reeled in the 1990s from the devaluation of the peso. The United States stumbled in the 2000s when collateralized debt obligations tied to the mortgage industry prompted a financial crisis.
To be clear, the Fastly incident wasn't a global crisis. The Fastly team responded remarkably well. But the outage underscored how trouble quickly can spread in the interconnected digital world. What's absolutely necessary is to pinpoint the problem immediately.
How Intelligent Observability Defuses the Threat
This is where intelligent observability comes in to analyze the impact of change. AIOps with observability work together to quickly spot the patterns and interconnections in the application data to identify the root cause of a problem before it cascades further and causes a widespread outage.
Every change, every software update, has some kind of record associated with it. So theoretically, when something goes wrong, a site reliability engineer or other IT expert would get an alert in which they could simply trace the issue back to the record of the change that triggered the issue. But in practice, the situation is very complicated. Thousands of other data points were created before and after this specific change occurred, so the challenge to identifying the root cause of the problem is linking the right data to the relevant change.
AIOps finds the right data. It applies algorithms to observability data such as metrics, logs and traces to identify anomalies, determine event significance, surface meaningful alerts and correlate data to provide valuable context. Observability makes the job easier by engineering the application infrastructure to make all of the data more observable. AIOps surfaces the right data amid an ocean of data so your IT teams can quickly spot and repair the problem.
Every change, every software update, leaves a clue behind. The problem is there are thousands and thousands of potential suspects. Intelligent observability can quickly solve the "whatdunnit" before any outage becomes much worse.