Skip to main content

The Average Organization Suffers 5 Critical IT Incidents per Month

The average organization suffers five critical IT incidents a month, with each one costing the IT department on average USD $36,326 and a further $105,302 to the rest of the business, according to new report by Splunk and analyst firm Quocirca, titled Damage Control - The Impact of Critical IT Incidents. This is forcing IT departments to take resources away from the development of new services to maintain existing infrastructure.

70 percent say a past critical incident has caused reputational damage to their organization

"It's clear that organizations are finding it challenging to maintain end-to-end visibility with the growing volume of data being generated by their IT systems and infrastructure," said Bob Tarzey, analyst, Quocirca. "This is holding IT teams back from being able to drill down and pinpoint the root cause of issues that are causing frequent and recurring problems. This often results in reputational damage and poor customer experience, impacting a company's bottom line. Organizations need to be able to collect and analyze data across all their IT infrastructure more effectively to reduce the time spent in damage control mode and increase time spent on pro-active digital innovation."

Other findings from the report include:

■ Critical IT incidents are negatively impacting businesses. , underlining the importance of timely detection to minimize impact.

■ The volume of IT incidents is hampering the ability to improve IT delivery. 96 percent of organizations are failing to learn from previous incidents. 13.3 percent of all incidents are repeats caused by an inability to properly determine the root cause of issues.

■ Incident detection and investigation is taking too long. 80 percent admitted they could improve the mean-time-to-detect incidents. Incidents on average take 5.81 hours to repair.

■ Organizations are failing to effectively monitor their entire IT estate. 80 percent have operational blind spots, particularly across next-generation technology stacks, hindering their ability to respond to IT incidents quickly. Only 2.5 percent have full visibility across all relevant infrastructure.

Methodology: Quocirca surveyed 1,000 companies in the US, UK, France, Germany, Sweden, Netherlands, Australia, Japan and Singapore.

Hot Topics

The Latest

Two years ago, almost every customer conversation about AI started with the same questions: Which model should we use? What can it do? Is it ready for the enterprise? Today, those discussions have moved on. CIOs are far more interested in how to govern AI, integrate it with existing systems, prepare their workforce and make it part of everyday operations. The challenge is no longer to prove that AI can deliver value. It's instead about how to embed AI into the business in a way that's secure, scalable and delivers measurable outcomes ...

 

Two things happened to production incidents between 2023 and now, and they did not happen at the same speed. The first is that a class of dependency that barely existed three years ago now accounts for one incident in ten. Incidents disclosed by AI model and AI application providers rose from 1.7% of all disclosed unplanned incidents in 2023 to 10.7% in 2026 year to date, roughly a sixfold rise; that counts only incidents at AI companies themselves, so the true share is higher. The second is that the time to close an incident has not come down ...

When an AI assistant gives an incomplete or incorrect answer, teams often blame the model. They adjust prompts, switch models, increase context windows or test a new retrieval strategy. However the model may not be a problem. In many enterprise AI workflows, the problem begins inside the document-ingestion pipeline ...

If you talk to any security or observability teams right now, they're all fighting the same fire: their tooling was built to ingest X, but their sources are pumping Y and soon to be doing Z. The knee-jerk reaction is always the same: we need more platform. However, this reaction is wrong. Let me explain why, because the solution to this problem is foundational, not financial. Instead of hurling yet more money at the problem, make sure you've done what's needed upstream ...

Rapid AI adoption and the unique ways AI workloads operate is redefining the scope and structure of what these teams must deliver. This shift is forcing organizations to rethink how they manage scale, automation, and control, according to The State of SRE and Platform Engineering 2026, a new report from Dynatrace ...

AI is usually talked about as a software tool, but it also depends heavily on the network behind it. Whether a company is using AI for chatbots, automation, monitoring, analytics, or employee support, all of that information has to move across the network in a reliable and secure way. That means AI is not just an application decision. It is also an infrastructure decision. Before organizations rush into AI, they should ask a simple question: Is our network ready to support it? ...

Enterprise AI often lacks governed access to where business processes actually execute. Without that access, AI agents may be able to reason, but they cannot operate reliably across enterprise workflows. For AI agents to effectively carry out workflows, they will require integration-layer context and controls. Organizations can implement these prerequisites by providing AI with managed access to the middleware layer ...

Enterprise networks rarely behave the same way for very long. A routing adjustment in one region may unexpectedly alter application performance in another. A cloud migration may introduce hidden dependencies that go unnoticed until an outage occurs. All the while, the network is managed by several different teams, each of whom use different tool sets — and as a result, have different views of the network ... There’s usually an engineer who remembers why traffic fails over a certain way between sites, or which transparent firewall was added where. The problem is that human memory cannot scale alongside enterprise-scale networks ...

Ask an infrastructure team how confident they are in their ability to govern AI, and most will tell you they've got it handled. A recent survey of 406 IT decision-makers and platform engineering leaders found 86% expressing exactly that confidence. Ask the same group whether they have a formal written AI governance policy, and the number drops to 30%, according to Spacelift's Infrastructure Automation Report ...

In MEAN TIME TO INSIGHT Episode 27, Shamus McGillicuddy, EMA VP of Research, Network Infrastructure and Operations, and Parker Hathcock, EMA Research Director covering IT Service/Operations (ServiceOps), discuss observability unification in modern IT operations ... 

The Average Organization Suffers 5 Critical IT Incidents per Month

The average organization suffers five critical IT incidents a month, with each one costing the IT department on average USD $36,326 and a further $105,302 to the rest of the business, according to new report by Splunk and analyst firm Quocirca, titled Damage Control - The Impact of Critical IT Incidents. This is forcing IT departments to take resources away from the development of new services to maintain existing infrastructure.

70 percent say a past critical incident has caused reputational damage to their organization

"It's clear that organizations are finding it challenging to maintain end-to-end visibility with the growing volume of data being generated by their IT systems and infrastructure," said Bob Tarzey, analyst, Quocirca. "This is holding IT teams back from being able to drill down and pinpoint the root cause of issues that are causing frequent and recurring problems. This often results in reputational damage and poor customer experience, impacting a company's bottom line. Organizations need to be able to collect and analyze data across all their IT infrastructure more effectively to reduce the time spent in damage control mode and increase time spent on pro-active digital innovation."

Other findings from the report include:

■ Critical IT incidents are negatively impacting businesses. , underlining the importance of timely detection to minimize impact.

■ The volume of IT incidents is hampering the ability to improve IT delivery. 96 percent of organizations are failing to learn from previous incidents. 13.3 percent of all incidents are repeats caused by an inability to properly determine the root cause of issues.

■ Incident detection and investigation is taking too long. 80 percent admitted they could improve the mean-time-to-detect incidents. Incidents on average take 5.81 hours to repair.

■ Organizations are failing to effectively monitor their entire IT estate. 80 percent have operational blind spots, particularly across next-generation technology stacks, hindering their ability to respond to IT incidents quickly. Only 2.5 percent have full visibility across all relevant infrastructure.

Methodology: Quocirca surveyed 1,000 companies in the US, UK, France, Germany, Sweden, Netherlands, Australia, Japan and Singapore.

Hot Topics

The Latest

Two years ago, almost every customer conversation about AI started with the same questions: Which model should we use? What can it do? Is it ready for the enterprise? Today, those discussions have moved on. CIOs are far more interested in how to govern AI, integrate it with existing systems, prepare their workforce and make it part of everyday operations. The challenge is no longer to prove that AI can deliver value. It's instead about how to embed AI into the business in a way that's secure, scalable and delivers measurable outcomes ...

 

Two things happened to production incidents between 2023 and now, and they did not happen at the same speed. The first is that a class of dependency that barely existed three years ago now accounts for one incident in ten. Incidents disclosed by AI model and AI application providers rose from 1.7% of all disclosed unplanned incidents in 2023 to 10.7% in 2026 year to date, roughly a sixfold rise; that counts only incidents at AI companies themselves, so the true share is higher. The second is that the time to close an incident has not come down ...

When an AI assistant gives an incomplete or incorrect answer, teams often blame the model. They adjust prompts, switch models, increase context windows or test a new retrieval strategy. However the model may not be a problem. In many enterprise AI workflows, the problem begins inside the document-ingestion pipeline ...

If you talk to any security or observability teams right now, they're all fighting the same fire: their tooling was built to ingest X, but their sources are pumping Y and soon to be doing Z. The knee-jerk reaction is always the same: we need more platform. However, this reaction is wrong. Let me explain why, because the solution to this problem is foundational, not financial. Instead of hurling yet more money at the problem, make sure you've done what's needed upstream ...

Rapid AI adoption and the unique ways AI workloads operate is redefining the scope and structure of what these teams must deliver. This shift is forcing organizations to rethink how they manage scale, automation, and control, according to The State of SRE and Platform Engineering 2026, a new report from Dynatrace ...

AI is usually talked about as a software tool, but it also depends heavily on the network behind it. Whether a company is using AI for chatbots, automation, monitoring, analytics, or employee support, all of that information has to move across the network in a reliable and secure way. That means AI is not just an application decision. It is also an infrastructure decision. Before organizations rush into AI, they should ask a simple question: Is our network ready to support it? ...

Enterprise AI often lacks governed access to where business processes actually execute. Without that access, AI agents may be able to reason, but they cannot operate reliably across enterprise workflows. For AI agents to effectively carry out workflows, they will require integration-layer context and controls. Organizations can implement these prerequisites by providing AI with managed access to the middleware layer ...

Enterprise networks rarely behave the same way for very long. A routing adjustment in one region may unexpectedly alter application performance in another. A cloud migration may introduce hidden dependencies that go unnoticed until an outage occurs. All the while, the network is managed by several different teams, each of whom use different tool sets — and as a result, have different views of the network ... There’s usually an engineer who remembers why traffic fails over a certain way between sites, or which transparent firewall was added where. The problem is that human memory cannot scale alongside enterprise-scale networks ...

Ask an infrastructure team how confident they are in their ability to govern AI, and most will tell you they've got it handled. A recent survey of 406 IT decision-makers and platform engineering leaders found 86% expressing exactly that confidence. Ask the same group whether they have a formal written AI governance policy, and the number drops to 30%, according to Spacelift's Infrastructure Automation Report ...

In MEAN TIME TO INSIGHT Episode 27, Shamus McGillicuddy, EMA VP of Research, Network Infrastructure and Operations, and Parker Hathcock, EMA Research Director covering IT Service/Operations (ServiceOps), discuss observability unification in modern IT operations ...