Skip to main content

Major Incident Management: Are You Prepared?

Troy McAlpin

If your critical business applications go down, or even run below peak level, your business pays a tremendous price. When a major IT incident occurs, engaging the right people quickly to restore service and manage communications is crucial. No big news flash there.

However I have to admit I was pretty alarmed when a new survey by Dimensional Research revealed an almost cavalier approach toward the handling of major IT incidents. Security and business incidents occur so regularly that we aren't even surprised anymore when they happen. They come in the form of data breaches, malware attacks, power outages, intermittent service availability and performance degradation to name a few.

Click here to see infographic below

In fact, according to the survey, 68 percent of companies surveyed experienced a major incident at least several times a year. For larger organizations with at least 5,000 employees, that figure rises to more than 90 percent.

The Consequences of Slow Response

Rapid, effective response can limit the damage. In a separate survey performed by Dimensional Research in April, 60 percent said finding and engaging the right person takes more than 15 minutes. But before 15 minutes have elapsed, almost half (45 percent) said the business has already started to suffer.

And the suffering is real, according to the most recent survey. A large majority (82 percent) says application downtime affects revenue. According to a 2014 study by industry analyst firm IDC, the average cost of a critical application failure per hour is $500,000 to $1 million.

Given how quickly, seriously and frequently a major incident affects businesses, why aren't they making critical investments in major incident management?

Money and Resources

First, a best-in-class intelligent communication platform is not cheap. So organizations that still view major incidents as unlikely events could be put off just by the cost.

Another factor is resources. Barely half of companies in the new survey (52 percent) have a major incident team. Only 44 percent of those companies have team members who are dedicated solely to major incident management.

Finally, maybe the word hasn't gotten out to all companies just how important rapid and effective major incident management is.

Is the Status Quo Working?

The effectiveness of current practices is not entirely clear because only 68 percent of companies even specify target times for resolving major incidents. But among those that are, the results are not good.

More than three-quarters of respondents, 76 percent, miss their target times sometimes or often. Most companies in the survey (58 percent) have target times between 30-90 minutes. Remember the IDC figure of up to $1 million per hour of application downtime? Do the math.

So What Have We Learned?

Regardless of why more companies haven't created processes and implemented solutions for resolving major incidents, the current state of affairs is troubling. And this article has only touched on the financial implications of major incidents. Business also suffer from reputational damage, loss of customer loyalty and trust, and sanctions from regulatory bodies.

Major incidents happen frequently, and every business should assume that sooner or later it will experience one. The ability to quickly, efficiently and effectively respond could save the business, its shareholders, its customers and partners.

Are you prepared?



Hot Topics

The Latest

Developers building AI applications are not just looking for fault patterns after deployment; they must detect issues quickly during development and have the ability to prevent issues after going live. Unfortunately, traditional observability tools can no longer meet the needs of AI-driven enterprise application development. AI-powered detection and auto-remediation tools designed to keep pace with rapid development are now emerging to proactively manage performance and prevent downtime ...

Every few years, the cybersecurity industry adopts a new buzzword. "Zero Trust" has endured longer than most — and for good reason. Its promise is simple: trust nothing by default, verify everything continuously. Yet many organizations still hesitate to implement Zero Trust Network Access (ZTNA). The problem isn't that ZTNA doesn't work. It's that it's often misunderstood ...

For many retail brands, peak season is the annual stress test of their digital infrastructure. It's also when often technical dashboards glow green, yet customer feedback, digital experience frustration, and conversion trends tell a different story entirely. Over the past several years, we've seen the same pattern across retail, financial services, travel, and media: internal application performance metrics fail to capture the true experience of users connecting over local broadband, mobile carriers, and congested networks using multiple devices across geographies ...

PostgreSQL promises greater flexibility, performance, and cost savings compared to proprietary alternatives. But successfully deploying it isn't always straightforward, and there are some hidden traps along the way that even seasoned IT leaders can stumble into. In this blog, I'll highlight five of the most common pitfalls with PostgreSQL deployment and offer guidance on how to avoid them, along with the best path forward ...

The rise of hybrid cloud environments, the explosion of IoT devices, the proliferation of remote work, and advanced cyber threats have created a monitoring challenge that traditional approaches simply cannot meet. IT teams find themselves drowning in a sea of data, struggling to identify critical threats amidst a deluge of alerts, and often reacting to incidents long after they've begun. This is where AI and ML are leveraged ...

Three practices, chaos testing, incident retrospectives, and AIOps-driven monitoring, are transforming platform teams from reactive responders into proactive builders of resilient, self-healing systems. The evolution is not just technical; it's cultural. The modern platform engineer isn't just maintaining infrastructure. They're product owners designing for reliability, observability, and continuous improvement ...

Getting applications into the hands of those who need them quickly and securely has long been the goal of a branch of IT often referred to as End User Computing (EUC). Over recent years, the way applications (and data) have been delivered to these "users" has changed noticeably. Organizations have many more choices available to them now, and there will be more to come ... But how did we get here? Where are we going? Is this all too complicated? ...

On November 18, a single database permission change inside Cloudflare set off a chain of failures that rippled across the Internet. Traffic stalled. Authentication broke. Workers KV returned waves of 5xx errors as systems fell in and out of sync. For nearly three hours, one of the most resilient networks on the planet struggled under the weight of a change no one expected to matter ... Cloudflare recovered quickly, but the deeper lesson reaches far beyond this incident ...

Chris Steffen and Ken Buckler from EMA discuss the Cloudflare outage and what availability means in the technology space ...

Every modern industry is confronting the same challenge: human reaction time is no longer fast enough for real-time decision environments. Across sectors, from financial services to manufacturing to cybersecurity and beyond, the stakes mirror those of autonomous vehicles — systems operating in complex, high-risk environments where milliseconds matter ...

Major Incident Management: Are You Prepared?

Troy McAlpin

If your critical business applications go down, or even run below peak level, your business pays a tremendous price. When a major IT incident occurs, engaging the right people quickly to restore service and manage communications is crucial. No big news flash there.

However I have to admit I was pretty alarmed when a new survey by Dimensional Research revealed an almost cavalier approach toward the handling of major IT incidents. Security and business incidents occur so regularly that we aren't even surprised anymore when they happen. They come in the form of data breaches, malware attacks, power outages, intermittent service availability and performance degradation to name a few.

Click here to see infographic below

In fact, according to the survey, 68 percent of companies surveyed experienced a major incident at least several times a year. For larger organizations with at least 5,000 employees, that figure rises to more than 90 percent.

The Consequences of Slow Response

Rapid, effective response can limit the damage. In a separate survey performed by Dimensional Research in April, 60 percent said finding and engaging the right person takes more than 15 minutes. But before 15 minutes have elapsed, almost half (45 percent) said the business has already started to suffer.

And the suffering is real, according to the most recent survey. A large majority (82 percent) says application downtime affects revenue. According to a 2014 study by industry analyst firm IDC, the average cost of a critical application failure per hour is $500,000 to $1 million.

Given how quickly, seriously and frequently a major incident affects businesses, why aren't they making critical investments in major incident management?

Money and Resources

First, a best-in-class intelligent communication platform is not cheap. So organizations that still view major incidents as unlikely events could be put off just by the cost.

Another factor is resources. Barely half of companies in the new survey (52 percent) have a major incident team. Only 44 percent of those companies have team members who are dedicated solely to major incident management.

Finally, maybe the word hasn't gotten out to all companies just how important rapid and effective major incident management is.

Is the Status Quo Working?

The effectiveness of current practices is not entirely clear because only 68 percent of companies even specify target times for resolving major incidents. But among those that are, the results are not good.

More than three-quarters of respondents, 76 percent, miss their target times sometimes or often. Most companies in the survey (58 percent) have target times between 30-90 minutes. Remember the IDC figure of up to $1 million per hour of application downtime? Do the math.

So What Have We Learned?

Regardless of why more companies haven't created processes and implemented solutions for resolving major incidents, the current state of affairs is troubling. And this article has only touched on the financial implications of major incidents. Business also suffer from reputational damage, loss of customer loyalty and trust, and sanctions from regulatory bodies.

Major incidents happen frequently, and every business should assume that sooner or later it will experience one. The ability to quickly, efficiently and effectively respond could save the business, its shareholders, its customers and partners.

Are you prepared?



Hot Topics

The Latest

Developers building AI applications are not just looking for fault patterns after deployment; they must detect issues quickly during development and have the ability to prevent issues after going live. Unfortunately, traditional observability tools can no longer meet the needs of AI-driven enterprise application development. AI-powered detection and auto-remediation tools designed to keep pace with rapid development are now emerging to proactively manage performance and prevent downtime ...

Every few years, the cybersecurity industry adopts a new buzzword. "Zero Trust" has endured longer than most — and for good reason. Its promise is simple: trust nothing by default, verify everything continuously. Yet many organizations still hesitate to implement Zero Trust Network Access (ZTNA). The problem isn't that ZTNA doesn't work. It's that it's often misunderstood ...

For many retail brands, peak season is the annual stress test of their digital infrastructure. It's also when often technical dashboards glow green, yet customer feedback, digital experience frustration, and conversion trends tell a different story entirely. Over the past several years, we've seen the same pattern across retail, financial services, travel, and media: internal application performance metrics fail to capture the true experience of users connecting over local broadband, mobile carriers, and congested networks using multiple devices across geographies ...

PostgreSQL promises greater flexibility, performance, and cost savings compared to proprietary alternatives. But successfully deploying it isn't always straightforward, and there are some hidden traps along the way that even seasoned IT leaders can stumble into. In this blog, I'll highlight five of the most common pitfalls with PostgreSQL deployment and offer guidance on how to avoid them, along with the best path forward ...

The rise of hybrid cloud environments, the explosion of IoT devices, the proliferation of remote work, and advanced cyber threats have created a monitoring challenge that traditional approaches simply cannot meet. IT teams find themselves drowning in a sea of data, struggling to identify critical threats amidst a deluge of alerts, and often reacting to incidents long after they've begun. This is where AI and ML are leveraged ...

Three practices, chaos testing, incident retrospectives, and AIOps-driven monitoring, are transforming platform teams from reactive responders into proactive builders of resilient, self-healing systems. The evolution is not just technical; it's cultural. The modern platform engineer isn't just maintaining infrastructure. They're product owners designing for reliability, observability, and continuous improvement ...

Getting applications into the hands of those who need them quickly and securely has long been the goal of a branch of IT often referred to as End User Computing (EUC). Over recent years, the way applications (and data) have been delivered to these "users" has changed noticeably. Organizations have many more choices available to them now, and there will be more to come ... But how did we get here? Where are we going? Is this all too complicated? ...

On November 18, a single database permission change inside Cloudflare set off a chain of failures that rippled across the Internet. Traffic stalled. Authentication broke. Workers KV returned waves of 5xx errors as systems fell in and out of sync. For nearly three hours, one of the most resilient networks on the planet struggled under the weight of a change no one expected to matter ... Cloudflare recovered quickly, but the deeper lesson reaches far beyond this incident ...

Chris Steffen and Ken Buckler from EMA discuss the Cloudflare outage and what availability means in the technology space ...

Every modern industry is confronting the same challenge: human reaction time is no longer fast enough for real-time decision environments. Across sectors, from financial services to manufacturing to cybersecurity and beyond, the stakes mirror those of autonomous vehicles — systems operating in complex, high-risk environments where milliseconds matter ...