One point to reinforce is that the digital war room — physical, virtual or hybrid — is not in retreat but in fact is growing in scope to include greater participation from development and security. It's also becoming more proactive, with on average more than 30% of "major incidents" before they impacted business service performance.
The reasons for this added (not diminished) level of relevance will be examined more in depth in my webinar on April 11th (and yes, there will be replays), but generally the answer lies in the fact that improved levels of team efficiency are critical to the future of IT, and the digital war room shines a spotlight on this evolving reality.
In this blog I'm providing a few additional highlights from the insights we got on digital war room organization and processes.
A Few Organizational Insights
One of the questions we asked was directed at finding out whether war rooms, as they evolve, were becoming more organizationally defined, or more sporadic and ad-hoc. The answer was solidly in the "more formalized" category (47%) versus the group with "more ad-hoc teams and processes" (28%). Another 22% indicated that their teams were already solidly formalized and established.
Then, when we evaluated success rates to this mix, we saw that those digital war rooms becoming "more formalized and established" were far more likely to align with digital war room effectiveness than the other groups.
Well defined teams that can be brought together across all domains provide a unique advantage over fragmented, technically isolated teams
If you think about this, it does suggest a contradiction to some of the trendier thinking endorsing multiple teams and more completely decentralized ways of working. But the logic for core consistency is clear. Well defined teams that can be brought together across all domains provide a unique advantage over fragmented, more technically isolated teams when confronting the full gamut of "major incident" possibilities.
And BTW, the average head count for these teams across small, medium and large was about 15. The implication being not that all 15 stakeholders are being activated for every single incident, but there are 15 individuals assigned and available for digital war room decision making on an on-going, as-needed basis. The trend, BTW, is toward growing not shrinking levels of involvement — in large part because of the accelerating need to include development and security professionals. The overall data also showed a significant role in digital war room decision making for non-IT, or business stakeholders.
Having a single organizational owner, also helps to drive war-room efficiencies. Interestingly, "Security/compliance" was in third place for war-room ownership after "ITSM" and the "executive suite." Having senior executive involvement helped, as well. The most prevalent was ongoing "director-level" involvement, but the most effective turned out to be "CIO-level" involvement.
Processes (or Lack Thereof)
In last week's blog, we enumerated the following critical processes that help to define war-room performance:
■ Initial awareness, which is usually driven by events or some other type of automated intelligence, or complaints to the service desk.
■ Response team engagement and coordination, bringing relevant stakeholders together and providing a context for them to work together.
■ Triage and diagnostics, where problems are understood in context and then detailed requirements for remediation can be defined.
■ Remediation, where active fixes to major incidents are made, often through change and configuration management procedures.
■ Validation, in which testing is done to ensure that actions for remediation were successful, ideally from a business impact as well as a purely technical perspective.
In the non-progressive category, we discovered that, based on our data, the average response indicated only a little more than half (2.57) of these processes were defined — a surprising revelation in a rather negative way. When we mapped "success rates" to the number of processes mapped out, however, we did get a reasonable correlation:
■ 3 for the extremely successful
■ 2.5 for the successful
■ 2 for the only marginally successful
The most prevalently defined process was response team coordination — which also turned out to be the most problematic or delay-causing process. In fact, identifying process with delay or problems mapped well to the processes that were most likely to be identified, suggesting that clarifying the reality of what's going on opens the door to realizing what's wrong and how improvements can be made.
Timing is Everything
Time to assemble an effective team, on average, was about 1.5 hours, which could be damaging when a serious outage occurred
We also asked about times associated with these processes. When we asked about the time to assemble an effective team, the average was about 1.5 hours, which could, of course, be meaningfully damaging when a serious outage occurred. When asked about total time to resolution, the average was about six hours, but 20 percent took more than eleven hours. Once again, as an average, this can be concerning for incidents with major business impacts.
These are again, just a few of many highlights from our research.
Don't forget to watch the webinar for a great many more insights.
Read my third and final blog on the digital war room: The Digital War Room in Changing Times: The Impacts of DevOps, Cloud and SecOps
While remote work policies have been gaining steam for the better part of the past decade across the enterprise space — driven in large part by more agile and scalable, cloud-delivered business solutions — recent events have pushed adoption into overdrive ...
Time-critical, unplanned work caused by IT disruptions continues to plague enterprises around the world, leading to lost revenue, significant employee morale problems and missed opportunities to innovate, according to the State of Unplanned Work Report 2020, conducted by Dimensional Research for PagerDuty ...
In today's iterative world, development teams care a lot more about how apps are running. There's a demand for fixing actionable items. Developers want to know exactly what's broken, what to fix right now, and what can wait. They want to know, "Do we build or fix?" This trade-off between building new features versus fixing bugs is one of the key factors behind the adoption of Application Stability management tools ...
With the rise of mobile apps and iterative development releases, Application Stability has answered the widespread need to monitor applications in a new way, shifting the focus from servers and networks to the customer experience. The emergence of Application Stability has caused some consternation for diehard APM fans. However, these two solutions embody very distinct monitoring focuses, which leads me to believe there's room for both tools, as well as different teams for both ...
The 2019 State of E-Commerce Infrastructure Report, from Webscale, analyzes findings from a comprehensive survey of more than 450 ecommerce professionals regarding how their online stores performed during the 2019 holiday season. Some key insights from the report include ...
Robinhood is a unicorn startup that has been disrupting the way by which many millennials have been investing and managing their money for the past few years. For Robinhood, the burden of proof was to show that they can provide an infrastructure that is as scalable, reliable and secure as that of major banks who have been developing their trading infrastructure for the last quarter-century. That promise fell flat last week, when the market volatility brought about a set of edge cases that brought Robinhood's trading app to its knees ...
Application backend monitoring is the key to acquiring visibility across the enterprise's application stack, from the application layer and underlying infrastructure to third-party API services, web servers and databases, be they on-premises, in a public or private cloud, or in a hybrid model. By tracking and reporting performance in real time, IT teams can ensure applications perform at peak efficiency — and guarantee a seamless customer experience. How can IT operations teams improve application backend monitoring? By embracing artificial intelligence for operations — AIOps ...
In 2020, DevOps teams will face heightened expectations for higher speed and frequency of code delivery, which means their IT environments will become even more modular, ephemeral and dynamic — and significantly more complicated to monitor. As a result, AIOps will further cement its position as the most effective technology that DevOps teams can use to see and control what's going on with their applications and their underlying infrastructure, so that they can prevent outages. Here I outline five key trends to watch related to how AIOps will impact DevOps in 2020 and beyond ...
With the spread of the coronavirus (COVID-19), CIOs should focus on three short-term actions to increase their organizations' resilience against disruptions and prepare for rebound and growth, according to Gartner ...
Whether you consider the first generation of APM or the updates that followed for SOA and microservices, the most basic premise of the tools remains the same — PROVIDE VISIBILITY ...