Skip to main content

How Good Are You At Blamestorming?

Blamestorming is a well-known game IT organizations spend hours playing every week. There are different leagues and names for this game. Some call it the "war room", others the "service restoration call", but they basically follow the same simple rules. The goal of the game is to figure out the origin of an IT service degradation issue and collectively identify the one person — or the team — to blame for it, then hold them responsible for the resolution and the business consequences.

It often occurs in the form of a meeting, where the players meet in a conference room or on the phone.

The game usually starts with an IT performance degradation that impacts your most critical business applications. The cause, and this is important, should not be obvious; nothing in plain sight like a load balancer or a web server being down.

And there are plenty of events that can make a great game if your company is:

- An e-commerce company and, for some reason, your payment processing system is unexpectedly slow, preventing sales from being booked online

- A manufacturer and your supply chain system does not respond, causing delays and production to stall

- A hospital and the clinicians start complaining about the system performance

- A bank servicing remote locations from where the tellers can’t access the system

Remember, the more complex the application architecture, the better — it will make the game more interesting with many different players. The reason these games are so popular is that they are most often unplanned and unforeseen.

So how do you get to play? You will most likely get invited by your boss directly. When your phone rings the game play is on.

There are some indicators that will let you know if this is going to be an interesting game.

First, is the location the usual meeting room or a different one because the former is too small? The larger meeting room means that there will be many players, which should make the game last longer.

Another sign is the attendee list: if your boss’s boss is invited, then you know it's going to be an intense game. We have heard of games being kicked off by the CIO or the CFO and hosted in a hotel conference room because they could not accommodate all the players. The number of players will give you an idea of the complexity of the mystery to be solved. We have seen teams so good that one game could last more than 24 hours.

There are different types of players and many styles you can adopt, just like playing poker:

- The guessers. They don't have a clue, but they will guess anyway. They usually don't bring anything relevant, and you will hear them say things like, “There must be something wrong with the web servers because the team in Houston can't even access the system, or maybe it's the configuration of the database, or maybe it's the network itself?”

- The fact checkers. They come to the table with some kind of evidence that they can't be blamed for the problem. We have found that the network teams are usually pretty good at this, as they are used to being suspected at the beginning of the game, "It has to be the network."

- The screamers. They will try to intimate the other players. Sometimes the loudest screamers win or at least buy some time.

- The quiet. They will stay quiet until other players question them. Then they start challenging the elements and ask for evidence.

Image removed.

Basically, decide who is to blame for the screw up, and be ready to give enough evidence that will justify a deeper investigation by the players.

The other teams, if able, will try to disprove your theory by showing you one piece of evidence that exonerates them.

And this goes on until the case is solved and all the false possibilities have been eliminated. In the end, whoever is wrong, owns the issue and resolution and loses the game!

It’s important to note that we are witnessing a trend in companies to limit the possibilities for IT staff to participate in blamestorming competitions. They claim that these games are costly for the company due to the loss of revenue involved with revenue-generating application issues. They also complain about drop-offs in employee productivity when the issue prevents end users from doing their job, such as ERP failures.

Be aware that IT directors will often try to limit the number of games and their duration because they think this activity consumes too much of the IT staff time. They feel this time could be better put to use on more strategic projects.

As a result, we have seen IT management teams put into place new generation APM solutions. These solutions provide too much troubleshooting information upfront, which eliminates the mystery and the suspense.

Recently, a large producer and distributor of eyewear reduced the average duration of their games by 85% from 20 hours to less than 3 hours today, reducing the number of players at the same time. They are just killing the game!

Comic strip by Siobhan Ohmart and Vincent Geffray

ABOUT Vincent Geffray

Vincent Geffray is Senior Product Marketing Manager of Enterprise Solutions for Compuware’s APM business and has more than 14 years of experience in the IT Operations Management industry, with specialization in datacenter performance improvement.

Hot Topics

The Latest

As businesses increasingly rely on high-performance applications to deliver seamless user experiences, the demand for fast, reliable, and scalable data storage systems has never been greater. Redis — an open-source, in-memory data structure store — has emerged as a popular choice for use cases ranging from caching to real-time analytics. But with great performance comes the need for vigilant monitoring ...

Kubernetes was not initially designed with AI's vast resource variability in mind, and the rapid rise of AI has exposed Kubernetes limitations, particularly when it comes to cost and resource efficiency. Indeed, AI workloads differ from traditional applications in that they require a staggering amount and variety of compute resources, and their consumption is far less consistent than traditional workloads ... Considering the speed of AI innovation, teams cannot afford to be bogged down by these constant infrastructure concerns. A solution is needed ...

AI is the catalyst for significant investment in data teams as enterprises require higher-quality data to power their AI applications, according to the State of Analytics Engineering Report from dbt Labs ...

Misaligned architecture can lead to business consequences, with 93% of respondents reporting negative outcomes such as service disruptions, high operational costs and security challenges ...

A Gartner analyst recently suggested that GenAI tools could create 25% time savings for network operational teams. Where might these time savings come from? How are GenAI tools helping NetOps teams today, and what other tasks might they take on in the future as models continue improving? In general, these savings come from automating or streamlining manual NetOps tasks ...

IT and line-of-business teams are increasingly aligned in their efforts to close the data gap and drive greater collaboration to alleviate IT bottlenecks and offload growing demands on IT teams, according to The 2025 Automation Benchmark Report: Insights from IT Leaders on Enterprise Automation & the Future of AI-Driven Businesses from Jitterbit ...

A large majority (86%) of data management and AI decision makers cite protecting data privacy as a top concern, with 76% of respondents citing ROI on data privacy and AI initiatives across their organization, according to a new Harris Poll from Collibra ...

According to Gartner, Inc. the following six trends will shape the future of cloud over the next four years, ultimately resulting in new ways of working that are digital in nature and transformative in impact ...

2020 was the equivalent of a wedding with a top-shelf open bar. As businesses scrambled to adjust to remote work, digital transformation accelerated at breakneck speed. New software categories emerged overnight. Tech stacks ballooned with all sorts of SaaS apps solving ALL the problems — often with little oversight or long-term integration planning, and yes frequently a lot of duplicated functionality ... But now the music's faded. The lights are on. Everyone from the CIO to the CFO is checking the bill. Welcome to the Great SaaS Hangover ...

Regardless of OpenShift being a scalable and flexible software, it can be a pain to monitor since complete visibility into the underlying operations is not guaranteed ... To effectively monitor an OpenShift environment, IT administrators should focus on these five key elements and their associated metrics ...

How Good Are You At Blamestorming?

Blamestorming is a well-known game IT organizations spend hours playing every week. There are different leagues and names for this game. Some call it the "war room", others the "service restoration call", but they basically follow the same simple rules. The goal of the game is to figure out the origin of an IT service degradation issue and collectively identify the one person — or the team — to blame for it, then hold them responsible for the resolution and the business consequences.

It often occurs in the form of a meeting, where the players meet in a conference room or on the phone.

The game usually starts with an IT performance degradation that impacts your most critical business applications. The cause, and this is important, should not be obvious; nothing in plain sight like a load balancer or a web server being down.

And there are plenty of events that can make a great game if your company is:

- An e-commerce company and, for some reason, your payment processing system is unexpectedly slow, preventing sales from being booked online

- A manufacturer and your supply chain system does not respond, causing delays and production to stall

- A hospital and the clinicians start complaining about the system performance

- A bank servicing remote locations from where the tellers can’t access the system

Remember, the more complex the application architecture, the better — it will make the game more interesting with many different players. The reason these games are so popular is that they are most often unplanned and unforeseen.

So how do you get to play? You will most likely get invited by your boss directly. When your phone rings the game play is on.

There are some indicators that will let you know if this is going to be an interesting game.

First, is the location the usual meeting room or a different one because the former is too small? The larger meeting room means that there will be many players, which should make the game last longer.

Another sign is the attendee list: if your boss’s boss is invited, then you know it's going to be an intense game. We have heard of games being kicked off by the CIO or the CFO and hosted in a hotel conference room because they could not accommodate all the players. The number of players will give you an idea of the complexity of the mystery to be solved. We have seen teams so good that one game could last more than 24 hours.

There are different types of players and many styles you can adopt, just like playing poker:

- The guessers. They don't have a clue, but they will guess anyway. They usually don't bring anything relevant, and you will hear them say things like, “There must be something wrong with the web servers because the team in Houston can't even access the system, or maybe it's the configuration of the database, or maybe it's the network itself?”

- The fact checkers. They come to the table with some kind of evidence that they can't be blamed for the problem. We have found that the network teams are usually pretty good at this, as they are used to being suspected at the beginning of the game, "It has to be the network."

- The screamers. They will try to intimate the other players. Sometimes the loudest screamers win or at least buy some time.

- The quiet. They will stay quiet until other players question them. Then they start challenging the elements and ask for evidence.

Image removed.

Basically, decide who is to blame for the screw up, and be ready to give enough evidence that will justify a deeper investigation by the players.

The other teams, if able, will try to disprove your theory by showing you one piece of evidence that exonerates them.

And this goes on until the case is solved and all the false possibilities have been eliminated. In the end, whoever is wrong, owns the issue and resolution and loses the game!

It’s important to note that we are witnessing a trend in companies to limit the possibilities for IT staff to participate in blamestorming competitions. They claim that these games are costly for the company due to the loss of revenue involved with revenue-generating application issues. They also complain about drop-offs in employee productivity when the issue prevents end users from doing their job, such as ERP failures.

Be aware that IT directors will often try to limit the number of games and their duration because they think this activity consumes too much of the IT staff time. They feel this time could be better put to use on more strategic projects.

As a result, we have seen IT management teams put into place new generation APM solutions. These solutions provide too much troubleshooting information upfront, which eliminates the mystery and the suspense.

Recently, a large producer and distributor of eyewear reduced the average duration of their games by 85% from 20 hours to less than 3 hours today, reducing the number of players at the same time. They are just killing the game!

Comic strip by Siobhan Ohmart and Vincent Geffray

ABOUT Vincent Geffray

Vincent Geffray is Senior Product Marketing Manager of Enterprise Solutions for Compuware’s APM business and has more than 14 years of experience in the IT Operations Management industry, with specialization in datacenter performance improvement.

Hot Topics

The Latest

As businesses increasingly rely on high-performance applications to deliver seamless user experiences, the demand for fast, reliable, and scalable data storage systems has never been greater. Redis — an open-source, in-memory data structure store — has emerged as a popular choice for use cases ranging from caching to real-time analytics. But with great performance comes the need for vigilant monitoring ...

Kubernetes was not initially designed with AI's vast resource variability in mind, and the rapid rise of AI has exposed Kubernetes limitations, particularly when it comes to cost and resource efficiency. Indeed, AI workloads differ from traditional applications in that they require a staggering amount and variety of compute resources, and their consumption is far less consistent than traditional workloads ... Considering the speed of AI innovation, teams cannot afford to be bogged down by these constant infrastructure concerns. A solution is needed ...

AI is the catalyst for significant investment in data teams as enterprises require higher-quality data to power their AI applications, according to the State of Analytics Engineering Report from dbt Labs ...

Misaligned architecture can lead to business consequences, with 93% of respondents reporting negative outcomes such as service disruptions, high operational costs and security challenges ...

A Gartner analyst recently suggested that GenAI tools could create 25% time savings for network operational teams. Where might these time savings come from? How are GenAI tools helping NetOps teams today, and what other tasks might they take on in the future as models continue improving? In general, these savings come from automating or streamlining manual NetOps tasks ...

IT and line-of-business teams are increasingly aligned in their efforts to close the data gap and drive greater collaboration to alleviate IT bottlenecks and offload growing demands on IT teams, according to The 2025 Automation Benchmark Report: Insights from IT Leaders on Enterprise Automation & the Future of AI-Driven Businesses from Jitterbit ...

A large majority (86%) of data management and AI decision makers cite protecting data privacy as a top concern, with 76% of respondents citing ROI on data privacy and AI initiatives across their organization, according to a new Harris Poll from Collibra ...

According to Gartner, Inc. the following six trends will shape the future of cloud over the next four years, ultimately resulting in new ways of working that are digital in nature and transformative in impact ...

2020 was the equivalent of a wedding with a top-shelf open bar. As businesses scrambled to adjust to remote work, digital transformation accelerated at breakneck speed. New software categories emerged overnight. Tech stacks ballooned with all sorts of SaaS apps solving ALL the problems — often with little oversight or long-term integration planning, and yes frequently a lot of duplicated functionality ... But now the music's faded. The lights are on. Everyone from the CIO to the CFO is checking the bill. Welcome to the Great SaaS Hangover ...

Regardless of OpenShift being a scalable and flexible software, it can be a pain to monitor since complete visibility into the underlying operations is not guaranteed ... To effectively monitor an OpenShift environment, IT administrators should focus on these five key elements and their associated metrics ...