Production Issues Take Longer to Solve Despite Advancements in Observability
April 25, 2022
Share this

Despite all of the advancement and maturity of observability systems, production issues take longer to resolve, according to the DevOps Pulse 2022 Report from

DevOps Pulse 2022 respondents consistently reported increasing complexity across their cloud environments — driven by everything from expanding microservices architecture to the proliferation and complexity of observability tools themselves — meaning they are struggling to maintain clear visibility, quickly resolve production issues and manage related monitoring costs.

64% of respondents reported an average MTTR of over an hour compared to 47% reported in the DevOps Pulse 2021.

What's more, 53.4% of respondents surveyed last year claimed to have resolved production issues within an hour on average — this year, that number dropped to 35.94%. This trend is being driven by factors ranging from growing cloud data volumes and systems complexity, to issues of observability tool sprawl, and the need for greater expertise among DevOps teams.

Additional key findings identified in the report include:

Observability costs and data volumes are growing concerns

27% of respondents ranked total cost of ownership and the large volumes of data being ingested into the tools among their main challenges in maintaining effective observability.

Observability is maturing

Some 77% of respondents rated their efforts over 3 on a scale of 1-5 when asked to indicate how extensive and mature their observability strategy has become — compared to last year, when more than 30% of respondents indicated a low score of two or less.

This is combined with a palpable increase in observability tool sprawl. 22% of those surveyed indicated their organization uses 5 or more observability tools, compared to 11% last year.

Open source capabilities are widely used

Open source capabilities are widely used by 90% of respondents. Open Source observability is the most common way to deploy observability.

A shared services model is growing in popularity

Over 85% of respondents indicated that their organizations operate using a shared services observability model in which a central team is responsible for implementing and maintaining tooling for other stakeholders such as app developers, SREs and DevOps teams.

Tools are further diversifying

This year’s survey responses highlighted the full range of observability tools in use by practitioners including log management and analysis (67%), and infrastructure monitoring (59%), followed somewhat behind by distributed tracing (27%) and APM (22%). Meanwhile some 21% said that they have deployed all of these capabilities

"Essentially, the 2022 DevOps Pulse Report reveals that there is too much data and the current model for observability is broken," says Tomer Levy, CEO of "As practices and implementations expand, organizations are becoming more concerned about the impact of data volumes on production quality and cost. In addition to offering an analysis of the evolving landscape, DevOps Pulse 2022 calls on organizations to think carefully about the impact of Kubernetes and Microservices and constantly evaluate telemetry data value and hygiene."

By closely tracking and analyzing data that is central to core observability requirements and finding ways to reduce MTTR despite identified challenges, organizations can better calculate associated spending and ROI. Emphasizing these factors combined with an increased focus on application and data security solve the challenges identified by DevOps teams and observability practitioners.

Share this

The Latest

May 26, 2022

Site reliability engineers are development-focused IT professionals who work on developing and implementing solutions that solve reliability, availability, and scale problems. On the other hand, DevOps engineers are ops-focused workers who solve development pipeline problems. While there is a divide between the two professions, both sets of engineers cross the gap regularly, delivering their expertise and opinions to the other side and vice versa ...

May 25, 2022

Site reliability engineering (SRE) is fast becoming an essential aspect of modern IT operations, particularly in highly scaled, big data environments. As businesses and industries shift to the digital and embrace new IT infrastructures and technologies to remain operational and competitive, the need for a new approach for IT teams to find and manage the balance between launching new systems and features and ensuring these are intuitive, reliable, and friendly for end users has intensified as well ...

May 24, 2022

The most sophisticated observability practitioners (leaders) are able to cut downtime costs by 90%, from an estimated $23.8 million annually to just $2.5 million, compared to observability beginners, according to the State of Observability 2022 from Splunk in collaboration with the Enterprise Strategy Group. What's more, leaders in observability are more innovative and more successful at achieving digital transformation outcomes and other initiatives ...

May 23, 2022

Programmatically tracked service level indicators (SLIs) are foundational to every site reliability engineering practice. When engineering teams have programmatic SLIs in place, they lessen the need to manually track performance and incident data. They're also able to reduce manual toil because our DevOps teams define the capabilities and metrics that define their SLI data, which they collect automatically — hence "programmatic" ...

May 19, 2022

Recently, a regional healthcare organization wanted to retire its legacy monitoring tools and adopt AIOps. The organization asked Windward Consulting to implement an AIOps strategy that would help streamline its outdated and unwieldy IT system management. Our team's AIOps implementation process helped this client and can help others in the industry too. Here's what my team did ...

May 18, 2022

You've likely heard it before: every business is a digital business. However, some businesses and sectors digitize more quickly than others. Healthcare has traditionally been on the slower side of digital transformation and technology adoption, but that's changing. As healthcare organizations roll out innovations at increasing velocity, they must build a long-term strategy for how they will maintain the uptime of their critical apps and services. And there's only one tool that can ensure this continuous availability in our modern IT ecosystems. AIOps can help IT Operations teams ensure the uptime of critical apps and services ...

May 17, 2022

Between 2012 to 2015 all of the hyperscalers attempted to use the legacy APM solutions to improve their own visibility. To no avail. The problem was that none of the previous generations of APM solutions could match the scaling demand, nor could they provide interoperability due to their proprietary and exclusive agentry ...

May 16, 2022

The DevOps journey begins by understanding a team's DevOps flow and identifying precisely what tasks deliver the best return on engineers' time when automated. The rest of this blog will help DevOps team managers by outlining what jobs can — and should be automated ...

May 12, 2022

A survey from Snow Software polled more than 500 IT leaders to determine the current state of cloud infrastructure. Nearly half of the IT leaders who responded agreed that cloud was critical to operations during the pandemic with the majority deploying a hybrid cloud strategy consisting of both public and private clouds. Unsurprisingly, over the last 12 months, the majority of respondents had increased overall cloud spend — a substantial increase over the 2020 findings ...

May 11, 2022

As we all know, the drastic changes in the world have caused the workforce to take a hybrid approach over the last two years. A lot of that time, being fully remote. With the back and forth between home and office, employees need ways to stay productive and access useful information necessary to complete their daily work. The ability to obtain a holistic view of data relevant to the user and get answers to topics, no matter the worker's location, is crucial for a successful and efficient hybrid working environment ...