Observability Has a Complexity Problem
June 01, 2023

Dotan Horovits
Logz.io

Share this

The journey of maturing observability practices for users entails navigating peaks and valleys. Users have clearly witnessed the maturation of their monitoring capabilities, embraced DevOps practices, and adopted cloud and cloud-native technologies.

Notwithstanding that, we witness the gradual increase of the Mean Time To Recovery (MTTR) for production issues year over year. In this year's (2023) DevOps Pulse survey, conducted by Logz.io, 73% of respondents stated that it took multiple hours on average to resolve production issues.


In comparison, in 2022 only 64% of respondents said the same and only 47% cited this in the year prior. Why is that? The answer seems to lie within the growing overall system complexity, due to the adoption of Kubernetes and cloud-native technologies and practices, as practitioners reported on the survey.

The findings from the report indicate that DevOps practices show signs of maturity and growth within organizations. One such example is that 45% of users have fully adopted and embraced DevOps practices — a 7% increase compared to 38% the year before. Similarly, there has been significant uptake of cloud adoption, as evidenced by the fact that 78% of organizations have either partially or fully transitioned to the cloud this year.

However, as maturity advances, a notable trend is surfacing: the mean time to recovery (MTTR) for production issues has been steadily increasing year over year, as witnessed in the above survey results. A leading cause for the substantial increase in MTTR for organizations is the rising adoption of cloud-native technologies and their intricacies.

Technologies such as Kubernetes generate abundant and complex data, making it difficult to monitor and troubleshoot. As such, these technologies were cited by 46% of respondents as the most difficult obstacle for organizations to gain full observability of their environment.

Kubernetes specifically stands out as a challenging tool for observability users, both in terms of monitoring and running it in production. Over 40% of survey responses stated that monitoring and gaining full observability is one of the key challenges of running Kubernetes. This statistic has grown from 2022, in which only 31% of respondents cited this exact figure. Outside of observability, respondents also noted challenges with Kubernetes security and cluster networking functionality.

In the 2023 report, security was highlighted as another area of major focus for DevOps practitioners, with over 30% of respondents stating that a shared model is used for security and observability. As these teams take on the responsibility of security, they are running into issues with centralizing security data and outlining clear roles and responsibilities for their teams.

One of the most challenging factors of security implementation is tool integration for cloud-native technologies. In fact, nearly 50% of respondents highlighted that implementing security for Kubernetes is the most difficult aspect of running it in production. The dynamic environment, distributed architecture, and overall complexity and scale of these technologies can further compound the difficulty of implementing comprehensive security practices.

The escalating complexity within IT systems also led to a surge in telemetry data volumes, consequently driving up the expenses associated with observability. In response, organizations have implemented various strategies to mitigate observability costs, including the adoption of open source tools as part of their overall approach.

The report shows that roughly 53% of this year's respondents stated that half or more of their observability tools are open source. The uptick from last year is notable, as only 37% of 2022 survey respondents indicated that half or more of their tools were open source. Respondents cited the lower cost of ownership (36%), ease of integration (47%), and benefits of the open source community (33%) as some of the key reasons for open source tool adoption.

As organizations face increasing complexity in observability, collecting and gathering insights from observability practitioners remains paramount. By utilizing their expertise, organizations can navigate the evolving landscape and make informed decisions to optimize their observability practices.

Dotan Horovits is Principal Developer Advocate at Logz.io
Share this

The Latest

April 24, 2024

Over the last 20 years Digital Employee Experience has become a necessity for companies committed to digital transformation and improving IT experiences. In fact, by 2025, more than 50% of IT organizations will use digital employee experience to prioritize and measure digital initiative success ...

April 23, 2024

While most companies are now deploying cloud-based technologies, the 2024 Secure Cloud Networking Field Report from Aviatrix found that there is a silent struggle to maximize value from those investments. Many of the challenges organizations have faced over the past several years have evolved, but continue today ...

April 22, 2024

In our latest research, Cisco's The App Attention Index 2023: Beware the Application Generation, 62% of consumers report their expectations for digital experiences are far higher than they were two years ago, and 64% state they are less forgiving of poor digital services than they were just 12 months ago ...

April 19, 2024

In MEAN TIME TO INSIGHT Episode 5, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses the network source of truth ...

April 18, 2024

A vast majority (89%) of organizations have rapidly expanded their technology in the past few years and three quarters (76%) say it's brought with it increased "chaos" that they have to manage, according to Situation Report 2024: Managing Technology Chaos from Software AG ...

April 17, 2024

In 2024 the number one challenge facing IT teams is a lack of skilled workers, and many are turning to automation as an answer, according to IT Trends: 2024 Industry Report ...

April 16, 2024

Organizations are continuing to embrace multicloud environments and cloud-native architectures to enable rapid transformation and deliver secure innovation. However, despite the speed, scale, and agility enabled by these modern cloud ecosystems, organizations are struggling to manage the explosion of data they create, according to The state of observability 2024: Overcoming complexity through AI-driven analytics and automation strategies, a report from Dynatrace ...

April 15, 2024

Organizations recognize the value of observability, but only 10% of them are actually practicing full observability of their applications and infrastructure. This is among the key findings from the recently completed Logz.io 2024 Observability Pulse Survey and Report ...

April 11, 2024

Businesses must adopt a comprehensive Internet Performance Monitoring (IPM) strategy, says Enterprise Management Associates (EMA), a leading IT analyst research firm. This strategy is crucial to bridge the significant observability gap within today's complex IT infrastructures. The recommendation is particularly timely, given that 99% of enterprises are expanding their use of the Internet as a primary connectivity conduit while facing challenges due to the inefficiency of multiple, disjointed monitoring tools, according to Modern Enterprises Must Boost Observability with Internet Performance Monitoring, a new report from EMA and Catchpoint ...

April 10, 2024

Choosing the right approach is critical with cloud monitoring in hybrid environments. Otherwise, you may drive up costs with features you don’t need and risk diminishing the visibility of your on-premises IT ...