Understanding Observability Data's Impact Across an Organization
January 11, 2023

Tucker Callaway

Share this

As demand for digital services increases and distributed systems become more complex, organizations must collect and process a growing amount of observability data (logs, metrics, and traces). Site reliability engineers (SREs), developers, and security engineers use observability data to learn how their applications and environments are performing so they can successfully respond to issues and mitigate risk.

With use cases expanding across many business units, it's important for organizations to know how users in various roles use observability data. A new report from The Harris Poll and Mezmo explores this concept. Based on a survey of 300 SREs, developers, and security engineers in the US, the study digs into key pain points and how companies might use observability pipelines to help make decisions faster.

Observability Data Is a Part of Daily Usage

More than half of SREs, developers, and security engineers use observability data daily, with another third of people in each role using it two to three times per week. Typical machine data interaction looks different for each role. SREs focus on troubleshooting, analytics, and monitoring uptime; developers on troubleshooting and debugging; and security engineers on cybersecurity, firewall integrity, and threat detection.

The Amount of Data Is Escalating

Data volume is increasing considerably and becoming difficult to control as data is spread across many systems and apps. While respondents in all three roles use a median of four data sources to get their jobs done, SREs and developers often use three separate products to access that data, and security engineers use two. And over the last 12 months, developers and security engineers have seen a median of two new data sources being added, and SREs have seen three.

Adding new data sources and controlling the flow of data has become an overly complex process involving many different tools that don't integrate well and provide delayed insights. Organizations must harness all this data to make real-time business decisions because a slight delay can cause issues.

Difficult to Control Skyrocketing Costs

In addition to data volume, the three groups listed cost control as a top challenge. Specifically, 92% of SREs, 99% of developers, and 97% of security engineers say it's hard to manage the costs of collecting and storing data. High volume of data creates budget pressures across the organization as budgets are not increasing proportionally to the cost. Organizations must look for ways to extract more value from their telemetry data by making data available to wider teams for additional use cases. This requires free flow of usable telemetry data to any platform of choice.

Making Data Actionable with Observability Pipelines

Most professionals in all three roles agree that newly adopted technology, like observability pipelines, must integrate with existing data management platforms. When looking at observability pipelines to help better control and take action on data, all three roles report that supporting cloud data sources is essential. SREs and developers are also interested in making sure that cloud application data sources are supported, while SREs and security engineers need to be sure that there is firewall data source support. However, teams are not just looking for collecting data but need various transformations to add additional context to the data. They are looking for capabilities such as log transformations, sampling, enrichment, and augmentation to make data more meaningful and actionable.

As the report reveals, the importance of observability data is growing, but organizations are challenged with making this data actionable. Observability data pipelines are an emerging technology organizations can use to collect, transform, and route all this data to various teams for greater actionability. Once organizations can understand how different groups use this data, they'll be able to extract greater value for the business.

Tucker Callaway is CEO of Mezmo
Share this

The Latest

April 19, 2024

In MEAN TIME TO INSIGHT Episode 5, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses the network source of truth ...

April 18, 2024

A vast majority (89%) of organizations have rapidly expanded their technology in the past few years and three quarters (76%) say it's brought with it increased "chaos" that they have to manage, according to Situation Report 2024: Managing Technology Chaos from Software AG ...

April 17, 2024

In 2024 the number one challenge facing IT teams is a lack of skilled workers, and many are turning to automation as an answer, according to IT Trends: 2024 Industry Report ...

April 16, 2024

Organizations are continuing to embrace multicloud environments and cloud-native architectures to enable rapid transformation and deliver secure innovation. However, despite the speed, scale, and agility enabled by these modern cloud ecosystems, organizations are struggling to manage the explosion of data they create, according to The state of observability 2024: Overcoming complexity through AI-driven analytics and automation strategies, a report from Dynatrace ...

April 15, 2024

Organizations recognize the value of observability, but only 10% of them are actually practicing full observability of their applications and infrastructure. This is among the key findings from the recently completed Logz.io 2024 Observability Pulse Survey and Report ...

April 11, 2024

Businesses must adopt a comprehensive Internet Performance Monitoring (IPM) strategy, says Enterprise Management Associates (EMA), a leading IT analyst research firm. This strategy is crucial to bridge the significant observability gap within today's complex IT infrastructures. The recommendation is particularly timely, given that 99% of enterprises are expanding their use of the Internet as a primary connectivity conduit while facing challenges due to the inefficiency of multiple, disjointed monitoring tools, according to Modern Enterprises Must Boost Observability with Internet Performance Monitoring, a new report from EMA and Catchpoint ...

April 10, 2024

Choosing the right approach is critical with cloud monitoring in hybrid environments. Otherwise, you may drive up costs with features you don’t need and risk diminishing the visibility of your on-premises IT ...

April 09, 2024

Consumers ranked the marketing strategies and missteps that most significantly impact brand trust, which 73% say is their biggest motivator to share first-party data, according to The Rules of the Marketing Game, a 2023 report from Pantheon ...

April 08, 2024

Digital experience monitoring is the practice of monitoring and analyzing the complete digital user journey of your applications, websites, APIs, and other digital services. It involves tracking the performance of your web application from the perspective of the end user, providing detailed insights on user experience, app performance, and customer satisfaction ...

April 04, 2024
Modern organizations race to launch their high-quality cloud applications as soon as possible. On the other hand, time to market also plays an essential role in determining the application's success. However, without effective testing, it's hard to be confident in the final product ...