Skip to main content

Understanding Observability Data's Impact Across an Organization

Tucker Callaway
Mezmo

As demand for digital services increases and distributed systems become more complex, organizations must collect and process a growing amount of observability data (logs, metrics, and traces). Site reliability engineers (SREs), developers, and security engineers use observability data to learn how their applications and environments are performing so they can successfully respond to issues and mitigate risk.

With use cases expanding across many business units, it's important for organizations to know how users in various roles use observability data. A new report from The Harris Poll and Mezmo explores this concept. Based on a survey of 300 SREs, developers, and security engineers in the US, the study digs into key pain points and how companies might use observability pipelines to help make decisions faster.

Observability Data Is a Part of Daily Usage

More than half of SREs, developers, and security engineers use observability data daily, with another third of people in each role using it two to three times per week. Typical machine data interaction looks different for each role. SREs focus on troubleshooting, analytics, and monitoring uptime; developers on troubleshooting and debugging; and security engineers on cybersecurity, firewall integrity, and threat detection.

The Amount of Data Is Escalating

Data volume is increasing considerably and becoming difficult to control as data is spread across many systems and apps. While respondents in all three roles use a median of four data sources to get their jobs done, SREs and developers often use three separate products to access that data, and security engineers use two. And over the last 12 months, developers and security engineers have seen a median of two new data sources being added, and SREs have seen three.

Adding new data sources and controlling the flow of data has become an overly complex process involving many different tools that don't integrate well and provide delayed insights. Organizations must harness all this data to make real-time business decisions because a slight delay can cause issues.

Difficult to Control Skyrocketing Costs

In addition to data volume, the three groups listed cost control as a top challenge. Specifically, 92% of SREs, 99% of developers, and 97% of security engineers say it's hard to manage the costs of collecting and storing data. High volume of data creates budget pressures across the organization as budgets are not increasing proportionally to the cost. Organizations must look for ways to extract more value from their telemetry data by making data available to wider teams for additional use cases. This requires free flow of usable telemetry data to any platform of choice.

Making Data Actionable with Observability Pipelines

Most professionals in all three roles agree that newly adopted technology, like observability pipelines, must integrate with existing data management platforms. When looking at observability pipelines to help better control and take action on data, all three roles report that supporting cloud data sources is essential. SREs and developers are also interested in making sure that cloud application data sources are supported, while SREs and security engineers need to be sure that there is firewall data source support. However, teams are not just looking for collecting data but need various transformations to add additional context to the data. They are looking for capabilities such as log transformations, sampling, enrichment, and augmentation to make data more meaningful and actionable.

As the report reveals, the importance of observability data is growing, but organizations are challenged with making this data actionable. Observability data pipelines are an emerging technology organizations can use to collect, transform, and route all this data to various teams for greater actionability. Once organizations can understand how different groups use this data, they'll be able to extract greater value for the business.

Tucker Callaway is CEO of Mezmo

Hot Topics

The Latest

As businesses increasingly rely on high-performance applications to deliver seamless user experiences, the demand for fast, reliable, and scalable data storage systems has never been greater. Redis — an open-source, in-memory data structure store — has emerged as a popular choice for use cases ranging from caching to real-time analytics. But with great performance comes the need for vigilant monitoring ...

Kubernetes was not initially designed with AI's vast resource variability in mind, and the rapid rise of AI has exposed Kubernetes limitations, particularly when it comes to cost and resource efficiency. Indeed, AI workloads differ from traditional applications in that they require a staggering amount and variety of compute resources, and their consumption is far less consistent than traditional workloads ... Considering the speed of AI innovation, teams cannot afford to be bogged down by these constant infrastructure concerns. A solution is needed ...

AI is the catalyst for significant investment in data teams as enterprises require higher-quality data to power their AI applications, according to the State of Analytics Engineering Report from dbt Labs ...

Misaligned architecture can lead to business consequences, with 93% of respondents reporting negative outcomes such as service disruptions, high operational costs and security challenges ...

A Gartner analyst recently suggested that GenAI tools could create 25% time savings for network operational teams. Where might these time savings come from? How are GenAI tools helping NetOps teams today, and what other tasks might they take on in the future as models continue improving? In general, these savings come from automating or streamlining manual NetOps tasks ...

IT and line-of-business teams are increasingly aligned in their efforts to close the data gap and drive greater collaboration to alleviate IT bottlenecks and offload growing demands on IT teams, according to The 2025 Automation Benchmark Report: Insights from IT Leaders on Enterprise Automation & the Future of AI-Driven Businesses from Jitterbit ...

A large majority (86%) of data management and AI decision makers cite protecting data privacy as a top concern, with 76% of respondents citing ROI on data privacy and AI initiatives across their organization, according to a new Harris Poll from Collibra ...

According to Gartner, Inc. the following six trends will shape the future of cloud over the next four years, ultimately resulting in new ways of working that are digital in nature and transformative in impact ...

2020 was the equivalent of a wedding with a top-shelf open bar. As businesses scrambled to adjust to remote work, digital transformation accelerated at breakneck speed. New software categories emerged overnight. Tech stacks ballooned with all sorts of SaaS apps solving ALL the problems — often with little oversight or long-term integration planning, and yes frequently a lot of duplicated functionality ... But now the music's faded. The lights are on. Everyone from the CIO to the CFO is checking the bill. Welcome to the Great SaaS Hangover ...

Regardless of OpenShift being a scalable and flexible software, it can be a pain to monitor since complete visibility into the underlying operations is not guaranteed ... To effectively monitor an OpenShift environment, IT administrators should focus on these five key elements and their associated metrics ...

Understanding Observability Data's Impact Across an Organization

Tucker Callaway
Mezmo

As demand for digital services increases and distributed systems become more complex, organizations must collect and process a growing amount of observability data (logs, metrics, and traces). Site reliability engineers (SREs), developers, and security engineers use observability data to learn how their applications and environments are performing so they can successfully respond to issues and mitigate risk.

With use cases expanding across many business units, it's important for organizations to know how users in various roles use observability data. A new report from The Harris Poll and Mezmo explores this concept. Based on a survey of 300 SREs, developers, and security engineers in the US, the study digs into key pain points and how companies might use observability pipelines to help make decisions faster.

Observability Data Is a Part of Daily Usage

More than half of SREs, developers, and security engineers use observability data daily, with another third of people in each role using it two to three times per week. Typical machine data interaction looks different for each role. SREs focus on troubleshooting, analytics, and monitoring uptime; developers on troubleshooting and debugging; and security engineers on cybersecurity, firewall integrity, and threat detection.

The Amount of Data Is Escalating

Data volume is increasing considerably and becoming difficult to control as data is spread across many systems and apps. While respondents in all three roles use a median of four data sources to get their jobs done, SREs and developers often use three separate products to access that data, and security engineers use two. And over the last 12 months, developers and security engineers have seen a median of two new data sources being added, and SREs have seen three.

Adding new data sources and controlling the flow of data has become an overly complex process involving many different tools that don't integrate well and provide delayed insights. Organizations must harness all this data to make real-time business decisions because a slight delay can cause issues.

Difficult to Control Skyrocketing Costs

In addition to data volume, the three groups listed cost control as a top challenge. Specifically, 92% of SREs, 99% of developers, and 97% of security engineers say it's hard to manage the costs of collecting and storing data. High volume of data creates budget pressures across the organization as budgets are not increasing proportionally to the cost. Organizations must look for ways to extract more value from their telemetry data by making data available to wider teams for additional use cases. This requires free flow of usable telemetry data to any platform of choice.

Making Data Actionable with Observability Pipelines

Most professionals in all three roles agree that newly adopted technology, like observability pipelines, must integrate with existing data management platforms. When looking at observability pipelines to help better control and take action on data, all three roles report that supporting cloud data sources is essential. SREs and developers are also interested in making sure that cloud application data sources are supported, while SREs and security engineers need to be sure that there is firewall data source support. However, teams are not just looking for collecting data but need various transformations to add additional context to the data. They are looking for capabilities such as log transformations, sampling, enrichment, and augmentation to make data more meaningful and actionable.

As the report reveals, the importance of observability data is growing, but organizations are challenged with making this data actionable. Observability data pipelines are an emerging technology organizations can use to collect, transform, and route all this data to various teams for greater actionability. Once organizations can understand how different groups use this data, they'll be able to extract greater value for the business.

Tucker Callaway is CEO of Mezmo

Hot Topics

The Latest

As businesses increasingly rely on high-performance applications to deliver seamless user experiences, the demand for fast, reliable, and scalable data storage systems has never been greater. Redis — an open-source, in-memory data structure store — has emerged as a popular choice for use cases ranging from caching to real-time analytics. But with great performance comes the need for vigilant monitoring ...

Kubernetes was not initially designed with AI's vast resource variability in mind, and the rapid rise of AI has exposed Kubernetes limitations, particularly when it comes to cost and resource efficiency. Indeed, AI workloads differ from traditional applications in that they require a staggering amount and variety of compute resources, and their consumption is far less consistent than traditional workloads ... Considering the speed of AI innovation, teams cannot afford to be bogged down by these constant infrastructure concerns. A solution is needed ...

AI is the catalyst for significant investment in data teams as enterprises require higher-quality data to power their AI applications, according to the State of Analytics Engineering Report from dbt Labs ...

Misaligned architecture can lead to business consequences, with 93% of respondents reporting negative outcomes such as service disruptions, high operational costs and security challenges ...

A Gartner analyst recently suggested that GenAI tools could create 25% time savings for network operational teams. Where might these time savings come from? How are GenAI tools helping NetOps teams today, and what other tasks might they take on in the future as models continue improving? In general, these savings come from automating or streamlining manual NetOps tasks ...

IT and line-of-business teams are increasingly aligned in their efforts to close the data gap and drive greater collaboration to alleviate IT bottlenecks and offload growing demands on IT teams, according to The 2025 Automation Benchmark Report: Insights from IT Leaders on Enterprise Automation & the Future of AI-Driven Businesses from Jitterbit ...

A large majority (86%) of data management and AI decision makers cite protecting data privacy as a top concern, with 76% of respondents citing ROI on data privacy and AI initiatives across their organization, according to a new Harris Poll from Collibra ...

According to Gartner, Inc. the following six trends will shape the future of cloud over the next four years, ultimately resulting in new ways of working that are digital in nature and transformative in impact ...

2020 was the equivalent of a wedding with a top-shelf open bar. As businesses scrambled to adjust to remote work, digital transformation accelerated at breakneck speed. New software categories emerged overnight. Tech stacks ballooned with all sorts of SaaS apps solving ALL the problems — often with little oversight or long-term integration planning, and yes frequently a lot of duplicated functionality ... But now the music's faded. The lights are on. Everyone from the CIO to the CFO is checking the bill. Welcome to the Great SaaS Hangover ...

Regardless of OpenShift being a scalable and flexible software, it can be a pain to monitor since complete visibility into the underlying operations is not guaranteed ... To effectively monitor an OpenShift environment, IT administrators should focus on these five key elements and their associated metrics ...