As demand for digital services increases and distributed systems become more complex, organizations must collect and process a growing amount of observability data (logs, metrics, and traces). Site reliability engineers (SREs), developers, and security engineers use observability data to learn how their applications and environments are performing so they can successfully respond to issues and mitigate risk.
With use cases expanding across many business units, it's important for organizations to know how users in various roles use observability data. A new report from The Harris Poll and Mezmo explores this concept. Based on a survey of 300 SREs, developers, and security engineers in the US, the study digs into key pain points and how companies might use observability pipelines to help make decisions faster.
Observability Data Is a Part of Daily Usage
More than half of SREs, developers, and security engineers use observability data daily, with another third of people in each role using it two to three times per week. Typical machine data interaction looks different for each role. SREs focus on troubleshooting, analytics, and monitoring uptime; developers on troubleshooting and debugging; and security engineers on cybersecurity, firewall integrity, and threat detection.
The Amount of Data Is Escalating
Data volume is increasing considerably and becoming difficult to control as data is spread across many systems and apps. While respondents in all three roles use a median of four data sources to get their jobs done, SREs and developers often use three separate products to access that data, and security engineers use two. And over the last 12 months, developers and security engineers have seen a median of two new data sources being added, and SREs have seen three.
Adding new data sources and controlling the flow of data has become an overly complex process involving many different tools that don't integrate well and provide delayed insights. Organizations must harness all this data to make real-time business decisions because a slight delay can cause issues.
Difficult to Control Skyrocketing Costs
In addition to data volume, the three groups listed cost control as a top challenge. Specifically, 92% of SREs, 99% of developers, and 97% of security engineers say it's hard to manage the costs of collecting and storing data. High volume of data creates budget pressures across the organization as budgets are not increasing proportionally to the cost. Organizations must look for ways to extract more value from their telemetry data by making data available to wider teams for additional use cases. This requires free flow of usable telemetry data to any platform of choice.
Making Data Actionable with Observability Pipelines
Most professionals in all three roles agree that newly adopted technology, like observability pipelines, must integrate with existing data management platforms. When looking at observability pipelines to help better control and take action on data, all three roles report that supporting cloud data sources is essential. SREs and developers are also interested in making sure that cloud application data sources are supported, while SREs and security engineers need to be sure that there is firewall data source support. However, teams are not just looking for collecting data but need various transformations to add additional context to the data. They are looking for capabilities such as log transformations, sampling, enrichment, and augmentation to make data more meaningful and actionable.
As the report reveals, the importance of observability data is growing, but organizations are challenged with making this data actionable. Observability data pipelines are an emerging technology organizations can use to collect, transform, and route all this data to various teams for greater actionability. Once organizations can understand how different groups use this data, they'll be able to extract greater value for the business.
The Latest
The 2024 State of the Data Center Report from CoreSite shows that although C-suite confidence in the economy remains high, a VUCA (volatile, uncertain, complex, ambiguous) environment has many business leaders proceeding with caution when it comes to their IT and data ecosystems, with an emphasis on cost control and predictability, flexibility and risk management ...
In June, New Relic published the State of Observability for Energy and Utilities Report to share insights, analysis, and data on the impact of full-stack observability software in energy and utilities organizations' service capabilities. Here are eight key takeaways from the report ...
The rapid rise of generative AI (GenAI) has caught everyone's attention, leaving many to wonder if the technology's impact will live up to the immense hype. A recent survey by Alteryx provides valuable insights into the current state of GenAI adoption, revealing a shift from inflated expectations to tangible value realization across enterprises ... Here are five key takeaways that underscore GenAI's progression from hype to real-world impact ...
As software development grows more intricate, the challenge for observability engineers tasked with ensuring optimal system performance becomes more daunting. Current methodologies are struggling to keep pace, with the annual Observability Pulse surveys indicating a rise in Mean Time to Remediation (MTTR). According to this survey, only a small fraction of organizations, around 10%, achieve full observability today. Generative AI, however, promises to significantly move the needle ...
While nearly all data leaders surveyed are building generative AI applications, most don't believe their data estate is actually prepared to support them, according to the State of Reliable AI report from Monte Carlo Data ...
Enterprises are putting a lot of effort into improving the digital employee experience (DEX), which has become essential to both improving organizational performance and attracting and retaining talented workers. But to date, most efforts to deliver outstanding DEX have focused on people working with laptops, PCs, or thin clients. Employees on the frontlines, using mobile devices to handle logistics ... have been largely overlooked ...
The average customer-facing incident takes nearly three hours to resolve (175 minutes) while the estimated cost of downtime is $4,537 per minute, meaning each incident can cost nearly $794,000, according to new research from PagerDuty ...
In MEAN TIME TO INSIGHT Episode 8, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses AutoCon with the conference founders Scott Robohn and Chris Grundemann ...
Numerous vendors and service providers have recently embraced the NaaS concept, yet there is still no industry consensus on its definition or the types of networks it involves. Furthermore, providers have varied in how they define the NaaS service delivery model. I conducted research for a new report, Network as a Service: Understanding the Cloud Consumption Model in Networking, to refine the concept of NaaS and reduce buyer confusion over what it is and how it can offer value ...
