As demand for digital services increases and distributed systems become more complex, organizations must collect and process a growing amount of observability data (logs, metrics, and traces). Site reliability engineers (SREs), developers, and security engineers use observability data to learn how their applications and environments are performing so they can successfully respond to issues and mitigate risk.
With use cases expanding across many business units, it's important for organizations to know how users in various roles use observability data. A new report from The Harris Poll and Mezmo explores this concept. Based on a survey of 300 SREs, developers, and security engineers in the US, the study digs into key pain points and how companies might use observability pipelines to help make decisions faster.
Observability Data Is a Part of Daily Usage
More than half of SREs, developers, and security engineers use observability data daily, with another third of people in each role using it two to three times per week. Typical machine data interaction looks different for each role. SREs focus on troubleshooting, analytics, and monitoring uptime; developers on troubleshooting and debugging; and security engineers on cybersecurity, firewall integrity, and threat detection.
The Amount of Data Is Escalating
Data volume is increasing considerably and becoming difficult to control as data is spread across many systems and apps. While respondents in all three roles use a median of four data sources to get their jobs done, SREs and developers often use three separate products to access that data, and security engineers use two. And over the last 12 months, developers and security engineers have seen a median of two new data sources being added, and SREs have seen three.
Adding new data sources and controlling the flow of data has become an overly complex process involving many different tools that don't integrate well and provide delayed insights. Organizations must harness all this data to make real-time business decisions because a slight delay can cause issues.
Difficult to Control Skyrocketing Costs
In addition to data volume, the three groups listed cost control as a top challenge. Specifically, 92% of SREs, 99% of developers, and 97% of security engineers say it's hard to manage the costs of collecting and storing data. High volume of data creates budget pressures across the organization as budgets are not increasing proportionally to the cost. Organizations must look for ways to extract more value from their telemetry data by making data available to wider teams for additional use cases. This requires free flow of usable telemetry data to any platform of choice.
Making Data Actionable with Observability Pipelines
Most professionals in all three roles agree that newly adopted technology, like observability pipelines, must integrate with existing data management platforms. When looking at observability pipelines to help better control and take action on data, all three roles report that supporting cloud data sources is essential. SREs and developers are also interested in making sure that cloud application data sources are supported, while SREs and security engineers need to be sure that there is firewall data source support. However, teams are not just looking for collecting data but need various transformations to add additional context to the data. They are looking for capabilities such as log transformations, sampling, enrichment, and augmentation to make data more meaningful and actionable.
As the report reveals, the importance of observability data is growing, but organizations are challenged with making this data actionable. Observability data pipelines are an emerging technology organizations can use to collect, transform, and route all this data to various teams for greater actionability. Once organizations can understand how different groups use this data, they'll be able to extract greater value for the business.
The Latest
Developers need a tool that can be portable and vendor agnostic, given the advent of microservices. It may be clear an issue is occurring; what may not be clear is if it's part of a distributed system or the app itself. Enter OpenTelemetry, commonly referred to as OTel, an open-source framework that provides a standardized way of collecting and exporting telemetry data (logs, metrics, and traces) from cloud-native software ...
As SLOs grow in popularity their usage is becoming more mature. For example, 82% of respondents intend to increase their use of SLOs, and 96% have mapped SLOs directly to their business operations or already have a plan to, according to The State of Service Level Objectives 2023 from Nobl9 ...
Observability has matured beyond its early adopter position and is now foundational for modern enterprises to achieve full visibility into today's complex technology environments, according to The State of Observability 2023, a report released by Splunk in collaboration with Enterprise Strategy Group ...
Before network engineers even begin the automation process, they tend to start with preconceived notions that oftentimes, if acted upon, can hinder the process. To prevent that from happening, it's important to identify and dispel a few common misconceptions currently out there and how networking teams can overcome them. So, let's address the three most common network automation myths ...
Many IT organizations apply AI/ML and AIOps technology across domains, correlating insights from the various layers of IT infrastructure and operations. However, Enterprise Management Associates (EMA) has observed significant interest in applying these AI technologies narrowly to network management, according to a new research report, titled AI-Driven Networks: Leveling Up Network Management with AI/ML and AIOps ...
When it comes to system outages, AIOps solutions with the right foundation can help reduce the blame game so the right teams can spend valuable time restoring the impacted services rather than improving their MTTI score (mean time to innocence). In fact, much of today's innovation around ChatGPT-style algorithms can be used to significantly improve the triage process and user experience ...
Gartner identified the top 10 data and analytics (D&A) trends for 2023 that can guide D&A leaders to create new sources of value by anticipating change and transforming extreme uncertainty into new business opportunities ...
The only way for companies to stay competitive is to modernize applications, yet there's no denying that bringing apps into the modern era can be challenging ... Let's look at a few ways to modernize applications and consider what new obstacles and opportunities 2023 presents ...
As online penetration grows, retailers' profits are shrinking — with the cost of serving customers anytime, anywhere, at any speed not bringing in enough topline growth to best monetize even existing investments in technology, systems, infrastructure, and people, let alone new investments, according to Digital-First Retail: Turning Profit Destruction into Customer and Shareholder Value, a new report from AlixPartners and World Retail Congress ...
