With input from industry experts — both analysts and vendors — this 8-part blog series will explore what is driving the convergence of observability and security, the challenges and advantages, and how it may transform the IT landscape.
The experts have all agreed that security teams can gain great benefits from utilizing observability data. But does this mean security and observability tools should be integrated, or even combined?
Chaim Mazal, Chief Security Officer at Gigamon says the answer to this question is a resounding yes.
"Observability tools are powerful at aiding organizations in identifying security anomalies and pinpointing performance bottlenecks at the application layer. Logging provides foundational visibility into the applications running across their hybrid cloud infrastructure. But, as threat actors apply increasingly sophisticated techniques to breach an organization's technology environment, network-derived intelligence is vital to detecting lateral movement should a threat actor successfully gain access. If successful, threat actors can move across an organization undetected seeking to exploit proprietary or confidential information for financial gain. It's only by integrating logging with network-derived intelligence that IT organizations gain deep observability across their hybrid and multi-cloud infrastructure to detect previously unseen threats, deliver defense in depth, and complete performance management."
"Security and observability tools should absolutely be combined," says Prashant Prahlad, VP of Cloud Security Products at Datadog. "Traditional security solutions are targeted solely at security professionals. But, while security pros are responsible for finding vulnerabilities, misconfigurations and risks, developers are the ones responsible for fixing them. This is especially true when it comes to cloud security as most of the remediation requires working with a DevOps team."
"For example, security can't change the configuration of a s3 bucket without the risk of breaking something in production, which is why it is critical to have the DevOps and security teams aligned," Prahlad continues. "Because traditional solutions are aimed at security pros — who traditionally managed network security — they don't provide the shared context that organizations need to fix issues quickly and efficiently. A unified platform for observability and security is needed so that developers can work directly with security pros to visualize how threats and vulnerabilities are impacting their cloud environments and prioritize fixes faster. This approach breaks down silos between DevOps and security teams and creates the shared context they need to secure cloud environments."
However, convergence is difficult to prescribe, cautions Asaf Yigal, CTO of Logz.io. "Literally every organization is going to require a unique approach based on its specific makeup, whether this is a large or mature org with a lot of people given responsibility for dev, ops, security or even platform engineering. The platforms and tooling need to match the people and process, or evolve with it."
"At the same time, we know for sure that there is a huge benefit in bringing together the relevant data, either to be actioned centrally, say in a smaller shop with only a few people responsible for DevSecOps, or to be communicated across teams in a larger org with multiple groups spanning the entire landscape."
"There's also the huge benefit of tapping into a common data set," Yigal adds, "namely logs, and using a shared platform; this is for a lot of reasons, from using a common language for querying engines, etc., to having fewer vendors to manage. This is why nearly every major observability vendor also markets a SIEM — it just makes a lot of sense."
Adam Hert, Director of Product at Riverbed agrees that tools should be integrated, but says, "Security and observability tools don't need to be combined. Some teams are trying to do this, but it does not make sense for organizations to do so, largely because you have two teams focused on very different goals. Security teams are tracking down threats, while observability teams are focused on making the enterprise more efficient and effective. Observability and security tools don't need to be combined, but they need to be able to integrate so that security tools can ask questions on the observability data."
Convergence Saves Money
"On the one hand, there's an argument to be made that security and observability tools should not be combined as most traditional monitoring and logging tools get bogged down by the strict retention requirements that are required by security tools for regulatory and compliance purposes of their products," says Jam Leomi, Lead Security Engineer at Honeycomb. "Applying that type of forensic-level, unsampled logging to observability tools would both be costly in terms of expense and speed, but also very inefficient."
"However, combining security and observability tools does have some functionality as it would cut down on costs drastically while creating an open field for collaboration between security, engineering, and the business to address incident response and the overall security posture assessment — generally, because there's a lot of natural crossover between the goals and initiatives for security and observability teams," Leomi continues. "For example, SOC2 controls require teams to keep up with performance metrics which observability platforms can offer fresh insights into data, even without having the granularity of each forensic row."
Colin Fallwell, Field CTO of Sumo Logic agrees that any time teams can unify data and interfaces for managing observability and security, it's a win, both in reducing the cost of ownership as well as ROI in uniformity and standards. "DevOps and SecOps need the same data, so why have two collection pipelines, for separate tools, capturing the same telemetric data? It really doesn't make sense. This redundancy is expensive and unnecessary."
"Additionally, there's a shortage of specialized security talent with the skillset needed to shift security left," Leomi from Honeycomb informs. "Organizations are under increasing pressure to reduce spend without sacrificing ability, so naturally, they look for tools that can perform multiple functions like the ability to observe application performance while also being able to identify security vulnerabilities."
"Further exacerbating this trend is the scarcity of security talent needed to drive and meet security initiatives," Leomi adds. "This has driven organizations to rely on what they have, which is often product and platform engineering departments that are already using a tool for observability and one that can provide a good enough starting point for security."
Navigating observability pricing models can be compared to solving a perplexing puzzle which includes financial variables and contractual intricacies. Predicting all potential costs in advance becomes an elusive endeavor, exemplified by a recent eye-popping $65 million observability bill ...
Generative AI may be a great tool for the enterprise to help drive further innovation and meaningful work, but it also runs the risk of generating massive amounts of spam that will counteract its intended benefits. From increased AI spam bots to data maintenance due to large volumes of outputs, enterprise AI applications can create a cascade of issues that end up detracting from productivity gains ...
A long-running study of DevOps practices ... suggests that any historical gains in MTTR reduction have now plateaued. For years now, the time it takes to restore services has stayed about the same: less than a day for high performers but up to a week for middle-tier teams and up to a month for laggards. The fact that progress is flat despite big investments in people, tools and automation is a cause for concern ...
Companies implementing observability benefit from increased operational efficiency, faster innovation, and better business outcomes overall, according to 2023 IT Trends Report: Lessons From Observability Leaders, a report from SolarWinds ...
Customer loyalty is changing as retailers get increasingly competitive. More than 75% of consumers say they would end business with a company after a single bad customer experience. This means that just one price discrepancy, inventory mishap or checkout issue in a physical or digital store, could have customers running out to the next store that can provide them with better service. Retailers must be able to predict business outages in advance, and act proactively before an incident occurs, impacting customer experience ...
Earlier this year, New Relic conducted a study on observability ... The 2023 Observability Forecast reveals observability's impact on the lives of technical professionals and businesses' bottom lines. Here are 10 key takeaways from the forecast ...
Only 33% of executives are "very confident" in their ability to operate in a public cloud environment, according to the 2023 State of CloudOps report from NetApp. This represents an increase from 2022 when only 21% reported feeling very confident ...