With input from industry experts — both analysts and vendors — this 8-part blog series will explore what is driving the convergence of observability and security, the challenges and advantages, and how it may transform the IT landscape.
If you have already read the previous blogs in this series exploring the convergence of observability and security, the challenges will not surprise you. The experts cite compatibility of tools, teams and cultures as challenges to convergence, among others.
The following are some of the challenges experts see with achieving convergence:
Aversion to Change
Colin Fallwell, Field CTO of Sumo Logic: "Probably the biggest challenge comes down to one word. Change. Most people don't like change, much less transformation. DevSecOps requires change, it requires thinking about transformation as a continuous process that is never-ending. Up until now, this kind of transformation really could not happen, but with the rise of the Cloud Native Computing Foundation, the proliferation of open standards, and the mass adoption of OSS tooling like OpenTelemetry, and the need for proprietary agents for collecting telemetry are at an end, and with them the siloes of data."
Prashant Prahlad, VP of Cloud Security Products at Datadog: "The biggest roadblock to the convergence of security and observability is culture. Security teams need to be able to trust observability teams with product security and still be able to get the visibility they need as a failsafe."
Mike Loukides, VP of Emerging Tech Content at O'Reilly Media: "I think the major challenges will be the ones we've had all along. Management wants to deliver a new version on April 1. Development is under the gun to release. Ops is under the gun to deploy. And you'll still have security experts saying: Let's make sure we didn't take any shortcuts writing the code; let's make sure we're tracing the right things. It would be nice if this conflict would go away, but I don't think it will. Not now, not ever. However, putting security and ops teams in the same group will help."
Kirsten Newcomer, Director, Cloud and DevSecOps Strategy at Red Hat: "The purchasing decision and budgets for observability and security may be in different organizations."
Buddy Brewer, Chief Product Officer at Mezmo: "Currently, many organizations unintentionally lock data in silos that only certain teams can access, which often means DevOps and SecOps teams are either not getting the right data or implementing their individual solutions to get data from the same sources. While converging security and observability will make data significantly more actionable, organizations will be met with challenges with getting the data in the correct formats to be used by different tools they may need. In addition, they must make sure that they are adhering to regulations such as GDPR and CCPA and handle personal identifiable information (PII) properly."
Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at Enterprise Management Associates (EMA) outlines several challenges to convergence. "First, the teams have separate tools with separate tool silos. Often, when these groups come together, they find the quality of the data collected by the other silo's tools are of poor quality. It's in a format that is useless to them, for instance. Also, there is no authoritative source of data. Both groups have their own data stores that represent the same truth about infrastructure and services, but the data disagrees with each other due to variations and data granularity, time stamping, etc."
"Neither group wants to give up control of tool strategy," McGillicuddy continues. "They're married to their individual tools. Which one will blink and give up their tool in favor of the other group's tool?"
Use the player or download the MP3 below to listen to EMA-APMdigest Podcast Episode 2 — Shamus McGillicuddy talks about Network Observability, the convergence of observability and security, and more.
"We have a lot of work to do to make the tools work properly, so this is not an easy integration – largely because the observability tools were designed for observability. They were not designed for security purposes," adds Adam Hert, Director of Product at Riverbed.
Ajit Sancheti, GM, Falcon LogScale at CrowdStrike: "Legacy logging and event management tools may not provide the scale or the performance to ingest all data, which leads to ingest backlogs and sluggish search speed. Organizations should carefully evaluate logging products before attempting to collect all security and observability data in one tool."
Jam Leomi, Lead Security Engineer at Honeycomb: "The heart of the challenge in converging the two goes back to the culture shift we're seeing in security. A lot of today's practitioners are stuck in compliance practices or philosophies that are 30+ years old. As technology evolves, our security approach has to shift. This creates an opportunity to really connect security with the overall bottom line of the business instead of just as an afterthought. Observability as a tool and practice has the power to do a lot of the heavy lifting toward this goal, enabling a higher level of efficiency, security, and privacy."
Kirsten Newcomer from Red Hat: "Some security data is not appropriate for sharing with all team members who need to consume observability data."
Security Experts are hard to find
Prashant Prahlad of Datadog: "Security experts are hard to find and take time to train within DevOps teams, so implementing DevSecOps is a long-term investment."
Asaf Yigal, CTO of Logz.io: "Even for those that desire, or are prone to converge responsibilities, there's still a knowledge gap. Most often this is coming from the DevOps side, as in 'how do we take this important data and communicate effectively to security?' And the answer is: this is an emerging practice, so there's no wrong way, and we are working on the proverbial airplane whilst in flight!"
Despite all these challenges, Chaim Mazal, Chief Security Officer at Gigamon offers a positive outlook: "There are far fewer downsides to this convergence than there are advantages."
Companies implementing observability benefit from increased operational efficiency, faster innovation, and better business outcomes overall, according to 2023 IT Trends Report: Lessons From Observability Leaders, a report from SolarWinds ...
Customer loyalty is changing as retailers get increasingly competitive. More than 75% of consumers say they would end business with a company after a single bad customer experience. This means that just one price discrepancy, inventory mishap or checkout issue in a physical or digital store, could have customers running out to the next store that can provide them with better service. Retailers must be able to predict business outages in advance, and act proactively before an incident occurs, impacting customer experience ...
Earlier this year, New Relic conducted a study on observability ... The 2023 Observability Forecast reveals observability's impact on the lives of technical professionals and businesses' bottom lines. Here are 10 key takeaways from the forecast ...
Only 33% of executives are "very confident" in their ability to operate in a public cloud environment, according to the 2023 State of CloudOps report from NetApp. This represents an increase from 2022 when only 21% reported feeling very confident ...
A large majority of organizations employ more than one cloud automation solution, and this practice creates significant challenges that are resulting in delays and added costs for businesses, according to Why companies lose efficiency and compliance with cloud automation solutions from Broadcom ...
Companies have historically relied on tools that warn IT teams when their digital systems are experiencing glitches or attacks. But in an age where consumer loyalty is fickle and hybrid workers' Digital Employee Experience (DEX) is paramount for productivity, companies cannot afford to retroactively deal with IT failures that slow down employee productivity ...