Exploring the Convergence of Observability and Security - Part 6: Challenges
June 13, 2023

Pete Goldin
APMdigest

Share this

With input from industry experts — both analysts and vendors — this 8-part blog series will explore what is driving the convergence of observability and security, the challenges and advantages, and how it may transform the IT landscape.

Start with: Exploring the Convergence of Observability and Security - Part 1

Start with: Exploring the Convergence of Observability and Security - Part 2: Logs, Metrics and Traces

Start with: Exploring the Convergence of Observability and Security - Part 3: Tools

Start with: Exploring the Convergence of Observability and Security - Part 4: Dashboards

Start with: Exploring the Convergence of Observability and Security - Part 5: Teams

If you have already read the previous blogs in this series exploring the convergence of observability and security, the challenges will not surprise you. The experts cite compatibility of tools, teams and cultures as challenges to convergence, among others.

The following are some of the challenges experts see with achieving convergence:

Aversion to Change

Colin Fallwell, Field CTO of Sumo Logic: "Probably the biggest challenge comes down to one word. Change. Most people don't like change, much less transformation. DevSecOps requires change, it requires thinking about transformation as a continuous process that is never-ending. Up until now, this kind of transformation really could not happen, but with the rise of the Cloud Native Computing Foundation, the proliferation of open standards, and the mass adoption of OSS tooling like OpenTelemetry, and the need for proprietary agents for collecting telemetry are at an end, and with them the siloes of data."

Different Cultures

Prashant Prahlad, VP of Cloud Security Products at Datadog: "The biggest roadblock to the convergence of security and observability is culture. Security teams need to be able to trust observability teams with product security and still be able to get the visibility they need as a failsafe."

Different Priorities

Mike Loukides, VP of Emerging Tech Content at O'Reilly Media: "I think the major challenges will be the ones we've had all along. Management wants to deliver a new version on April 1. Development is under the gun to release. Ops is under the gun to deploy. And you'll still have security experts saying: Let's make sure we didn't take any shortcuts writing the code; let's make sure we're tracing the right things. It would be nice if this conflict would go away, but I don't think it will. Not now, not ever. However, putting security and ops teams in the same group will help."

Different Budgets

Kirsten Newcomer, Director, Cloud and DevSecOps Strategy at Red Hat: "The purchasing decision and budgets for observability and security may be in different organizations."

Data Silos

Buddy Brewer, Chief Product Officer at Mezmo: "Currently, many organizations unintentionally lock data in silos that only certain teams can access, which often means DevOps and SecOps teams are either not getting the right data or implementing their individual solutions to get data from the same sources. While converging security and observability will make data significantly more actionable, organizations will be met with challenges with getting the data in the correct formats to be used by different tools they may need. In addition, they must make sure that they are adhering to regulations such as GDPR and CCPA and handle personal identifiable information (PII) properly."

Tool Silos

Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at Enterprise Management Associates (EMA) outlines several challenges to convergence. "First, the teams have separate tools with separate tool silos. Often, when these groups come together, they find the quality of the data collected by the other silo's tools are of poor quality. It's in a format that is useless to them, for instance. Also, there is no authoritative source of data. Both groups have their own data stores that represent the same truth about infrastructure and services, but the data disagrees with each other due to variations and data granularity, time stamping, etc."

"Neither group wants to give up control of tool strategy," McGillicuddy continues. "They're married to their individual tools. Which one will blink and give up their tool in favor of the other group's tool?"

Use the player or download the MP3 below to listen to EMA-APMdigest Podcast Episode 2 — Shamus McGillicuddy talks about Network Observability, the convergence of observability and security, and more.

Click here for a direct MP3 download of Episode 2 - Part 1

"We have a lot of work to do to make the tools work properly, so this is not an easy integration – largely because the observability tools were designed for observability. They were not designed for security purposes," adds Adam Hert, Director of Product at Riverbed.

Legacy Tools

Ajit Sancheti, GM, Falcon LogScale at CrowdStrike: "Legacy logging and event management tools may not provide the scale or the performance to ingest all data, which leads to ingest backlogs and sluggish search speed. Organizations should carefully evaluate logging products before attempting to collect all security and observability data in one tool."

Legacy Philosophies

Jam Leomi, Lead Security Engineer at Honeycomb: "The heart of the challenge in converging the two goes back to the culture shift we're seeing in security. A lot of today's practitioners are stuck in compliance practices or philosophies that are 30+ years old. As technology evolves, our security approach has to shift. This creates an opportunity to really connect security with the overall bottom line of the business instead of just as an afterthought. Observability as a tool and practice has the power to do a lot of the heavy lifting toward this goal, enabling a higher level of efficiency, security, and privacy."

Confidential Data

Kirsten Newcomer from Red Hat: "Some security data is not appropriate for sharing with all team members who need to consume observability data."

Security Experts are hard to find

Prashant Prahlad of Datadog: "Security experts are hard to find and take time to train within DevOps teams, so implementing DevSecOps is a long-term investment."

Knowledge Gap

Asaf Yigal, CTO of Logz.io: "Even for those that desire, or are prone to converge responsibilities, there's still a knowledge gap. Most often this is coming from the DevOps side, as in 'how do we take this important data and communicate effectively to security?' And the answer is: this is an emerging practice, so there's no wrong way, and we are working on the proverbial airplane whilst in flight!"

Despite all these challenges, Chaim Mazal, Chief Security Officer at Gigamon offers a positive outlook: "There are far fewer downsides to this convergence than there are advantages."

Go to: Exploring the Convergence of Observability and Security - Part 7: Advantages

Pete Goldin is Editor and Publisher of APMdigest
Share this

The Latest

September 21, 2023

Companies implementing observability benefit from increased operational efficiency, faster innovation, and better business outcomes overall, according to 2023 IT Trends Report: Lessons From Observability Leaders, a report from SolarWinds ...

September 20, 2023

IT leaders are driving an increasing number of automation initiatives as a way to stay competitive, reduce costs and scale as they navigate an unpredictable social and economic environment, according to the 2023 State of Automation in IT survey conducted by Jitterbit ...

September 19, 2023

Customer loyalty is changing as retailers get increasingly competitive. More than 75% of consumers say they would end business with a company after a single bad customer experience. This means that just one price discrepancy, inventory mishap or checkout issue in a physical or digital store, could have customers running out to the next store that can provide them with better service. Retailers must be able to predict business outages in advance, and act proactively before an incident occurs, impacting customer experience ...

September 18, 2023
Digital transformation is key to ensuring companies keep up with the competitive market landscape. Putting digital at the core of a business can significantly reduce operating expenses and inefficiencies. However, this process often means changing the way internal teams work with one another. To help with the transition, this blog offers chief experience officers (CXOs) advice on how to lead a successful digital transformation project ...
September 14, 2023

Earlier this year, New Relic conducted a study on observability ... The 2023 Observability Forecast reveals observability's impact on the lives of technical professionals and businesses' bottom lines. Here are 10 key takeaways from the forecast ...

September 13, 2023
On September 10, MGM Resorts experienced what it called a "cybersecurity issue" that had a major impact on the company's systems, showing how cyberattacks can bring down applications, ultimately causing problems for a company in many ways ...
September 12, 2023

Only 33% of executives are "very confident" in their ability to operate in a public cloud environment, according to the 2023 State of CloudOps report from NetApp. This represents an increase from 2022 when only 21% reported feeling very confident ...

September 11, 2023

The majority of organizations across Australia and New Zealand (A/NZ) breached over the last year had personally identifiable information (PII) compromised, but most have not yet modified their data management policies, according to the Cybersecurity and PII Report from ManageEngine ...

September 07, 2023

A large majority of organizations employ more than one cloud automation solution, and this practice creates significant challenges that are resulting in delays and added costs for businesses, according to Why companies lose efficiency and compliance with cloud automation solutions from Broadcom ...

September 06, 2023

Companies have historically relied on tools that warn IT teams when their digital systems are experiencing glitches or attacks. But in an age where consumer loyalty is fickle and hybrid workers' Digital Employee Experience (DEX) is paramount for productivity, companies cannot afford to retroactively deal with IT failures that slow down employee productivity ...