Exploring the Convergence of Observability and Security - Part 6: Challenges

Pete Goldin
APMdigest

With input from industry experts — both analysts and vendors — this 8-part blog series will explore what is driving the convergence of observability and security, the challenges and advantages, and how it may transform the IT landscape.

Start with: Exploring the Convergence of Observability and Security - Part 1

Start with: Exploring the Convergence of Observability and Security - Part 2: Logs, Metrics and Traces

Start with: Exploring the Convergence of Observability and Security - Part 3: Tools

Start with: Exploring the Convergence of Observability and Security - Part 4: Dashboards

Start with: Exploring the Convergence of Observability and Security - Part 5: Teams

If you have already read the previous blogs in this series exploring the convergence of observability and security, the challenges will not surprise you. The experts cite compatibility of tools, teams and cultures as challenges to convergence, among others.

The following are some of the challenges experts see with achieving convergence:

Aversion to Change

Colin Fallwell, Field CTO of Sumo Logic: "Probably the biggest challenge comes down to one word. Change. Most people don't like change, much less transformation. DevSecOps requires change, it requires thinking about transformation as a continuous process that is never-ending. Up until now, this kind of transformation really could not happen, but with the rise of the Cloud Native Computing Foundation, the proliferation of open standards, and the mass adoption of OSS tooling like OpenTelemetry, and the need for proprietary agents for collecting telemetry are at an end, and with them the siloes of data."

Different Cultures

Prashant Prahlad, VP of Cloud Security Products at Datadog: "The biggest roadblock to the convergence of security and observability is culture. Security teams need to be able to trust observability teams with product security and still be able to get the visibility they need as a failsafe."

Different Priorities

Mike Loukides, VP of Emerging Tech Content at O'Reilly Media: "I think the major challenges will be the ones we've had all along. Management wants to deliver a new version on April 1. Development is under the gun to release. Ops is under the gun to deploy. And you'll still have security experts saying: Let's make sure we didn't take any shortcuts writing the code; let's make sure we're tracing the right things. It would be nice if this conflict would go away, but I don't think it will. Not now, not ever. However, putting security and ops teams in the same group will help."

Different Budgets

Kirsten Newcomer, Director, Cloud and DevSecOps Strategy at Red Hat: "The purchasing decision and budgets for observability and security may be in different organizations."

Data Silos

Buddy Brewer, Chief Product Officer at Mezmo: "Currently, many organizations unintentionally lock data in silos that only certain teams can access, which often means DevOps and SecOps teams are either not getting the right data or implementing their individual solutions to get data from the same sources. While converging security and observability will make data significantly more actionable, organizations will be met with challenges with getting the data in the correct formats to be used by different tools they may need. In addition, they must make sure that they are adhering to regulations such as GDPR and CCPA and handle personal identifiable information (PII) properly."

Tool Silos

Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at Enterprise Management Associates (EMA) outlines several challenges to convergence. "First, the teams have separate tools with separate tool silos. Often, when these groups come together, they find the quality of the data collected by the other silo's tools are of poor quality. It's in a format that is useless to them, for instance. Also, there is no authoritative source of data. Both groups have their own data stores that represent the same truth about infrastructure and services, but the data disagrees with each other due to variations and data granularity, time stamping, etc."

"Neither group wants to give up control of tool strategy," McGillicuddy continues. "They're married to their individual tools. Which one will blink and give up their tool in favor of the other group's tool?"

Use the player or download the MP3 below to listen to EMA-APMdigest Podcast Episode 2 — Shamus McGillicuddy talks about Network Observability, the convergence of observability and security, and more.

Click here for a direct MP3 download of Episode 2 - Part 1

"We have a lot of work to do to make the tools work properly, so this is not an easy integration – largely because the observability tools were designed for observability. They were not designed for security purposes," adds Adam Hert, Director of Product at Riverbed.

Legacy Tools

Ajit Sancheti, GM, Falcon LogScale at CrowdStrike: "Legacy logging and event management tools may not provide the scale or the performance to ingest all data, which leads to ingest backlogs and sluggish search speed. Organizations should carefully evaluate logging products before attempting to collect all security and observability data in one tool."

Legacy Philosophies

Jam Leomi, Lead Security Engineer at Honeycomb: "The heart of the challenge in converging the two goes back to the culture shift we're seeing in security. A lot of today's practitioners are stuck in compliance practices or philosophies that are 30+ years old. As technology evolves, our security approach has to shift. This creates an opportunity to really connect security with the overall bottom line of the business instead of just as an afterthought. Observability as a tool and practice has the power to do a lot of the heavy lifting toward this goal, enabling a higher level of efficiency, security, and privacy."

Confidential Data

Kirsten Newcomer from Red Hat: "Some security data is not appropriate for sharing with all team members who need to consume observability data."

Security Experts are hard to find

Prashant Prahlad of Datadog: "Security experts are hard to find and take time to train within DevOps teams, so implementing DevSecOps is a long-term investment."

Knowledge Gap

Asaf Yigal, CTO of Logz.io: "Even for those that desire, or are prone to converge responsibilities, there's still a knowledge gap. Most often this is coming from the DevOps side, as in 'how do we take this important data and communicate effectively to security?' And the answer is: this is an emerging practice, so there's no wrong way, and we are working on the proverbial airplane whilst in flight!"

Despite all these challenges, Chaim Mazal, Chief Security Officer at Gigamon offers a positive outlook: "There are far fewer downsides to this convergence than there are advantages."

Go to: Exploring the Convergence of Observability and Security - Part 7: Advantages

Pete Goldin is Editor and Publisher of APMdigest

Related Links

Hot Topics

The Latest

How AI Agents Are Reshaping DataOps for the Always-On Enterprise

Enterprises today operate in a real-time environment where uninterrupted access to trusted data has become a baseline expectation for users, applications and automated systems. Traditional DataOps models, built on manual effort and human triage, cannot keep pace with this always active demand. AI agents are emerging as the operational backbone, ensuring consistent data availability, reinforcing trustworthiness and enabling a level of scale that manual processes cannot achieve ...

AI Deepfakes: Rethinking Trust in the Workplace

For decades, trust in the digital workplace rested on familiar signals. We trusted faces on video calls, voices on the phone, and emails that appeared to come from people we knew. These cues felt human and intuitive. They anchored how decisions were made, approvals were granted, and access was authorized. AI-powered deepfakes have quietly broken that model ...

Nine in Ten Enterprises Plan Cloud Data Repatriation amid Rising Cloud Costs and Data Sovereignty Mandates

Cloud migration was supposed to be a one-way door. For most enterprises, it turns out it isn't. Cloud data repatriation is a real and growing trend. A new survey ... finds that 89% of organizations plan to expand their on-premises infrastructure footprint over the next two years — and 75% have already moved at least some workloads back from public cloud in the past 24 months. The findings point to a broad rethinking of where data belongs ...

Why ITOps Need Right-Sized AI, Not Bigger Models

Over the past few years, large language models (LLMs) have revolutionized the software industry. Given their ability to excel at multi-step reasoning, LLMs have helped enterprises streamline workflows and adapt to the unknown. However, employing such models comes with sky-high costs, latency issues, and limited flexibility. In the realm of IT operations, it is generally wiser to employ smaller, domain-specific models instead ...

The End of Reactive DevOps: AI-Driven Observability for Zero-Defect Digital Systems

For years, DevOps teams operated under a simple assumption: collect enough telemetry, and you can find and fix any problem. That assumption is breaking down. Modern enterprises now operate across microservices, hybrid cloud environments, APIs, Kubernetes, and highly automated delivery pipelines. Releases happen continuously, dependencies shift constantly, and failures spread faster than teams can diagnose them ...

5 Takeaways from the Observability Forecast for Media and Entertainment

New Relic surveyed IT and engineering leaders from the media and entertainment (M&E) sector to understand what's working — and where challenges persist with their observability practices. The findings reveal how M&E organizations are navigating rising platform complexity, audience expectations, and AI-driven change. Below are five takeaways that stand out ...

Why Moving Off the Cloud Is the Easy Part and What Comes Next Is Where Things Get Hard

Let me start with something I've seen play out more times than I can count. A team hits a wall with the cloud. Costs creep up, then spike. Performance starts to feel inconsistent. Someone in finance asks a simple question like "why did this double?" and nobody has a clean answer ... Maybe this isn't the right place for everything. That realization feels like a breakthrough, like you've identified the problem. In reality, you've just identified the starting line ...

MEAN TIME TO INSIGHT Podcast - Episode 24: Network Observability Tool Sprawl

In MEAN TIME TO INSIGHT Episode 24, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses network observability tool sprawl ... 

Capacity Isn't a Guess: Observability-Driven Sizing for On-Prem Databases

In cloud-native systems, scaling is often as simple as moving a slider. For on-premise databases, the stakes are different. Over-provisioning hardware is expensive. Under-provisioning leads to performance bottlenecks that are difficult to fix once the equipment is in the rack ...

5 Security Principles Every Entrepreneur Should Apply to Leadership

When most people think about cybersecurity, they picture firewalls, encryption, and access controls — technical tools designed to protect systems and data. But beneath the technology lies a deeper set of principles about trust, decision-making, and resilience ... The best leaders don't eliminate risk. They manage it intelligently. And in many ways, cybersecurity offers a surprisingly useful playbook for doing exactly that ...

Exploring the Convergence of Observability and Security - Part 6: Challenges

Pete Goldin
APMdigest

With input from industry experts — both analysts and vendors — this 8-part blog series will explore what is driving the convergence of observability and security, the challenges and advantages, and how it may transform the IT landscape.

Start with: Exploring the Convergence of Observability and Security - Part 1

Start with: Exploring the Convergence of Observability and Security - Part 2: Logs, Metrics and Traces

Start with: Exploring the Convergence of Observability and Security - Part 3: Tools

Start with: Exploring the Convergence of Observability and Security - Part 4: Dashboards

Start with: Exploring the Convergence of Observability and Security - Part 5: Teams

If you have already read the previous blogs in this series exploring the convergence of observability and security, the challenges will not surprise you. The experts cite compatibility of tools, teams and cultures as challenges to convergence, among others.

The following are some of the challenges experts see with achieving convergence:

Aversion to Change

Colin Fallwell, Field CTO of Sumo Logic: "Probably the biggest challenge comes down to one word. Change. Most people don't like change, much less transformation. DevSecOps requires change, it requires thinking about transformation as a continuous process that is never-ending. Up until now, this kind of transformation really could not happen, but with the rise of the Cloud Native Computing Foundation, the proliferation of open standards, and the mass adoption of OSS tooling like OpenTelemetry, and the need for proprietary agents for collecting telemetry are at an end, and with them the siloes of data."

Different Cultures

Prashant Prahlad, VP of Cloud Security Products at Datadog: "The biggest roadblock to the convergence of security and observability is culture. Security teams need to be able to trust observability teams with product security and still be able to get the visibility they need as a failsafe."

Different Priorities

Mike Loukides, VP of Emerging Tech Content at O'Reilly Media: "I think the major challenges will be the ones we've had all along. Management wants to deliver a new version on April 1. Development is under the gun to release. Ops is under the gun to deploy. And you'll still have security experts saying: Let's make sure we didn't take any shortcuts writing the code; let's make sure we're tracing the right things. It would be nice if this conflict would go away, but I don't think it will. Not now, not ever. However, putting security and ops teams in the same group will help."

Different Budgets

Kirsten Newcomer, Director, Cloud and DevSecOps Strategy at Red Hat: "The purchasing decision and budgets for observability and security may be in different organizations."

Data Silos

Buddy Brewer, Chief Product Officer at Mezmo: "Currently, many organizations unintentionally lock data in silos that only certain teams can access, which often means DevOps and SecOps teams are either not getting the right data or implementing their individual solutions to get data from the same sources. While converging security and observability will make data significantly more actionable, organizations will be met with challenges with getting the data in the correct formats to be used by different tools they may need. In addition, they must make sure that they are adhering to regulations such as GDPR and CCPA and handle personal identifiable information (PII) properly."

Tool Silos

Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at Enterprise Management Associates (EMA) outlines several challenges to convergence. "First, the teams have separate tools with separate tool silos. Often, when these groups come together, they find the quality of the data collected by the other silo's tools are of poor quality. It's in a format that is useless to them, for instance. Also, there is no authoritative source of data. Both groups have their own data stores that represent the same truth about infrastructure and services, but the data disagrees with each other due to variations and data granularity, time stamping, etc."

"Neither group wants to give up control of tool strategy," McGillicuddy continues. "They're married to their individual tools. Which one will blink and give up their tool in favor of the other group's tool?"

Use the player or download the MP3 below to listen to EMA-APMdigest Podcast Episode 2 — Shamus McGillicuddy talks about Network Observability, the convergence of observability and security, and more.

Click here for a direct MP3 download of Episode 2 - Part 1

"We have a lot of work to do to make the tools work properly, so this is not an easy integration – largely because the observability tools were designed for observability. They were not designed for security purposes," adds Adam Hert, Director of Product at Riverbed.

Legacy Tools

Ajit Sancheti, GM, Falcon LogScale at CrowdStrike: "Legacy logging and event management tools may not provide the scale or the performance to ingest all data, which leads to ingest backlogs and sluggish search speed. Organizations should carefully evaluate logging products before attempting to collect all security and observability data in one tool."

Legacy Philosophies

Jam Leomi, Lead Security Engineer at Honeycomb: "The heart of the challenge in converging the two goes back to the culture shift we're seeing in security. A lot of today's practitioners are stuck in compliance practices or philosophies that are 30+ years old. As technology evolves, our security approach has to shift. This creates an opportunity to really connect security with the overall bottom line of the business instead of just as an afterthought. Observability as a tool and practice has the power to do a lot of the heavy lifting toward this goal, enabling a higher level of efficiency, security, and privacy."

Confidential Data

Kirsten Newcomer from Red Hat: "Some security data is not appropriate for sharing with all team members who need to consume observability data."

Security Experts are hard to find

Prashant Prahlad of Datadog: "Security experts are hard to find and take time to train within DevOps teams, so implementing DevSecOps is a long-term investment."

Knowledge Gap

Asaf Yigal, CTO of Logz.io: "Even for those that desire, or are prone to converge responsibilities, there's still a knowledge gap. Most often this is coming from the DevOps side, as in 'how do we take this important data and communicate effectively to security?' And the answer is: this is an emerging practice, so there's no wrong way, and we are working on the proverbial airplane whilst in flight!"

Despite all these challenges, Chaim Mazal, Chief Security Officer at Gigamon offers a positive outlook: "There are far fewer downsides to this convergence than there are advantages."

Go to: Exploring the Convergence of Observability and Security - Part 7: Advantages

Pete Goldin is Editor and Publisher of APMdigest

Related Links

Hot Topics

The Latest

How AI Agents Are Reshaping DataOps for the Always-On Enterprise

Enterprises today operate in a real-time environment where uninterrupted access to trusted data has become a baseline expectation for users, applications and automated systems. Traditional DataOps models, built on manual effort and human triage, cannot keep pace with this always active demand. AI agents are emerging as the operational backbone, ensuring consistent data availability, reinforcing trustworthiness and enabling a level of scale that manual processes cannot achieve ...

AI Deepfakes: Rethinking Trust in the Workplace

For decades, trust in the digital workplace rested on familiar signals. We trusted faces on video calls, voices on the phone, and emails that appeared to come from people we knew. These cues felt human and intuitive. They anchored how decisions were made, approvals were granted, and access was authorized. AI-powered deepfakes have quietly broken that model ...

Nine in Ten Enterprises Plan Cloud Data Repatriation amid Rising Cloud Costs and Data Sovereignty Mandates

Cloud migration was supposed to be a one-way door. For most enterprises, it turns out it isn't. Cloud data repatriation is a real and growing trend. A new survey ... finds that 89% of organizations plan to expand their on-premises infrastructure footprint over the next two years — and 75% have already moved at least some workloads back from public cloud in the past 24 months. The findings point to a broad rethinking of where data belongs ...

Why ITOps Need Right-Sized AI, Not Bigger Models

Over the past few years, large language models (LLMs) have revolutionized the software industry. Given their ability to excel at multi-step reasoning, LLMs have helped enterprises streamline workflows and adapt to the unknown. However, employing such models comes with sky-high costs, latency issues, and limited flexibility. In the realm of IT operations, it is generally wiser to employ smaller, domain-specific models instead ...

The End of Reactive DevOps: AI-Driven Observability for Zero-Defect Digital Systems

For years, DevOps teams operated under a simple assumption: collect enough telemetry, and you can find and fix any problem. That assumption is breaking down. Modern enterprises now operate across microservices, hybrid cloud environments, APIs, Kubernetes, and highly automated delivery pipelines. Releases happen continuously, dependencies shift constantly, and failures spread faster than teams can diagnose them ...

5 Takeaways from the Observability Forecast for Media and Entertainment

New Relic surveyed IT and engineering leaders from the media and entertainment (M&E) sector to understand what's working — and where challenges persist with their observability practices. The findings reveal how M&E organizations are navigating rising platform complexity, audience expectations, and AI-driven change. Below are five takeaways that stand out ...

Why Moving Off the Cloud Is the Easy Part and What Comes Next Is Where Things Get Hard

Let me start with something I've seen play out more times than I can count. A team hits a wall with the cloud. Costs creep up, then spike. Performance starts to feel inconsistent. Someone in finance asks a simple question like "why did this double?" and nobody has a clean answer ... Maybe this isn't the right place for everything. That realization feels like a breakthrough, like you've identified the problem. In reality, you've just identified the starting line ...

MEAN TIME TO INSIGHT Podcast - Episode 24: Network Observability Tool Sprawl

In MEAN TIME TO INSIGHT Episode 24, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses network observability tool sprawl ... 

Capacity Isn't a Guess: Observability-Driven Sizing for On-Prem Databases

In cloud-native systems, scaling is often as simple as moving a slider. For on-premise databases, the stakes are different. Over-provisioning hardware is expensive. Under-provisioning leads to performance bottlenecks that are difficult to fix once the equipment is in the rack ...

5 Security Principles Every Entrepreneur Should Apply to Leadership

When most people think about cybersecurity, they picture firewalls, encryption, and access controls — technical tools designed to protect systems and data. But beneath the technology lies a deeper set of principles about trust, decision-making, and resilience ... The best leaders don't eliminate risk. They manage it intelligently. And in many ways, cybersecurity offers a surprisingly useful playbook for doing exactly that ...