In APMdigest's new series, Exploring AI Observability, experts across observability and AI provide insight and opinions on the need for AI Observability, the challenges, the benefits, the current state, and predictions for the future. Part 5 outlines the organizational roles that utilize AI observability.
"AI observability serves several roles across an organization," explains Bernd Greifeneder, CTO and Founder, Dynatrace. "AI engineers use it to build, evaluate, and improve AI systems. SREs use it to operate those systems reliably and efficiently in production, including managing their cost. Leaders need it to connect AI behavior and operational performance to business outcomes. The common need is a clear understanding of whether AI systems are delivering the intended result."
MACHINE LEARNING AND AI ENGINEERS
What is new is that machine learning and AI engineering teams have a direct stake, because they need to see how a model behaves once real traffic reaches it rather than how it scored on a benchmark.
Juraci Paixão Kröhling, CEO, Ollygarden
AI engineers and scientists are watching for drift and trying to understand how the system is actually behaving against real world data and scenarios it wasn't necessarily trained on.
Joseph Voyles, Principal, AI & Emerging Technology, PwC US
Machine-learning and evaluation teams need AI observability to measure answer quality, groundedness, consistency, drift, and regression across model or application changes.
Steve Giguere, Principal AI Security Advocate with the Office of the CTO, Check Point
Engineers and ML practitioners need granular, technical views — token-level detail for LLMs, or full reasoning and tool-call traces for agentic systems — to debug failures.
Jason Beres, COO, Infragistics
Data scientists and ML engineers use it to monitor model quality, drift and evaluation metrics.
Pretam Mondal, Lead Product Manager, Observability and Service Management, OpenText
SRE
SREs and infrastructure teams need real-time line of sight into agentic actions to verify that routine automated steps execute safely without introducing downtime or policy violations.
Phil Christianson, CPO, Xurrent
SREs live and die by their observability tooling, especially during an incident. With AI, the incident may not even look like an outage. It may be a subtle degradation in model performance, or a shift in the distribution of non-deterministic results. It's always going to be a little random, but the shape of that randomness can drift over time, and detecting that requires tools most teams don't have yet.
Jackson Stakeman, GM, Sparq
ITOPS
IT Operations should play a central role in AI observability. This team is typically responsible for enterprise visibility, governance, and operational stability. At a minimum, IT needs insight into AI adoption, shadow AI, infrastructure utilization, and organizational risk.
Phil Lenton, Head of Product Management for SaaS Services and AIOps, Riverbed Technology
IT operations leaders rely on it to check the AI agent's behavior and measure the automation results. Any team focused on performance, reliability, or cost should care about AI observability beyond just incident handling.
Gowrisankar Chinnayan, Director of Product Management, ManageEngine
Infrastructure and operations teams are best positioned to handle the functional and commercial governance side. They need to make sure an agent is drawing from the right data sources, staying within its guardrails, and not wandering into systems it shouldn't touch.
Corey Beck, VP of Service Delivery, DataStrike
IT operations teams need it to monitor autonomous agents, troubleshoot non-deterministic system workflows and maintain performance across AI-enhanced infrastructure.
Gou Rao, Co-Founder and CEO, NeuBird AI
DEVELOPMENT
Application developers need insight into prompt effectiveness and response quality.
Jeremy Rossbach, Chief Technical Evangelist, Network Observability, Broadcom
Software engineering teams require behavioral telemetry to trace model logic, optimize latency, catch hallucinations and control costs at the application layer.
Gou Rao, Co-Founder and CEO, NeuBird AI
Application developers use it to troubleshoot model behavior and optimize prompts and workflows.
Pretam Mondal, Lead Product Manager, Observability and Service Management, OpenText
PLATFORM ENGINEERING
Platform engineering teams use AI observability to ensure AI workloads run properly on their infrastructure.
Gowrisankar Chinnayan, Director of Product Management, ManageEngine
Platform engineers need AI observability to understand infrastructure utilization and model performance.
Jeremy Rossbach, Chief Technical Evangelist, Network Observability, Broadcom
FINOPS
The FinOps teams use AI observability to track token use and model spending, which is very different from traditional infrastructure costs.
Gowrisankar Chinnayan, Director of Product Management, ManageEngine
FinOps may need this most of all. The financial impact of model performance and the decisions these tools make is larger than anything we saw with cloud infrastructure, and the potential for runaway token spend is enormous. Finance needs dashboards that show what AI usage actually is, how to attribute the cost, and what's in or out of range. That's the only way to determine ROI. Those signals feed back to the development teams: do we need a more affordable model, are we getting the value we assumed? In a landscape moving this fast, that evaluation has to be continuous.
Jackson Stakeman, GM, Sparq
SECURITY
Security teams need visibility into prompt injection, unsafe data access, abnormal tool use, privilege escalation, sensitive-data exposure, policy violations, and suspicious sequences of otherwise legitimate actions.
Steve Giguere, Principal AI Security Advocate with the Office of the CTO, Check Point
Security and SOC teams need it to catch behavioral drift and shadow AI before it becomes an incident.
Beth Miller, Global Field CISO, Mimecast
COMPLIANCE AND GOVERNANCE
Compliance and risk teams need evidence that required business processes and policies were followed. Governance teams need visibility across agents rather than examining individual traces in isolation.
Sekhar Sarukkai, Co-Founder and CEO, ChatSee.ai
CISO
CISOs need AI observability to manage risk and stay compliant.
Gidi Cohen, Co-Founder and CEO, Bonfy AI
CIO
CIOs need AI observability to keep things running smoothly.
Gidi Cohen, Co-Founder and CEO, Bonfy AI
BUSINESS AND PRODUCT OWNERS
Business and product owners also need observability. They must know whether the AI system is completing useful work, whether humans are frequently correcting it, whether automation is improving an outcome, and whether the operational and model costs are justified.
Steve Giguere, Principal AI Security Advocate with the Office of the CTO, Check Point
AI observability is leaving IT's hands. The real user is whoever is accountable for what an agent does, and that is increasingly a business role, not a technical one. As agents move into finance, operations, service, and every other function, each inherits an accountability that used to sit in IT. But in an agentic enterprise, AI observability stops being a monitoring tool and becomes business infrastructure.
Rohit Choudhary, CEO, Acceldata
For agentic AI, the audience becomes much broader because the observable unit is increasingly a business process rather than an individual model call. Business owners need to understand failed outcomes and their economic impact.
Sekhar Sarukkai, Co-Founder and CEO, ChatSee.ai
Product managers and business stakeholders need a view focused on reliability, cost, and user-facing quality trends rather than the underlying mechanics. Increasingly, compliance and risk teams need visibility too, particularly for agentic systems that can take real-world actions autonomously.
Jason Beres, COO, Infragistics
Go to: Exploring AI Observability – Part 6: Metrics