Skip to main content

Exploring AI Observability – Part 8: Features

Pete Goldin
APMdigest

When it's time for an enterprise to actually evaluate an AI observability tool, the key question is: what are the most important features or capabilities that an AI observability tool should provide? Part 8 outlines the features needed for AI observability.

DISCOVERY AND INVENTORY

An AI observability capability should first provide discovery and inventory. It should help the enterprise identify models, agents, tools, prompts, data sources, and external dependencies, including systems introduced outside conventional deployment processes. Steve Giguere, Principal AI Security Advocate with the Office of the CTO, Check Point

AI discovery – both sanctioned (allowed AI tools) and Shadow AI (unauthorized) discovery. All of it – Agents, models (LLM), MCP Servers, other tools – external and internal. Dror Zelber, VP of Product Marketing, Radware

OPEN AND INTEROPERABLE ARCHITECTURE

The underlying architecture should remain open and interoperable. Teams need portable schemas, open data formats, programmatic access, and infrastructure capable of correlating, searching, and retaining high-volume, high-cardinality telemetry across AI applications, traditional observability systems, and business data. Both people and agents should be able to work from the same evidence without being constrained to a single framework or backend. Aaron Winston, Head of Content, Communications, and Analyst Relations, Arize AI

Enterprise readiness depends on open interoperability, such as integration with open protocols that allow AI systems themselves to securely query stored trace data, enabling organizations to integrate AI evaluation into their observability strategy rather than seeing it as a separated workflow. Bianca Lewis, Executive Director, OpenSearch Software Foundation

UNIFIED TELEMETRY LAYER

A unified telemetry layer comes first. AI needs a complete, correlated picture before it acts. Fragmented data produces fragmented decisions. Gowrisankar Chinnayan, Director of Product Management, ManageEngine

AUTO-INSTRUMENTATION

Seamless auto-instrumentation across common model libraries and agent frameworks is essential to reduce developer friction and eliminate blind spots. Bianca Lewis, Executive Director, OpenSearch Software Foundation

FULL FIDELITY

Full fidelity with no forced sampling, because sampling discards the incidents. Gabriel-James Safar, CEO, Tsuga

FULL STACK OBSERVABILITY

An AI observability tool needs unified visibility across the full software stack, including the AI layer. Bernd Greifeneder, CTO and Founder, Dynatrace

END-TO-END TRACING

The table-stakes capability is tracing every prompt, model call, tool call and retrieval step, then evaluating the outputs. Rohit Choudhary, CEO, Acceldata

First, it requires agent-level tracing that visualizes non-linear execution paths, such as DAG graphs, decision trees, and tool calls, to reveal exactly how an agent reasoned through a task. Bianca Lewis, Executive Director, OpenSearch Software Foundation

A strong AI observability tool needs end-to-end, session-aware tracing. For an LLM application, that includes prompts, model versions, retrieval, outputs, latency, tokens, and cost. For an agentic system, it also includes tools, state, memory, routing, permissions, handoffs, retries, loops, and the complete task trajectory. Aaron Winston, Arize AI

AI observability should provide end-to-end tracing across the complete AI execution. For an LLM application, this includes the prompt, retrieval process, model call, response, and relevant application behavior. For an agent, it should extend through planning, tool selection, authorization, execution, memory changes, and downstream effects. Steve Giguere, Principal AI Security Advocate with the Office of the CTO, Check Point

An AI observability tool needs to trace an entire agentic workflow step by step, every tool call, every intermediate decision, not just log the final input and output. Without that granularity, teams can see that something went wrong but have no way to find out where in the chain it happened. Andi Mann, Chief Product & Technology Officer, Apica

END-TO-END CORRELATION

The most valuable capability is end-to-end correlation. Organizations need to connect prompts, model responses, retrieval quality, agent actions, infrastructure telemetry, application performance, network health, security events, user experience, and business outcomes into a single operational narrative. Without that correlation, teams spend excessive time investigating individual components while missing the broader operational picture. Jeremy Rossbach, Chief Technical Evangelist, Network Observability, Broadcom

Correlation is going to matter more than simply collecting more telemetry. Most enterprises already have plenty of data. What they lack oftentimes is the ability to connect it quickly enough to understand why something failed. For an AI application, that means correlating model behavior with application performance, databases, infrastructure, networks and external dependencies. For agentic systems, it also means tracing the sequence of actions the agent took and understanding which dependency was involved at each step. Don Boxley, CEO and Co-Founder, DH2i

CONTINUOUS MONITORING OF OUTPUT QUALITY

If you're shopping for an AI observability tool built for LLMs, your top priority should be something that never stops checking output quality once the model hits production. You can’t stop at initial benchmarks, because real-world edge cases always behave differently than test data, and a model that passed evaluation can still drift or degrade once it’s handling live, unpredictable inputs. That means continuous monitoring of response quality, not a one-time stamp of approval before launch. Krishna Sai, CTO, SolarWinds

The evaluation layer must work in development and production. Teams need deterministic checks, model-based judges, human annotation, curated datasets, online monitoring, and methods for calibrating evaluator quality. Agentic evaluation should score the trajectory and task result along with the final response. Aaron Winston, Arize AI

Continuous, automated evaluation of output quality in production, not just in pre-deployment testing. Groundedness and correctness should be scored on live traffic, not assumed to hold once the model ships. Douglas James, VP, Skylar AI, ScienceLogic

Continuous quality evaluation on live traffic: Offline testing before launch isn’t enough, because production inputs are messier than test sets. Tools need to be able to score real production interactions for correctness, relevance, or policy adherence on an ongoing basis, and surface degradation before users complain about it. David Girvin, Lead Technical Advocate, Sumo Logic

The single most important capability is continuous quality evaluation in production. Offline evals before launch are table stakes. What matters is knowing your quality score dropped four points this Tuesday, before your users tell you. Kunal Agarwal, Co-founder and CEO, Unravel Data

Behavioral failure detection

The system needs to identify failures that do not present as technical errors — unresolved outcomes, missed escalations, inappropriate actions, policy deviations, execution errors, and other behavioral failures. This is especially important given that our research found roughly 90% of observed enterprise AI failures were operational rather than traditional hallucination failures. Sekhar Sarukkai, Co-Founder and CEO, ChatSee.ai

Dynamic anomaly detection that learns from actual system behavior matters more than static thresholds set months ago and never revisited. Gowrisankar Chinnayan, Director of Product Management, ManageEngine

Consistent failure taxonomy and diagnosis

Enterprises cannot operationalize agent reliability if every team describes the same underlying failure differently. Failures need to be classified consistently across agents, models, workflows, and time. Sekhar Sarukkai, ChatSee.ai

cross-interaction and cross-agent recurrence detection

If the same missed escalation occurs hundreds of times, the enterprise needs to recognize one systemic problem rather than hundreds of independent anomalous traces. Sekhar Sarukkai, ChatSee.ai

shared failure memory

Prior diagnoses, human adjudications, affected contexts, treatments, and remediation results should become reusable intelligence accessible to humans, assurance systems, and agents. The system should get smarter as the enterprise resolves failures. Sekhar Sarukkai, ChatSee.ai

CAUSAL REASONING

Transparent Causal Reasoning: Provides detailed, step-by-step explanations of why an anomaly occurred, fully backed by concrete evidence and citations (linking directly to OTel trace spans, prompt/response pairs, vector queries or commit diffs). Gou Rao, co-founder and CEO, NeuBird AI

RECONSTRUCTION

The most important capability is context reconstruction. When something goes wrong, a team should be able to reconstruct what the AI system was trying to do, what information it used, what it did next, and where the failure entered the chain. For LLM applications, that means visibility into prompts, context, model responses, evaluation results, feedback, and cost. For agentic systems, it also means timelines of tool calls, browser actions, file edits, workspace versions, external responses, approvals, retries, and recovery attempts. A strong AI observability tool should help a team decide whether to continue, retry, roll back to a known-good version, escalate to a human, or change the agent's action boundaries. Kylian Cros, Co-Founder and CMO, Molted

The most important capability isn’t a dashboard, it’s a black box. Every airplane carries one because when something goes wrong, investigators don’t start with did it fail, they start with why it thought that was the right call. An AI observability tool needs the same thing, the ability to reconstruct what the agent knew, what it decided, and why, after the fact. Corey Beck, VP of Service Delivery, DataStrike

The ability to reconstruct an end-to-end decision path is essential, whether that’s understanding why a model produced a given response or why an agent chose a particular multi-step action, so teams can see not just what happened but why. Stephen Ochs, Head of Marketing, Selector

Proactive Context Seeking

Actively prompts engineers for missing context when signals are ambiguous (e.g., asking whether a sudden token surge stems from a new deployment or an agent loop) to refine hypothesis trees faster. Gou Rao, NeuBird AI

CLEAR AUDIT TRAIL

A clear audit trail for every agent action: Humans review what triggered a decision and what was skipped after the fact, so that the trail is complete. Gowrisankar Chinnayan, Director of Product Management, ManageEngine

Audit trails, access controls, approval gates, and rollback allow agents to investigate and propose changes while people retain authority where judgment and risk require it. Aaron Winston, Arize AI

Action-level audit trail for agents: For agentic systems specifically, the tool needs to record not just what the agent decided but what it actually did which systems it touched, what data it accessed, and what permissions it exercised in a form detailed enough to support both incident investigation and compliance evidence after the fact. David Girvin, Sumo Logic

The most critical capability for monitoring AI is maintaining a controlled, human-in-the-loop experience that allows operators to inspect, approve and reverse any action. Observability capabilities must provide complete transparency into credential usage, ensuring that agent actions are explicitly tagged rather than masked under a human user account. Phil Christianson, CPO, Xurrent

REMEDIATION

Closed-loop remediation and effectiveness measurement: Observability should not end when the root cause appears on a dashboard. It should support the transition from diagnosis into treatment and then verify whether recurrence actually declined — including after subsequent agent, model, prompt, or workflow changes. Sekhar Sarukkai, ChatSee.ai

Assisted Remediation: Recommends and executes concrete recovery actions, such as dynamic model failover, prompt guardrail enforcement or rate-limiting, backed by human-in-the-loop approval controls. Gou Rao, NeuBird AI

IDENTITY AND POLICY CONTEXT

AI observability should preserve identity and policy context. Teams need to know which user, service, or agent initiated an interaction, what permissions were available, which policy was evaluated, what decision was made, and which version of that policy was active. Steve Giguere, Principal AI Security Advocate with the Office of the CTO, Check Point

BUSINESS-AWARE ALERTING

Here's what separates tools that matter from tools that make dashboards: they have to connect observability to action. Drift detection that just alerts is useless. Business-aware alerting that tells you the specific decision type that’s degrading is actionable. Lior Matkovich, Senior Director, ITOM, ServiceNow

PRIVACY CONTROLS

AI observability should provide strong privacy controls. Sensitive content should be selectively captured, redacted, encrypted, access-controlled, and retained according to its purpose. Teams should be able to record the metadata required for investigation without automatically storing every complete prompt and response. Steve Giguere, Principal AI Security Advocate with the Office of the CTO, Check Point

AI-DATA LAKEHOUSE

AI observability needs an AI-data lakehouse that brings together traces, behavior, performance, cost, and availability data from across the stack, then make that context available for analytics, optimization, and feature development. Bernd Greifeneder, CTO and Founder, Dynatrace

DATA RESIDENCY

Data residency: keeping the telemetry inside the customer’s own cloud, so that observing AI never means surrendering the most sensitive data in the business. Gabriel-James Safar, Tsuga

REAL-TIME TRACKING OF USAGE AND COST

You need real-time tracking of token usage and cost. Without it, you’re flying blind on budget, since usage patterns in production rarely match what you projected in testing, and costs can spiral quickly once real traffic and real users are in the mix. The best tools tie quality monitoring and cost visibility together. Krishna Sai, CTO, SolarWinds

Cost and volume management: Given how much telemetry AI systems generate, the tool needs to help teams manage the volume and cost of that data itself, routing and filtering intelligently rather than forcing a choice between blind spots and runaway ingestion costs. Andi Mann, Apica

Cost and token-usage visibility tied to specific workflows also matters, letting teams manage spend proactively instead of discovering a runaway cost or agent only after the bill or the damage has landed. Stephen Ochs, Selector

Cost and performance in the same view: Token usage and cost per interaction need to sit next to latency and quality, because in AI systems those three things trade off against each other constantly, and optimizing one blindly tends to hurt the others. David Girvin, Sumo Logic

Check back tomorrow for Exploring AI Observability – Part 9: Benefits

Pete Goldin is Editor and Publisher of APMdigest

Hot Topics

Exploring AI Observability – Part 8: Features

Pete Goldin
APMdigest

When it's time for an enterprise to actually evaluate an AI observability tool, the key question is: what are the most important features or capabilities that an AI observability tool should provide? Part 8 outlines the features needed for AI observability.

DISCOVERY AND INVENTORY

An AI observability capability should first provide discovery and inventory. It should help the enterprise identify models, agents, tools, prompts, data sources, and external dependencies, including systems introduced outside conventional deployment processes. Steve Giguere, Principal AI Security Advocate with the Office of the CTO, Check Point

AI discovery – both sanctioned (allowed AI tools) and Shadow AI (unauthorized) discovery. All of it – Agents, models (LLM), MCP Servers, other tools – external and internal. Dror Zelber, VP of Product Marketing, Radware

OPEN AND INTEROPERABLE ARCHITECTURE

The underlying architecture should remain open and interoperable. Teams need portable schemas, open data formats, programmatic access, and infrastructure capable of correlating, searching, and retaining high-volume, high-cardinality telemetry across AI applications, traditional observability systems, and business data. Both people and agents should be able to work from the same evidence without being constrained to a single framework or backend. Aaron Winston, Head of Content, Communications, and Analyst Relations, Arize AI

Enterprise readiness depends on open interoperability, such as integration with open protocols that allow AI systems themselves to securely query stored trace data, enabling organizations to integrate AI evaluation into their observability strategy rather than seeing it as a separated workflow. Bianca Lewis, Executive Director, OpenSearch Software Foundation

UNIFIED TELEMETRY LAYER

A unified telemetry layer comes first. AI needs a complete, correlated picture before it acts. Fragmented data produces fragmented decisions. Gowrisankar Chinnayan, Director of Product Management, ManageEngine

AUTO-INSTRUMENTATION

Seamless auto-instrumentation across common model libraries and agent frameworks is essential to reduce developer friction and eliminate blind spots. Bianca Lewis, Executive Director, OpenSearch Software Foundation

FULL FIDELITY

Full fidelity with no forced sampling, because sampling discards the incidents. Gabriel-James Safar, CEO, Tsuga

FULL STACK OBSERVABILITY

An AI observability tool needs unified visibility across the full software stack, including the AI layer. Bernd Greifeneder, CTO and Founder, Dynatrace

END-TO-END TRACING

The table-stakes capability is tracing every prompt, model call, tool call and retrieval step, then evaluating the outputs. Rohit Choudhary, CEO, Acceldata

First, it requires agent-level tracing that visualizes non-linear execution paths, such as DAG graphs, decision trees, and tool calls, to reveal exactly how an agent reasoned through a task. Bianca Lewis, Executive Director, OpenSearch Software Foundation

A strong AI observability tool needs end-to-end, session-aware tracing. For an LLM application, that includes prompts, model versions, retrieval, outputs, latency, tokens, and cost. For an agentic system, it also includes tools, state, memory, routing, permissions, handoffs, retries, loops, and the complete task trajectory. Aaron Winston, Arize AI

AI observability should provide end-to-end tracing across the complete AI execution. For an LLM application, this includes the prompt, retrieval process, model call, response, and relevant application behavior. For an agent, it should extend through planning, tool selection, authorization, execution, memory changes, and downstream effects. Steve Giguere, Principal AI Security Advocate with the Office of the CTO, Check Point

An AI observability tool needs to trace an entire agentic workflow step by step, every tool call, every intermediate decision, not just log the final input and output. Without that granularity, teams can see that something went wrong but have no way to find out where in the chain it happened. Andi Mann, Chief Product & Technology Officer, Apica

END-TO-END CORRELATION

The most valuable capability is end-to-end correlation. Organizations need to connect prompts, model responses, retrieval quality, agent actions, infrastructure telemetry, application performance, network health, security events, user experience, and business outcomes into a single operational narrative. Without that correlation, teams spend excessive time investigating individual components while missing the broader operational picture. Jeremy Rossbach, Chief Technical Evangelist, Network Observability, Broadcom

Correlation is going to matter more than simply collecting more telemetry. Most enterprises already have plenty of data. What they lack oftentimes is the ability to connect it quickly enough to understand why something failed. For an AI application, that means correlating model behavior with application performance, databases, infrastructure, networks and external dependencies. For agentic systems, it also means tracing the sequence of actions the agent took and understanding which dependency was involved at each step. Don Boxley, CEO and Co-Founder, DH2i

CONTINUOUS MONITORING OF OUTPUT QUALITY

If you're shopping for an AI observability tool built for LLMs, your top priority should be something that never stops checking output quality once the model hits production. You can’t stop at initial benchmarks, because real-world edge cases always behave differently than test data, and a model that passed evaluation can still drift or degrade once it’s handling live, unpredictable inputs. That means continuous monitoring of response quality, not a one-time stamp of approval before launch. Krishna Sai, CTO, SolarWinds

The evaluation layer must work in development and production. Teams need deterministic checks, model-based judges, human annotation, curated datasets, online monitoring, and methods for calibrating evaluator quality. Agentic evaluation should score the trajectory and task result along with the final response. Aaron Winston, Arize AI

Continuous, automated evaluation of output quality in production, not just in pre-deployment testing. Groundedness and correctness should be scored on live traffic, not assumed to hold once the model ships. Douglas James, VP, Skylar AI, ScienceLogic

Continuous quality evaluation on live traffic: Offline testing before launch isn’t enough, because production inputs are messier than test sets. Tools need to be able to score real production interactions for correctness, relevance, or policy adherence on an ongoing basis, and surface degradation before users complain about it. David Girvin, Lead Technical Advocate, Sumo Logic

The single most important capability is continuous quality evaluation in production. Offline evals before launch are table stakes. What matters is knowing your quality score dropped four points this Tuesday, before your users tell you. Kunal Agarwal, Co-founder and CEO, Unravel Data

Behavioral failure detection

The system needs to identify failures that do not present as technical errors — unresolved outcomes, missed escalations, inappropriate actions, policy deviations, execution errors, and other behavioral failures. This is especially important given that our research found roughly 90% of observed enterprise AI failures were operational rather than traditional hallucination failures. Sekhar Sarukkai, Co-Founder and CEO, ChatSee.ai

Dynamic anomaly detection that learns from actual system behavior matters more than static thresholds set months ago and never revisited. Gowrisankar Chinnayan, Director of Product Management, ManageEngine

Consistent failure taxonomy and diagnosis

Enterprises cannot operationalize agent reliability if every team describes the same underlying failure differently. Failures need to be classified consistently across agents, models, workflows, and time. Sekhar Sarukkai, ChatSee.ai

cross-interaction and cross-agent recurrence detection

If the same missed escalation occurs hundreds of times, the enterprise needs to recognize one systemic problem rather than hundreds of independent anomalous traces. Sekhar Sarukkai, ChatSee.ai

shared failure memory

Prior diagnoses, human adjudications, affected contexts, treatments, and remediation results should become reusable intelligence accessible to humans, assurance systems, and agents. The system should get smarter as the enterprise resolves failures. Sekhar Sarukkai, ChatSee.ai

CAUSAL REASONING

Transparent Causal Reasoning: Provides detailed, step-by-step explanations of why an anomaly occurred, fully backed by concrete evidence and citations (linking directly to OTel trace spans, prompt/response pairs, vector queries or commit diffs). Gou Rao, co-founder and CEO, NeuBird AI

RECONSTRUCTION

The most important capability is context reconstruction. When something goes wrong, a team should be able to reconstruct what the AI system was trying to do, what information it used, what it did next, and where the failure entered the chain. For LLM applications, that means visibility into prompts, context, model responses, evaluation results, feedback, and cost. For agentic systems, it also means timelines of tool calls, browser actions, file edits, workspace versions, external responses, approvals, retries, and recovery attempts. A strong AI observability tool should help a team decide whether to continue, retry, roll back to a known-good version, escalate to a human, or change the agent's action boundaries. Kylian Cros, Co-Founder and CMO, Molted

The most important capability isn’t a dashboard, it’s a black box. Every airplane carries one because when something goes wrong, investigators don’t start with did it fail, they start with why it thought that was the right call. An AI observability tool needs the same thing, the ability to reconstruct what the agent knew, what it decided, and why, after the fact. Corey Beck, VP of Service Delivery, DataStrike

The ability to reconstruct an end-to-end decision path is essential, whether that’s understanding why a model produced a given response or why an agent chose a particular multi-step action, so teams can see not just what happened but why. Stephen Ochs, Head of Marketing, Selector

Proactive Context Seeking

Actively prompts engineers for missing context when signals are ambiguous (e.g., asking whether a sudden token surge stems from a new deployment or an agent loop) to refine hypothesis trees faster. Gou Rao, NeuBird AI

CLEAR AUDIT TRAIL

A clear audit trail for every agent action: Humans review what triggered a decision and what was skipped after the fact, so that the trail is complete. Gowrisankar Chinnayan, Director of Product Management, ManageEngine

Audit trails, access controls, approval gates, and rollback allow agents to investigate and propose changes while people retain authority where judgment and risk require it. Aaron Winston, Arize AI

Action-level audit trail for agents: For agentic systems specifically, the tool needs to record not just what the agent decided but what it actually did which systems it touched, what data it accessed, and what permissions it exercised in a form detailed enough to support both incident investigation and compliance evidence after the fact. David Girvin, Sumo Logic

The most critical capability for monitoring AI is maintaining a controlled, human-in-the-loop experience that allows operators to inspect, approve and reverse any action. Observability capabilities must provide complete transparency into credential usage, ensuring that agent actions are explicitly tagged rather than masked under a human user account. Phil Christianson, CPO, Xurrent

REMEDIATION

Closed-loop remediation and effectiveness measurement: Observability should not end when the root cause appears on a dashboard. It should support the transition from diagnosis into treatment and then verify whether recurrence actually declined — including after subsequent agent, model, prompt, or workflow changes. Sekhar Sarukkai, ChatSee.ai

Assisted Remediation: Recommends and executes concrete recovery actions, such as dynamic model failover, prompt guardrail enforcement or rate-limiting, backed by human-in-the-loop approval controls. Gou Rao, NeuBird AI

IDENTITY AND POLICY CONTEXT

AI observability should preserve identity and policy context. Teams need to know which user, service, or agent initiated an interaction, what permissions were available, which policy was evaluated, what decision was made, and which version of that policy was active. Steve Giguere, Principal AI Security Advocate with the Office of the CTO, Check Point

BUSINESS-AWARE ALERTING

Here's what separates tools that matter from tools that make dashboards: they have to connect observability to action. Drift detection that just alerts is useless. Business-aware alerting that tells you the specific decision type that’s degrading is actionable. Lior Matkovich, Senior Director, ITOM, ServiceNow

PRIVACY CONTROLS

AI observability should provide strong privacy controls. Sensitive content should be selectively captured, redacted, encrypted, access-controlled, and retained according to its purpose. Teams should be able to record the metadata required for investigation without automatically storing every complete prompt and response. Steve Giguere, Principal AI Security Advocate with the Office of the CTO, Check Point

AI-DATA LAKEHOUSE

AI observability needs an AI-data lakehouse that brings together traces, behavior, performance, cost, and availability data from across the stack, then make that context available for analytics, optimization, and feature development. Bernd Greifeneder, CTO and Founder, Dynatrace

DATA RESIDENCY

Data residency: keeping the telemetry inside the customer’s own cloud, so that observing AI never means surrendering the most sensitive data in the business. Gabriel-James Safar, Tsuga

REAL-TIME TRACKING OF USAGE AND COST

You need real-time tracking of token usage and cost. Without it, you’re flying blind on budget, since usage patterns in production rarely match what you projected in testing, and costs can spiral quickly once real traffic and real users are in the mix. The best tools tie quality monitoring and cost visibility together. Krishna Sai, CTO, SolarWinds

Cost and volume management: Given how much telemetry AI systems generate, the tool needs to help teams manage the volume and cost of that data itself, routing and filtering intelligently rather than forcing a choice between blind spots and runaway ingestion costs. Andi Mann, Apica

Cost and token-usage visibility tied to specific workflows also matters, letting teams manage spend proactively instead of discovering a runaway cost or agent only after the bill or the damage has landed. Stephen Ochs, Selector

Cost and performance in the same view: Token usage and cost per interaction need to sit next to latency and quality, because in AI systems those three things trade off against each other constantly, and optimizing one blindly tends to hurt the others. David Girvin, Sumo Logic

Check back tomorrow for Exploring AI Observability – Part 9: Benefits

Pete Goldin is Editor and Publisher of APMdigest

Hot Topics