Skip to main content

Exploring AI Observability – Part 11: Cost Management

Pete Goldin
APMdigest

Two AI agents got caught in a loop with each other over a weekend and racked up $47,000 in charges, according to John D'Emic, CTO of Revenium. No alert fired because nothing "broke."

Stories like this are why cost has become one of the most pressing questions in AI observability. Traditional monitoring was built to catch failures, not wasted spend, and AI systems can burn through budget while every dashboard stays green. Part 11 explores how AI observability relates to AI cost management, and whether they are one discipline or two.

Limited Visibility

"When it comes to both AI observability and AI cost management, it's limited visibility," explains Steve Daheb, CMO at Flexera. "People are looking through a telescope at one slice of their AI program and lack the panoramic view across their organization. They’re missing the forest for the trees. In the rush to deploy AI, many organizations never tracked or governed and today they can't answer fundamental questions of what technology is being used, how it’s being used, by whom, and at what cost."

"Today, most enterprises still have difficulty tracing AI spend to actual outcomes, which makes it harder to understand where investments are creating value," says Gargi Dasgupta, CTO of Mozark AI.

"I've talked to so many companies that can tell you their token spend down to the penny but have no clue whether a specific agent made them money or lost them money," D’Emic of Revenium relates. "That's a big gap in the observability-meets-cost-management picture for enterprise AI deployment. You absolutely need something that connects what agents do to what they produce, not just a number on what it burned through to get there."

Related, Not the Same

AI cost management tells you the size of the bill. AI observability helps explain what created the charges, says Camden Swita, Head of AI at New Relic. 

AI observability and AI cost management are related, but not the same thing, according to Krishna Sai, CTO at SolarWinds. 

"They are different categories today, and the overlap is smaller than most vendors suggest," says Gou Rao, Co-Founder and CEO, NeuBird AI.

"I would not lump them together," Sai from SolarWinds continues. "AI observability is about keeping tabs on how your systems and AI models are actually performing, whether they are drifting, breaking, or behaving oddly. AI cost management, sometimes people call it FinOps for AI, is about tracking what you are spending on compute, tokens, and inference." 

"The two are closely related but serve different purposes," Pretam Mondal, Lead Product Manager, Observability and Service Management, OpenText, agrees. "AI cost management focuses on maximizing efficiency and controlling the resources required to run AI systems. AI observability focuses on understanding how AI systems behave and whether they are delivering the desired outcomes. The disciplines work best together."

"Enterprises with serious spend at stake still tend to want a purpose-built cost management layer for granular chargeback and budget enforcement that a general observability tool isn’t built to do," David Girvin, Lead Technical Advocate, Sumo Logic, clarifies.

Moving Toward Convergence

"Where AI observability and AI cost management overlap is the underlying data," says Sai from SolarWinds. "Both need the same kind of telemetry, like resource usage, latency, and API call volume, so it makes sense that these worlds are starting to bleed into each other."

Andi Mann, Chief Product & Technology Officer at Apica, elaborates: "In practice, the two need to work off the same underlying telemetry, because a cost spike is often a symptom of an observability problem, such as an agent stuck in a retry loop or a prompt pattern that's become inefficient. Enterprises are better served by tools that share a data foundation across both functions rather than standing up separate, disconnected systems for cost and for quality."

Jeremy Rossbach, Chief Technical Evangelist, Network Observability, Broadcom, says, "For example, observability may reveal that an Agentic workflow repeatedly retries failed tool calls or unnecessarily invokes multiple models for a single task. Those operational insights naturally lead to cost optimization. Understanding behavior is often the first step toward controlling cost."

Omer Bhutta, Head of AI and ML at Secure.com, adds: "A cost dashboard can show that spend spiked; only the observability trace can show that the spike came from an agent stuck in a retry loop or a prompt template that doubled in size."

Douglas James, VP, Skylar AI at ScienceLogic, adds, "Cost is one dimension of observability, not a separate discipline. The telemetry that tells you why an agent failed, including the traces, token counts, and model calls, is the same telemetry that tells you where the money went, which is why the two are converging onto shared platforms. Cost management alone answers 'how much.' Observability has to answer how much, how well, and how safely. A tool that optimizes spend without visibility into quality has solved only the cheaper half of the problem."

Brij Mohan Singh, Head of AI at The Modern Data Company, adds, "My expectation is convergence. Cost becomes one more metric inside the same observability platform rather than staying a separate tool, the same way infra cost management eventually got folded into broader cloud observability."

Kunal Agarwal, Co-Founder and CEO, Unravel Data, says, "They’re converging, and they should, because in AI they’re the same decision. Model choice, prompt length, caching strategy, retrieval depth. Every one of those levers moves cost and quality simultaneously. If your cost tool says switch to the cheaper model and your observability tool can’t tell you what that does to accuracy, you're optimizing with one eye closed. It doesn't have to be one product, but it has to be one dataset and one decision."

Cost and Performance Belong Together

"AI cost management is an important dimension of AI observability, but cost should not be considered in isolation," Bernd Greifeneder, CTO and Founder at Dynatrace, explains. "Observability provides the context to understand why costs rise, whether the cause is an increase in demand, inefficient behavior, or an operational issue. Cost and performance belong together: teams need to see how spending relates to reliability, quality, and the outcomes an Agentic AI system delivers."

Gowrisankar Chinnayan, Director of Product Management at ManageEngine, adds: "For LLM workloads, cost and performance are closely linked. A trace that shows a slow response should also show token use and which model was used. This connection helps teams find inefficient prompts, spot models that are too large for the job, and make real cost-performance decisions based on data, not guesses. Whether a single tool manages both depends on the setup, but the signals should be combined. If performance and cost are tracked separately, both teams end up with only part of the picture."

Steve Giguere, Principal AI Security Advocate with the Office of the CTO at Check Point, explains the connection between cost and performance: "The most useful question is not simply, 'How many tokens did this agent consume?' It is, 'What did it cost to complete a successful, policy-compliant task?' That requires cost data to be correlated with execution traces, task outcomes, quality, security decisions, and human intervention." 

Traditional AI cost management examines model selection, token consumption, caching, latency, and infrastructure utilization. Those are important input costs. But for agentic AI, minimizing inference cost alone can create a false economy, explains  Sekhar Sarukkai, Co-Founder and CEO, ChatSee.ai. A cheaper model that results in more unresolved cases, unnecessary escalations, incorrect actions, human reviews, or expensive incident investigations may produce a higher overall operating cost. The more meaningful economic model therefore combines the cost of execution with the cost of failure. As agents increasingly perform business processes, AI cost management will need observability data that connects compute consumption to completion rates, failure rates, human intervention, investigation cost, propagation, reversibility, and ultimately business outcomes.

AI cost management and AI observability should share a data foundation because cost without behavior is easy to misread, advises Aaron Winston, Head of Content, Communications, and Analyst Relations at Arize AI. Cost management identifies where money is being spent across models, tokens, caching, tools, compute, users, sessions, workflows, and agents. Observability explains why a run became expensive, whether the system succeeded, and what tradeoff was made between quality, latency, and cost.

For an LLM application, request-level cost, quality per dollar, and cost by use case are useful, Winston continues. In an agentic system, long trajectories, repeated tool calls, retries, loops, sandbox execution, external services, and human review can outweigh the initial model call. Cost per successful outcome becomes the most useful unit. These capabilities can live in one product or in connected systems, but the telemetry, identifiers, and outcome definitions must be shared.

Daheb from Flexera adds, "Visibility into performance without visibility into cost leaves unanswered questions about value. Cost visibility without operational insight leaves unanswered questions about effectiveness. Together, AI observability and cost management give organizations the visibility and insight needed to make better investment decisions, govern AI more effectively, and scale adoption with clarity and confidence."

In the end, managing AI costs isn't just about lowering expenses, observes Dasgupta from Mozark. It's about connecting what is being spent to how AI is performing and the experience it is delivering, so organizations can spend where the earnings come from.

Both Cost and Performance in One Dashboard

Most experts see the current separation between cost tools and observability tools as a temporary state of the market rather than a permanent architecture.

Krishna Sai at SolarWinds predicted the disciplines would stay organizationally distinct for a while even as the tooling converges: "My take is these will stay separate disciplines for a while, often built by different teams, but customers are going to want them living inside the same platform. Nobody wants to flip between two different tools to connect a performance problem to a cost problem."

Lior Matkovich, Senior Director, ITOM, ServiceNow, says: "You need one system that shows both dimensions simultaneously. When you see an agent burning tokens, you need to know: is it overthinking because the problem is genuinely complex, or is it stuck in a loop? Is accuracy holding up? Can you afford this decision at scale? You can also get full control on what what models have been used, by whom, and what prompts have been used."

"Observability should be capturing token consumption at every single stage, not just the total," explains Joseph Voyles, Principal, AI & Emerging Technology, PwC US. "That matters even more with agentic systems because they’re way more token hungry than a simple LLM call. When you can see token use step by step, you can actually build a strategy to bring it down. You can isolate the steps that are eating up the most tokens and test cheaper models against them to see if they hold up. This is really the only way to understand what these systems actually cost. The true cost only becomes clear once you’re measuring real-world usage. And you don’t need an entirely new toolset to do it. Traditional logging still works – you just need a solution that brings everything together into a single, end-to-end view."

Escalating a request to a frontier model instead of routing it to something smaller can cost many times more, and without observability you can't see which requests actually needed that escalation and which didn't, says Cameron Daniel, CTO at Megaport. That's the balancing act tech and finance leaders are working through right now: How much quality do you genuinely need for a given task, and where's the threshold before you're paying for capability nobody's actually using?

Gidi Cohen, Co-Founder and CEO, Bonfy AI, concludes: "Observability data is what makes smart cost decisions possible, and the best setups blend both into one view."

Dan Albright, Global AdvisoryX Leader at DXC Technology, warns "Organizations that treat observability and cost management as separate concerns will struggle to keep AI spend under control and remain competitive."

Go to: Exploring AI Observability – Part 12: AI Monitoring AI 

Pete Goldin is Editor and Publisher of APMdigest

The Latest

"AI readiness" has quickly become one of the defining priorities of the modern enterprise. Business decision makers across virtually every industry are working to build roadmaps, solidify strategies, and set expectations around technological investment and adoption. However, at the very same time these leaders are laying their plans, the state of AI continues to change and shift beneath their feet. This raises a fundamental question about the notion of AI readiness: what is it we're actually getting ready for? ...

Genpact and HFS Research recently found that 92% of senior executives at Global 2000 companies believe agentic AI will fundamentally change how work gets done. Yet 85% of them also recognize enterprise debt as a factor limiting their AI value. To move beyond pilots and deliver tangible value, CIOs need to address enterprise debt — the accumulated operational drag from outdated technology, fragmented data, inefficient processes, and talent gaps ...

Proactive is going to become the default expectation, not a nice-to-have. Predicting problems before they happen and fixing them ahead of time will become table stakes, especially for smaller teams and MSPs who cannot afford a big NOC staff ...

The final two installments of the series, Exploring AI Observability, cover predictions from the experts about where AI observability is heading ... In the next few years, tracking your AI won't just be a nice-to-have tool, it will be a survival requirement for businesses ...

Cloud absolutely changed IT for the better. But after more than a decade of using it, companies have also learned something else. Where an application runs matters, and there isn't one environment that makes sense for everything ...

Exploring AI Observability – Part 11: Cost Management

Pete Goldin
APMdigest

Two AI agents got caught in a loop with each other over a weekend and racked up $47,000 in charges, according to John D'Emic, CTO of Revenium. No alert fired because nothing "broke."

Stories like this are why cost has become one of the most pressing questions in AI observability. Traditional monitoring was built to catch failures, not wasted spend, and AI systems can burn through budget while every dashboard stays green. Part 11 explores how AI observability relates to AI cost management, and whether they are one discipline or two.

Limited Visibility

"When it comes to both AI observability and AI cost management, it's limited visibility," explains Steve Daheb, CMO at Flexera. "People are looking through a telescope at one slice of their AI program and lack the panoramic view across their organization. They’re missing the forest for the trees. In the rush to deploy AI, many organizations never tracked or governed and today they can't answer fundamental questions of what technology is being used, how it’s being used, by whom, and at what cost."

"Today, most enterprises still have difficulty tracing AI spend to actual outcomes, which makes it harder to understand where investments are creating value," says Gargi Dasgupta, CTO of Mozark AI.

"I've talked to so many companies that can tell you their token spend down to the penny but have no clue whether a specific agent made them money or lost them money," D’Emic of Revenium relates. "That's a big gap in the observability-meets-cost-management picture for enterprise AI deployment. You absolutely need something that connects what agents do to what they produce, not just a number on what it burned through to get there."

Related, Not the Same

AI cost management tells you the size of the bill. AI observability helps explain what created the charges, says Camden Swita, Head of AI at New Relic. 

AI observability and AI cost management are related, but not the same thing, according to Krishna Sai, CTO at SolarWinds. 

"They are different categories today, and the overlap is smaller than most vendors suggest," says Gou Rao, Co-Founder and CEO, NeuBird AI.

"I would not lump them together," Sai from SolarWinds continues. "AI observability is about keeping tabs on how your systems and AI models are actually performing, whether they are drifting, breaking, or behaving oddly. AI cost management, sometimes people call it FinOps for AI, is about tracking what you are spending on compute, tokens, and inference." 

"The two are closely related but serve different purposes," Pretam Mondal, Lead Product Manager, Observability and Service Management, OpenText, agrees. "AI cost management focuses on maximizing efficiency and controlling the resources required to run AI systems. AI observability focuses on understanding how AI systems behave and whether they are delivering the desired outcomes. The disciplines work best together."

"Enterprises with serious spend at stake still tend to want a purpose-built cost management layer for granular chargeback and budget enforcement that a general observability tool isn’t built to do," David Girvin, Lead Technical Advocate, Sumo Logic, clarifies.

Moving Toward Convergence

"Where AI observability and AI cost management overlap is the underlying data," says Sai from SolarWinds. "Both need the same kind of telemetry, like resource usage, latency, and API call volume, so it makes sense that these worlds are starting to bleed into each other."

Andi Mann, Chief Product & Technology Officer at Apica, elaborates: "In practice, the two need to work off the same underlying telemetry, because a cost spike is often a symptom of an observability problem, such as an agent stuck in a retry loop or a prompt pattern that's become inefficient. Enterprises are better served by tools that share a data foundation across both functions rather than standing up separate, disconnected systems for cost and for quality."

Jeremy Rossbach, Chief Technical Evangelist, Network Observability, Broadcom, says, "For example, observability may reveal that an Agentic workflow repeatedly retries failed tool calls or unnecessarily invokes multiple models for a single task. Those operational insights naturally lead to cost optimization. Understanding behavior is often the first step toward controlling cost."

Omer Bhutta, Head of AI and ML at Secure.com, adds: "A cost dashboard can show that spend spiked; only the observability trace can show that the spike came from an agent stuck in a retry loop or a prompt template that doubled in size."

Douglas James, VP, Skylar AI at ScienceLogic, adds, "Cost is one dimension of observability, not a separate discipline. The telemetry that tells you why an agent failed, including the traces, token counts, and model calls, is the same telemetry that tells you where the money went, which is why the two are converging onto shared platforms. Cost management alone answers 'how much.' Observability has to answer how much, how well, and how safely. A tool that optimizes spend without visibility into quality has solved only the cheaper half of the problem."

Brij Mohan Singh, Head of AI at The Modern Data Company, adds, "My expectation is convergence. Cost becomes one more metric inside the same observability platform rather than staying a separate tool, the same way infra cost management eventually got folded into broader cloud observability."

Kunal Agarwal, Co-Founder and CEO, Unravel Data, says, "They’re converging, and they should, because in AI they’re the same decision. Model choice, prompt length, caching strategy, retrieval depth. Every one of those levers moves cost and quality simultaneously. If your cost tool says switch to the cheaper model and your observability tool can’t tell you what that does to accuracy, you're optimizing with one eye closed. It doesn't have to be one product, but it has to be one dataset and one decision."

Cost and Performance Belong Together

"AI cost management is an important dimension of AI observability, but cost should not be considered in isolation," Bernd Greifeneder, CTO and Founder at Dynatrace, explains. "Observability provides the context to understand why costs rise, whether the cause is an increase in demand, inefficient behavior, or an operational issue. Cost and performance belong together: teams need to see how spending relates to reliability, quality, and the outcomes an Agentic AI system delivers."

Gowrisankar Chinnayan, Director of Product Management at ManageEngine, adds: "For LLM workloads, cost and performance are closely linked. A trace that shows a slow response should also show token use and which model was used. This connection helps teams find inefficient prompts, spot models that are too large for the job, and make real cost-performance decisions based on data, not guesses. Whether a single tool manages both depends on the setup, but the signals should be combined. If performance and cost are tracked separately, both teams end up with only part of the picture."

Steve Giguere, Principal AI Security Advocate with the Office of the CTO at Check Point, explains the connection between cost and performance: "The most useful question is not simply, 'How many tokens did this agent consume?' It is, 'What did it cost to complete a successful, policy-compliant task?' That requires cost data to be correlated with execution traces, task outcomes, quality, security decisions, and human intervention." 

Traditional AI cost management examines model selection, token consumption, caching, latency, and infrastructure utilization. Those are important input costs. But for agentic AI, minimizing inference cost alone can create a false economy, explains  Sekhar Sarukkai, Co-Founder and CEO, ChatSee.ai. A cheaper model that results in more unresolved cases, unnecessary escalations, incorrect actions, human reviews, or expensive incident investigations may produce a higher overall operating cost. The more meaningful economic model therefore combines the cost of execution with the cost of failure. As agents increasingly perform business processes, AI cost management will need observability data that connects compute consumption to completion rates, failure rates, human intervention, investigation cost, propagation, reversibility, and ultimately business outcomes.

AI cost management and AI observability should share a data foundation because cost without behavior is easy to misread, advises Aaron Winston, Head of Content, Communications, and Analyst Relations at Arize AI. Cost management identifies where money is being spent across models, tokens, caching, tools, compute, users, sessions, workflows, and agents. Observability explains why a run became expensive, whether the system succeeded, and what tradeoff was made between quality, latency, and cost.

For an LLM application, request-level cost, quality per dollar, and cost by use case are useful, Winston continues. In an agentic system, long trajectories, repeated tool calls, retries, loops, sandbox execution, external services, and human review can outweigh the initial model call. Cost per successful outcome becomes the most useful unit. These capabilities can live in one product or in connected systems, but the telemetry, identifiers, and outcome definitions must be shared.

Daheb from Flexera adds, "Visibility into performance without visibility into cost leaves unanswered questions about value. Cost visibility without operational insight leaves unanswered questions about effectiveness. Together, AI observability and cost management give organizations the visibility and insight needed to make better investment decisions, govern AI more effectively, and scale adoption with clarity and confidence."

In the end, managing AI costs isn't just about lowering expenses, observes Dasgupta from Mozark. It's about connecting what is being spent to how AI is performing and the experience it is delivering, so organizations can spend where the earnings come from.

Both Cost and Performance in One Dashboard

Most experts see the current separation between cost tools and observability tools as a temporary state of the market rather than a permanent architecture.

Krishna Sai at SolarWinds predicted the disciplines would stay organizationally distinct for a while even as the tooling converges: "My take is these will stay separate disciplines for a while, often built by different teams, but customers are going to want them living inside the same platform. Nobody wants to flip between two different tools to connect a performance problem to a cost problem."

Lior Matkovich, Senior Director, ITOM, ServiceNow, says: "You need one system that shows both dimensions simultaneously. When you see an agent burning tokens, you need to know: is it overthinking because the problem is genuinely complex, or is it stuck in a loop? Is accuracy holding up? Can you afford this decision at scale? You can also get full control on what what models have been used, by whom, and what prompts have been used."

"Observability should be capturing token consumption at every single stage, not just the total," explains Joseph Voyles, Principal, AI & Emerging Technology, PwC US. "That matters even more with agentic systems because they’re way more token hungry than a simple LLM call. When you can see token use step by step, you can actually build a strategy to bring it down. You can isolate the steps that are eating up the most tokens and test cheaper models against them to see if they hold up. This is really the only way to understand what these systems actually cost. The true cost only becomes clear once you’re measuring real-world usage. And you don’t need an entirely new toolset to do it. Traditional logging still works – you just need a solution that brings everything together into a single, end-to-end view."

Escalating a request to a frontier model instead of routing it to something smaller can cost many times more, and without observability you can't see which requests actually needed that escalation and which didn't, says Cameron Daniel, CTO at Megaport. That's the balancing act tech and finance leaders are working through right now: How much quality do you genuinely need for a given task, and where's the threshold before you're paying for capability nobody's actually using?

Gidi Cohen, Co-Founder and CEO, Bonfy AI, concludes: "Observability data is what makes smart cost decisions possible, and the best setups blend both into one view."

Dan Albright, Global AdvisoryX Leader at DXC Technology, warns "Organizations that treat observability and cost management as separate concerns will struggle to keep AI spend under control and remain competitive."

Go to: Exploring AI Observability – Part 12: AI Monitoring AI 

Pete Goldin is Editor and Publisher of APMdigest

The Latest

"AI readiness" has quickly become one of the defining priorities of the modern enterprise. Business decision makers across virtually every industry are working to build roadmaps, solidify strategies, and set expectations around technological investment and adoption. However, at the very same time these leaders are laying their plans, the state of AI continues to change and shift beneath their feet. This raises a fundamental question about the notion of AI readiness: what is it we're actually getting ready for? ...

Genpact and HFS Research recently found that 92% of senior executives at Global 2000 companies believe agentic AI will fundamentally change how work gets done. Yet 85% of them also recognize enterprise debt as a factor limiting their AI value. To move beyond pilots and deliver tangible value, CIOs need to address enterprise debt — the accumulated operational drag from outdated technology, fragmented data, inefficient processes, and talent gaps ...

Proactive is going to become the default expectation, not a nice-to-have. Predicting problems before they happen and fixing them ahead of time will become table stakes, especially for smaller teams and MSPs who cannot afford a big NOC staff ...

The final two installments of the series, Exploring AI Observability, cover predictions from the experts about where AI observability is heading ... In the next few years, tracking your AI won't just be a nice-to-have tool, it will be a survival requirement for businesses ...

Cloud absolutely changed IT for the better. But after more than a decade of using it, companies have also learned something else. Where an application runs matters, and there isn't one environment that makes sense for everything ...