Skip to main content

The APM Blog

Dan Twing
EMA

Automation often starts with the people closest to an immediate need. A developer may schedule a script with cron or Amazon EventBridge Scheduler; a business team may adopt an RPA bot, low-code platform, or agent. Many tools stay useful within one domain. Some become consequential across organizational boundaries because other systems and teams depend on them or business outcomes rely on their timing and recovery. Those cases may call for broader coordination ...

Eshaan Jain

Enterprise AI projects are often justified with a familiar metric: hours saved. A team automates a manual process, employees spend fewer hours reviewing documents or entering data, and the productivity gain becomes the headline. Time savings matter, but they capture only part of the financial impact AI can have inside an organization ... A more useful way to evaluate enterprise AI is to examine the financial decisions and risks surrounding the workflows it supports ...

Nitish Mane
Philo

We run an internal AI assistant that answers engineering questions in Slack ... Each question kicks off what we call a turn ... We could already audit a single turn after the fact. Every reply carries a short lookup token in its footer, and a slash command pulls up the full receipt for that turn. What we could not do was answer questions about the whole fleet. How long do turns take? What do they cost? Which tools does the agent reach for? Where does the time go inside a single answer? Those are APM questions. So we treated the agent like any other service and instrumented it with OpenTelemetry, end to end. This post covers what we built, the design choices that turned out to matter, and the things that surprised us ...

Dan Twing
EMA

Applications have long used bounded, deterministic logic to control what happens within their own functions. Process orchestration coordinates activity across people, applications, decisions, and automation tools. Workload orchestration coordinates executable work and dependencies across applications, platforms, and infrastructure. As agentic capabilities enter applications and domain systems, more planning and decision-making will also occur inside those environments. These forms of automation will coexist. Sometimes they will overlap, and they will increasingly need to exchange triggers, state, and results. But they are not becoming interchangeable ...

Marcelo Tamassia
Syntax

For many organizations, the AI agenda has landed squarely on the CIO's desk. At first glance, that makes perfect sense. AI depends on data, infrastructure, security, integration, governance and scalable technology platforms — all areas where the CIO plays a central role. But there is a problem with treating AI primarily as an IT initiative: AI is not simply a technology deployment. It’s also a business transformation ...

Pete Goldin
APMdigest

Who is watching the watcher? This may not be the first question that comes up when discussing AI observability but sooner or later you start to wonder. Part 12 covers the dilemma of AI monitoring AI ...

Pete Goldin
APMdigest

Two AI agents got caught in a loop with each other over a weekend and racked up $47,000 in charges, according to John D'Emic, CTO of Revenium. No alert fired because nothing "broke." Stories like this are why cost has become one of the most pressing questions in AI observability. Traditional monitoring was built to catch failures, not wasted spend, and AI systems can burn through budget while every dashboard stays green. Part 11 explores how AI observability relates to AI cost management, and whether they are one discipline or two ...

Dan Twing
EMA

AI has sharply increased attention on automation and raised expectations for what it can do. It has also brought the fragmentation of enterprise automation into sharper focus: activity spans systems, teams, and domains, while visibility and authority remain divided among them. Control planes are familiar in other technical contexts, but applying the concept to enterprise automation is a newer development. The emerging enterprise control plane is a way to rationalize that fragmented environment through broader visibility, coordination, and governance, even as some capabilities consolidate and others remain specialized ...

Herb Blecher
Enterprise Management Associates (EMA)

Anthropic announced Claude Dashboards following OpenAI's similar release in September. All of a sudden, the world's gone from the end of dashboards to being awash with them. I set it loose on my "clean" cloud data, and it did a great job of building a visually appealing dashboard, complete with SQL logic, in minutes. Then I set it loose on the "dirty" data … and it did a great job of building a visually appealing dashboard, complete with SQL logic, in minutes. Both looked great. Both had traceability. One was right ...

Pete Goldin
APMdigest

Every new layer of the technology stack eventually faces the same question from the CFO: what are we getting for this? AI observability is no exception. The experts agree the payoff is real, but proving it means rethinking what ROI means for AI. Part 10 covers the return on investment for AI observability ...

Pete Goldin
APMdigest

The bottom-line benefit of AI observability is the ability to balance innovation speed, risk, and cost. It can help teams improve time to market, resilience, and operational efficiency by reducing the time and effort needed to understand and resolve issues. ROI should not be measured through cost alone; organizations should assess whether their AI systems are delivering measurable business value while operating reliably and efficiently ...

Pete Goldin
APMdigest

When it's time for an enterprise to actually evaluate an AI observability tool, the key question is: what are the most important features or capabilities that an AI observability tool should provide? Part 8 outlines the features needed for AI observability ...

Pete Goldin
APMdigest

By now, this series has made the case that AI observability is its own discipline, with its own data, its own metrics, and arguably its own set of stakeholders. That naturally raises a very practical question for any enterprise: does any of this mean starting over? Or can the observability, APM, and AIOps platforms already running in production actually do the job? Part 7 covers the tools needed for AI observability ...

Pete Goldin
APMdigest

Ask an operations team what they monitor and you will hear a familiar list: latency, errors, throughput, uptime. Those signals still matter for AI, but they no longer answer the most important question. As Krishna Sai, CTO of SolarWinds, puts it: "Traditional observability tells you a system is running. AI observability tells you whether it's still reasoning soundly. Those are completely different questions" ...

Pete Goldin
APMdigest

AI observability serves several roles across an organization. AI engineers use it to build, evaluate, and improve AI systems. SREs use it to operate those systems reliably and efficiently in production, including managing their cost. Leaders need it to connect AI behavior and operational performance to business outcomes. The common need is a clear understanding of whether AI systems are delivering the intended result ...

Pete Goldin
APMdigest

Part 4 outlines the challenges enterprise face when adopting AI observability. The main challenge is that AI complexity compounds at every stage of maturity. LLM-based applications introduce new behavior, dependencies, and quality considerations, while Agentic AI adds further failure modes as systems make decisions and take actions ...

Pete Goldin
APMdigest

Part 3 reports on the current state of AI observability. Most experts agree: enterprises are moving faster with AI than they are with the operational discipline needed to run it safely ...

Pete Goldin
APMdigest

When discussing AI Observability, one of the first questions that arises: Is this just an extension of traditional observability or is this a whole new type of tool? Part 2 compares traditional observability with AI observability ...

Pete Goldin
APMdigest

In this extensive 14-part series that will post over the next three weeks, experts across observability and AI will provide insight and opinions on the need for AI Observability, the challenges, the benefits, the current state, and predictions for the future ...

Avery Chi
Made4Vital

AI, automation, and enterprise software give organizations an extraordinary amount of leverage. Work that once required hours can happen in minutes. Information buried across systems can be analyzed almost instantly. Complex workflows can be coordinated across teams with far less manual intervention. That leverage creates a temptation to start with the technology ...

The APM Blog

Dan Twing
EMA

Automation often starts with the people closest to an immediate need. A developer may schedule a script with cron or Amazon EventBridge Scheduler; a business team may adopt an RPA bot, low-code platform, or agent. Many tools stay useful within one domain. Some become consequential across organizational boundaries because other systems and teams depend on them or business outcomes rely on their timing and recovery. Those cases may call for broader coordination ...

Eshaan Jain

Enterprise AI projects are often justified with a familiar metric: hours saved. A team automates a manual process, employees spend fewer hours reviewing documents or entering data, and the productivity gain becomes the headline. Time savings matter, but they capture only part of the financial impact AI can have inside an organization ... A more useful way to evaluate enterprise AI is to examine the financial decisions and risks surrounding the workflows it supports ...

Nitish Mane
Philo

We run an internal AI assistant that answers engineering questions in Slack ... Each question kicks off what we call a turn ... We could already audit a single turn after the fact. Every reply carries a short lookup token in its footer, and a slash command pulls up the full receipt for that turn. What we could not do was answer questions about the whole fleet. How long do turns take? What do they cost? Which tools does the agent reach for? Where does the time go inside a single answer? Those are APM questions. So we treated the agent like any other service and instrumented it with OpenTelemetry, end to end. This post covers what we built, the design choices that turned out to matter, and the things that surprised us ...

Dan Twing
EMA

Applications have long used bounded, deterministic logic to control what happens within their own functions. Process orchestration coordinates activity across people, applications, decisions, and automation tools. Workload orchestration coordinates executable work and dependencies across applications, platforms, and infrastructure. As agentic capabilities enter applications and domain systems, more planning and decision-making will also occur inside those environments. These forms of automation will coexist. Sometimes they will overlap, and they will increasingly need to exchange triggers, state, and results. But they are not becoming interchangeable ...

Marcelo Tamassia
Syntax

For many organizations, the AI agenda has landed squarely on the CIO's desk. At first glance, that makes perfect sense. AI depends on data, infrastructure, security, integration, governance and scalable technology platforms — all areas where the CIO plays a central role. But there is a problem with treating AI primarily as an IT initiative: AI is not simply a technology deployment. It’s also a business transformation ...

Pete Goldin
APMdigest

Who is watching the watcher? This may not be the first question that comes up when discussing AI observability but sooner or later you start to wonder. Part 12 covers the dilemma of AI monitoring AI ...

Pete Goldin
APMdigest

Two AI agents got caught in a loop with each other over a weekend and racked up $47,000 in charges, according to John D'Emic, CTO of Revenium. No alert fired because nothing "broke." Stories like this are why cost has become one of the most pressing questions in AI observability. Traditional monitoring was built to catch failures, not wasted spend, and AI systems can burn through budget while every dashboard stays green. Part 11 explores how AI observability relates to AI cost management, and whether they are one discipline or two ...

Dan Twing
EMA

AI has sharply increased attention on automation and raised expectations for what it can do. It has also brought the fragmentation of enterprise automation into sharper focus: activity spans systems, teams, and domains, while visibility and authority remain divided among them. Control planes are familiar in other technical contexts, but applying the concept to enterprise automation is a newer development. The emerging enterprise control plane is a way to rationalize that fragmented environment through broader visibility, coordination, and governance, even as some capabilities consolidate and others remain specialized ...

Herb Blecher
Enterprise Management Associates (EMA)

Anthropic announced Claude Dashboards following OpenAI's similar release in September. All of a sudden, the world's gone from the end of dashboards to being awash with them. I set it loose on my "clean" cloud data, and it did a great job of building a visually appealing dashboard, complete with SQL logic, in minutes. Then I set it loose on the "dirty" data … and it did a great job of building a visually appealing dashboard, complete with SQL logic, in minutes. Both looked great. Both had traceability. One was right ...

Pete Goldin
APMdigest

Every new layer of the technology stack eventually faces the same question from the CFO: what are we getting for this? AI observability is no exception. The experts agree the payoff is real, but proving it means rethinking what ROI means for AI. Part 10 covers the return on investment for AI observability ...

Pete Goldin
APMdigest

The bottom-line benefit of AI observability is the ability to balance innovation speed, risk, and cost. It can help teams improve time to market, resilience, and operational efficiency by reducing the time and effort needed to understand and resolve issues. ROI should not be measured through cost alone; organizations should assess whether their AI systems are delivering measurable business value while operating reliably and efficiently ...

Pete Goldin
APMdigest

When it's time for an enterprise to actually evaluate an AI observability tool, the key question is: what are the most important features or capabilities that an AI observability tool should provide? Part 8 outlines the features needed for AI observability ...

Pete Goldin
APMdigest

By now, this series has made the case that AI observability is its own discipline, with its own data, its own metrics, and arguably its own set of stakeholders. That naturally raises a very practical question for any enterprise: does any of this mean starting over? Or can the observability, APM, and AIOps platforms already running in production actually do the job? Part 7 covers the tools needed for AI observability ...

Pete Goldin
APMdigest

Ask an operations team what they monitor and you will hear a familiar list: latency, errors, throughput, uptime. Those signals still matter for AI, but they no longer answer the most important question. As Krishna Sai, CTO of SolarWinds, puts it: "Traditional observability tells you a system is running. AI observability tells you whether it's still reasoning soundly. Those are completely different questions" ...

Pete Goldin
APMdigest

AI observability serves several roles across an organization. AI engineers use it to build, evaluate, and improve AI systems. SREs use it to operate those systems reliably and efficiently in production, including managing their cost. Leaders need it to connect AI behavior and operational performance to business outcomes. The common need is a clear understanding of whether AI systems are delivering the intended result ...

Pete Goldin
APMdigest

Part 4 outlines the challenges enterprise face when adopting AI observability. The main challenge is that AI complexity compounds at every stage of maturity. LLM-based applications introduce new behavior, dependencies, and quality considerations, while Agentic AI adds further failure modes as systems make decisions and take actions ...

Pete Goldin
APMdigest

Part 3 reports on the current state of AI observability. Most experts agree: enterprises are moving faster with AI than they are with the operational discipline needed to run it safely ...

Pete Goldin
APMdigest

When discussing AI Observability, one of the first questions that arises: Is this just an extension of traditional observability or is this a whole new type of tool? Part 2 compares traditional observability with AI observability ...

Pete Goldin
APMdigest

In this extensive 14-part series that will post over the next three weeks, experts across observability and AI will provide insight and opinions on the need for AI Observability, the challenges, the benefits, the current state, and predictions for the future ...

Avery Chi
Made4Vital

AI, automation, and enterprise software give organizations an extraordinary amount of leverage. Work that once required hours can happen in minutes. Information buried across systems can be analyzed almost instantly. Complex workflows can be coordinated across teams with far less manual intervention. That leverage creates a temptation to start with the technology ...