Datadog announced the general availability of LLM Observability, which allows AI application developers and machine learning (ML) engineers to efficiently monitor, improve and secure large language model (LLM) applications.
With LLM Observability, companies can accelerate the deployment of generative AI applications to production environments and scale them reliably.
Datadog LLM Observability helps customers confidently deploy and monitor their generative AI applications. This new product provides visibility into each step of the LLM chain to easily identify the root cause of errors and unexpected responses such as hallucinations. Users can also monitor operational metrics like latency and token usage to optimize performance and cost, and can evaluate the quality of their AI applications—such as topic relevance or toxicity—and gain insights to mitigate security and privacy risks with out-of-the-box quality and safety evaluations.
Datadog’s LLM Observability offers prompt and response clustering, seamless integration with Datadog Application Performance Monitoring (APM), and out-of-the-box evaluation and sensitive data scanning capabilities to enhance the performance, accuracy and security of generative AI applications while helping to keep data private and secure.
“There’s a rush to adopt new LLM-based technologies, but organizations of all sizes and industries are finding it difficult to do so in a way that is both cost effective and doesn’t negatively impact the end user experience,” said Yrieix Garnier, VP of Product at Datadog. “Datadog LLM Observability provides the deep visibility needed to help teams manage and understand performance, detect drifts or biases, and resolve issues before they have a significant impact on the business or end-user experience.”
LLM Observability helps organizations:
- Evaluate Inference Quality: Visualize the quality and effectiveness of LLM applications’ conversations—such as failure to answer—to monitor any hallucinations, drifts and the overall experience of the apps’ end users.
- Identify Root Causes: Quickly pinpoint the root cause of errors and failures in the LLM chain with full visibility into end-to-end traces for each user request.
- Improve Costs and Performance: Efficiently monitor key operational metrics for applications across all major platforms—including OpenAI, Anthropic, Azure OpenAI, Amazon Bedrock, Vertex AI and more—in a unified dashboard to uncover opportunities for performance and cost optimization.
- Protect Against Security Threats: Safeguard applications against prompt hacking and help prevent leaks of sensitive data, such as PII, emails and IP addresses, using built-in security and privacy scanners powered by Datadog Sensitive Data Scanner.
Datadog LLM Observability is generally available now.
The Latest
Two years ago, almost every customer conversation about AI started with the same questions: Which model should we use? What can it do? Is it ready for the enterprise? Today, those discussions have moved on. CIOs are far more interested in how to govern AI, integrate it with existing systems, prepare their workforce and make it part of everyday operations. The challenge is no longer to prove that AI can deliver value. It's instead about how to embed AI into the business in a way that's secure, scalable and delivers measurable outcomes ...
Two things happened to production incidents between 2023 and now, and they did not happen at the same speed. The first is that a class of dependency that barely existed three years ago now accounts for one incident in ten. Incidents disclosed by AI model and AI application providers rose from 1.7% of all disclosed unplanned incidents in 2023 to 10.7% in 2026 year to date, roughly a sixfold rise; that counts only incidents at AI companies themselves, so the true share is higher. The second is that the time to close an incident has not come down ...
When an AI assistant gives an incomplete or incorrect answer, teams often blame the model. They adjust prompts, switch models, increase context windows or test a new retrieval strategy. However the model may not be a problem. In many enterprise AI workflows, the problem begins inside the document-ingestion pipeline ...
If you talk to any security or observability teams right now, they're all fighting the same fire: their tooling was built to ingest X, but their sources are pumping Y and soon to be doing Z. The knee-jerk reaction is always the same: we need more platform. However, this reaction is wrong. Let me explain why, because the solution to this problem is foundational, not financial. Instead of hurling yet more money at the problem, make sure you've done what's needed upstream ...
Rapid AI adoption and the unique ways AI workloads operate is redefining the scope and structure of what these teams must deliver. This shift is forcing organizations to rethink how they manage scale, automation, and control, according to The State of SRE and Platform Engineering 2026, a new report from Dynatrace ...
AI is usually talked about as a software tool, but it also depends heavily on the network behind it. Whether a company is using AI for chatbots, automation, monitoring, analytics, or employee support, all of that information has to move across the network in a reliable and secure way. That means AI is not just an application decision. It is also an infrastructure decision. Before organizations rush into AI, they should ask a simple question: Is our network ready to support it? ...
Enterprise AI often lacks governed access to where business processes actually execute. Without that access, AI agents may be able to reason, but they cannot operate reliably across enterprise workflows. For AI agents to effectively carry out workflows, they will require integration-layer context and controls. Organizations can implement these prerequisites by providing AI with managed access to the middleware layer ...
Enterprise networks rarely behave the same way for very long. A routing adjustment in one region may unexpectedly alter application performance in another. A cloud migration may introduce hidden dependencies that go unnoticed until an outage occurs. All the while, the network is managed by several different teams, each of whom use different tool sets — and as a result, have different views of the network ... There’s usually an engineer who remembers why traffic fails over a certain way between sites, or which transparent firewall was added where. The problem is that human memory cannot scale alongside enterprise-scale networks ...
Ask an infrastructure team how confident they are in their ability to govern AI, and most will tell you they've got it handled. A recent survey of 406 IT decision-makers and platform engineering leaders found 86% expressing exactly that confidence. Ask the same group whether they have a formal written AI governance policy, and the number drops to 30%, according to Spacelift's Infrastructure Automation Report ...
In MEAN TIME TO INSIGHT Episode 27, Shamus McGillicuddy, EMA VP of Research, Network Infrastructure and Operations, and Parker Hathcock, EMA Research Director covering IT Service/Operations (ServiceOps), discuss observability unification in modern IT operations ...