Skip to main content

The State of Cloud Costs 2024

Containers are a common theme of wasted spend among organizations, according to the State of Cloud Costs 2024 report from Datadog.

In fact, 83% of container costs were associated with idle resources. About 54% of this wasted spend was on cluster idle, which is the cost of overprovisioning cluster infrastructure, while 29% was associated with workload idle, which comes from resource requests that are larger than their workloads require. This wasted spend comes as organizations allocate more of their EC2 compute to running containers, up to 35% compared to 30% a year ago.

Other report findings include:

GPU Spend Increasing

The report found organizations that use graphics processing unit (GPU) instances have increased their average spending on those instances by 40% in the last year. This growth in spend on GPU instances comes as more companies are experimenting with AI and large language models (LLMs). GPUs' capacity for parallel processing makes them critical for training LLMs and executing other AI workloads, where they can be more than 200% faster than CPUs.

"Today, the most widely used type of GPU-based instance is also the least expensive. This suggests that many customers are still in the experimentation phase with AI and applying the GPU instance to their early efforts in adaptive AI, machine learning inference and small-scale training," said Yrieix Garnier, VP of Product at Datadog. "We expect that as organizations expand their AI activities and move them into production, they will be spending a larger proportion of their cloud compute budget as they use more expensive types of GPU-based instances."

Outdated Technologies Are Widely Used

AWS's current infrastructure offerings commonly both outperform their previous-generation versions and cost less, but 83% of organizations still spend an average of 17% of their EC2 budgets on previous-generation technologies.

Cross-AZ traffic makes up half of data transfer costs

The report states that, "On average, organizations spend almost as much on sending data from one availability zone (AZ) to another as they do on all other types of data transfer combined — including VPNs, gateways, ingress, and egress."

The report found that 98% of organizations are affected by cross-AZ charges, representing an opportunity to optimize cloud costs, such as by colocating related resources within a single AZ whenever availability requirements allow.

"In some cases, cloud providers have stopped charging for certain types of data transfer. It's difficult to predict how these changes might evolve, but if providers relax data transfer costs further, future cross-AZ traffic may become less of a factor in cloud cost efficiency," the report adds.

Fewer Organizations Taking Advantage of Discounts

Cloud service providers offer commitment-based discounts on many of their services — for example, AWS has discount programs for Amazon EC2, Amazon RDS, Amazon SageMaker and others — but only 67% of organizations are participating in these discounts, down from 72% last year.

Green Technology on the Rise

On average, organizations that use Arm-based instances spend 18% of their EC2 compute budget on them — twice as much as they did a year ago. Instance types based on the Arm processor use up to 60% less energy than similar EC2s and often provide better performance at a lower cost.

Hot Topics

The Latest

Automation often starts with the people closest to an immediate need. A developer may schedule a script with cron or Amazon EventBridge Scheduler; a business team may adopt an RPA bot, low-code platform, or agent. Many tools stay useful within one domain. Some become consequential across organizational boundaries because other systems and teams depend on them or business outcomes rely on their timing and recovery. Those cases may call for broader coordination ...

The conventional wisdom in our industry says that AI demands a new operating model and specific data tailored for it. I disagree. AI is changing the rules, but not in the way people think ...

Most organizations invest heavily in security tools, policies, and controls. Yet despite these efforts, one reality persists: employees often find ways around them. This isn't always malicious. In many cases, it’s a response to friction, urgency, or misalignment between how work actually happens and how systems are designed to protect it. When employees bypass security, it creates blind spots. Sensitive data may move through unmonitored channels, access controls may be weakened, and risks that security teams intended to mitigate reappear in new forms ...

Enterprise AI projects are often justified with a familiar metric: hours saved. A team automates a manual process, employees spend fewer hours reviewing documents or entering data, and the productivity gain becomes the headline. Time savings matter, but they capture only part of the financial impact AI can have inside an organization ... A more useful way to evaluate enterprise AI is to examine the financial decisions and risks surrounding the workflows it supports ...

We run an internal AI assistant that answers engineering questions in Slack ... Each question kicks off what we call a turn ... We could already audit a single turn after the fact. Every reply carries a short lookup token in its footer, and a slash command pulls up the full receipt for that turn. What we could not do was answer questions about the whole fleet. How long do turns take? What do they cost? Which tools does the agent reach for? Where does the time go inside a single answer? Those are APM questions. So we treated the agent like any other service and instrumented it with OpenTelemetry, end to end. This post covers what we built, the design choices that turned out to matter, and the things that surprised us ...

Applications have long used bounded, deterministic logic to control what happens within their own functions. Process orchestration coordinates activity across people, applications, decisions, and automation tools. Workload orchestration coordinates executable work and dependencies across applications, platforms, and infrastructure. As agentic capabilities enter applications and domain systems, more planning and decision-making will also occur inside those environments. These forms of automation will coexist. Sometimes they will overlap, and they will increasingly need to exchange triggers, state, and results. But they are not becoming interchangeable ...

For many organizations, the AI agenda has landed squarely on the CIO's desk. At first glance, that makes perfect sense. AI depends on data, infrastructure, security, integration, governance and scalable technology platforms — all areas where the CIO plays a central role. But there is a problem with treating AI primarily as an IT initiative: AI is not simply a technology deployment. It’s also a business transformation ...

In MEAN TIME TO INSIGHT Episode 28, Shamus McGillicuddy, EMA VP of Research, Network Infrastructure and Operations, and special guest Sanjay Gidwani, Founder & CEO of Kosmos AI Labs, discuss root cause analysis ... 

A gap is growing between how quickly organizations are putting AI to work and how much autonomy they are prepared to give it, according to The 2026 State of Observability Report from Grafana Labs ...

AI agents and agentic browsers now make up a growing share of the traffic reaching websites and applications. They search, compare, sign up, and increasingly complete purchases on behalf of the people who delegate those tasks. Digital experience monitoring (DEM) was built on the assumption that every session is performed by a person. That assumption no longer holds, and the measurements built on it are shifting without notice ...

The State of Cloud Costs 2024

Containers are a common theme of wasted spend among organizations, according to the State of Cloud Costs 2024 report from Datadog.

In fact, 83% of container costs were associated with idle resources. About 54% of this wasted spend was on cluster idle, which is the cost of overprovisioning cluster infrastructure, while 29% was associated with workload idle, which comes from resource requests that are larger than their workloads require. This wasted spend comes as organizations allocate more of their EC2 compute to running containers, up to 35% compared to 30% a year ago.

Other report findings include:

GPU Spend Increasing

The report found organizations that use graphics processing unit (GPU) instances have increased their average spending on those instances by 40% in the last year. This growth in spend on GPU instances comes as more companies are experimenting with AI and large language models (LLMs). GPUs' capacity for parallel processing makes them critical for training LLMs and executing other AI workloads, where they can be more than 200% faster than CPUs.

"Today, the most widely used type of GPU-based instance is also the least expensive. This suggests that many customers are still in the experimentation phase with AI and applying the GPU instance to their early efforts in adaptive AI, machine learning inference and small-scale training," said Yrieix Garnier, VP of Product at Datadog. "We expect that as organizations expand their AI activities and move them into production, they will be spending a larger proportion of their cloud compute budget as they use more expensive types of GPU-based instances."

Outdated Technologies Are Widely Used

AWS's current infrastructure offerings commonly both outperform their previous-generation versions and cost less, but 83% of organizations still spend an average of 17% of their EC2 budgets on previous-generation technologies.

Cross-AZ traffic makes up half of data transfer costs

The report states that, "On average, organizations spend almost as much on sending data from one availability zone (AZ) to another as they do on all other types of data transfer combined — including VPNs, gateways, ingress, and egress."

The report found that 98% of organizations are affected by cross-AZ charges, representing an opportunity to optimize cloud costs, such as by colocating related resources within a single AZ whenever availability requirements allow.

"In some cases, cloud providers have stopped charging for certain types of data transfer. It's difficult to predict how these changes might evolve, but if providers relax data transfer costs further, future cross-AZ traffic may become less of a factor in cloud cost efficiency," the report adds.

Fewer Organizations Taking Advantage of Discounts

Cloud service providers offer commitment-based discounts on many of their services — for example, AWS has discount programs for Amazon EC2, Amazon RDS, Amazon SageMaker and others — but only 67% of organizations are participating in these discounts, down from 72% last year.

Green Technology on the Rise

On average, organizations that use Arm-based instances spend 18% of their EC2 compute budget on them — twice as much as they did a year ago. Instance types based on the Arm processor use up to 60% less energy than similar EC2s and often provide better performance at a lower cost.

Hot Topics

The Latest

Automation often starts with the people closest to an immediate need. A developer may schedule a script with cron or Amazon EventBridge Scheduler; a business team may adopt an RPA bot, low-code platform, or agent. Many tools stay useful within one domain. Some become consequential across organizational boundaries because other systems and teams depend on them or business outcomes rely on their timing and recovery. Those cases may call for broader coordination ...

The conventional wisdom in our industry says that AI demands a new operating model and specific data tailored for it. I disagree. AI is changing the rules, but not in the way people think ...

Most organizations invest heavily in security tools, policies, and controls. Yet despite these efforts, one reality persists: employees often find ways around them. This isn't always malicious. In many cases, it’s a response to friction, urgency, or misalignment between how work actually happens and how systems are designed to protect it. When employees bypass security, it creates blind spots. Sensitive data may move through unmonitored channels, access controls may be weakened, and risks that security teams intended to mitigate reappear in new forms ...

Enterprise AI projects are often justified with a familiar metric: hours saved. A team automates a manual process, employees spend fewer hours reviewing documents or entering data, and the productivity gain becomes the headline. Time savings matter, but they capture only part of the financial impact AI can have inside an organization ... A more useful way to evaluate enterprise AI is to examine the financial decisions and risks surrounding the workflows it supports ...

We run an internal AI assistant that answers engineering questions in Slack ... Each question kicks off what we call a turn ... We could already audit a single turn after the fact. Every reply carries a short lookup token in its footer, and a slash command pulls up the full receipt for that turn. What we could not do was answer questions about the whole fleet. How long do turns take? What do they cost? Which tools does the agent reach for? Where does the time go inside a single answer? Those are APM questions. So we treated the agent like any other service and instrumented it with OpenTelemetry, end to end. This post covers what we built, the design choices that turned out to matter, and the things that surprised us ...

Applications have long used bounded, deterministic logic to control what happens within their own functions. Process orchestration coordinates activity across people, applications, decisions, and automation tools. Workload orchestration coordinates executable work and dependencies across applications, platforms, and infrastructure. As agentic capabilities enter applications and domain systems, more planning and decision-making will also occur inside those environments. These forms of automation will coexist. Sometimes they will overlap, and they will increasingly need to exchange triggers, state, and results. But they are not becoming interchangeable ...

For many organizations, the AI agenda has landed squarely on the CIO's desk. At first glance, that makes perfect sense. AI depends on data, infrastructure, security, integration, governance and scalable technology platforms — all areas where the CIO plays a central role. But there is a problem with treating AI primarily as an IT initiative: AI is not simply a technology deployment. It’s also a business transformation ...

In MEAN TIME TO INSIGHT Episode 28, Shamus McGillicuddy, EMA VP of Research, Network Infrastructure and Operations, and special guest Sanjay Gidwani, Founder & CEO of Kosmos AI Labs, discuss root cause analysis ... 

A gap is growing between how quickly organizations are putting AI to work and how much autonomy they are prepared to give it, according to The 2026 State of Observability Report from Grafana Labs ...

AI agents and agentic browsers now make up a growing share of the traffic reaching websites and applications. They search, compare, sign up, and increasingly complete purchases on behalf of the people who delegate those tasks. Digital experience monitoring (DEM) was built on the assumption that every session is performed by a person. That assumption no longer holds, and the measurements built on it are shifting without notice ...