Skip to main content

The State of Cloud Costs 2024

Containers are a common theme of wasted spend among organizations, according to the State of Cloud Costs 2024 report from Datadog.

In fact, 83% of container costs were associated with idle resources. About 54% of this wasted spend was on cluster idle, which is the cost of overprovisioning cluster infrastructure, while 29% was associated with workload idle, which comes from resource requests that are larger than their workloads require. This wasted spend comes as organizations allocate more of their EC2 compute to running containers, up to 35% compared to 30% a year ago.

Other report findings include:

GPU Spend Increasing

The report found organizations that use graphics processing unit (GPU) instances have increased their average spending on those instances by 40% in the last year. This growth in spend on GPU instances comes as more companies are experimenting with AI and large language models (LLMs). GPUs' capacity for parallel processing makes them critical for training LLMs and executing other AI workloads, where they can be more than 200% faster than CPUs.

"Today, the most widely used type of GPU-based instance is also the least expensive. This suggests that many customers are still in the experimentation phase with AI and applying the GPU instance to their early efforts in adaptive AI, machine learning inference and small-scale training," said Yrieix Garnier, VP of Product at Datadog. "We expect that as organizations expand their AI activities and move them into production, they will be spending a larger proportion of their cloud compute budget as they use more expensive types of GPU-based instances."

Outdated Technologies Are Widely Used

AWS's current infrastructure offerings commonly both outperform their previous-generation versions and cost less, but 83% of organizations still spend an average of 17% of their EC2 budgets on previous-generation technologies.

Cross-AZ traffic makes up half of data transfer costs

The report states that, "On average, organizations spend almost as much on sending data from one availability zone (AZ) to another as they do on all other types of data transfer combined — including VPNs, gateways, ingress, and egress."

The report found that 98% of organizations are affected by cross-AZ charges, representing an opportunity to optimize cloud costs, such as by colocating related resources within a single AZ whenever availability requirements allow.

"In some cases, cloud providers have stopped charging for certain types of data transfer. It's difficult to predict how these changes might evolve, but if providers relax data transfer costs further, future cross-AZ traffic may become less of a factor in cloud cost efficiency," the report adds.

Fewer Organizations Taking Advantage of Discounts

Cloud service providers offer commitment-based discounts on many of their services — for example, AWS has discount programs for Amazon EC2, Amazon RDS, Amazon SageMaker and others — but only 67% of organizations are participating in these discounts, down from 72% last year.

Green Technology on the Rise

On average, organizations that use Arm-based instances spend 18% of their EC2 compute budget on them — twice as much as they did a year ago. Instance types based on the Arm processor use up to 60% less energy than similar EC2s and often provide better performance at a lower cost.

Hot Topics

The Latest

As businesses increasingly rely on high-performance applications to deliver seamless user experiences, the demand for fast, reliable, and scalable data storage systems has never been greater. Redis — an open-source, in-memory data structure store — has emerged as a popular choice for use cases ranging from caching to real-time analytics. But with great performance comes the need for vigilant monitoring ...

Kubernetes was not initially designed with AI's vast resource variability in mind, and the rapid rise of AI has exposed Kubernetes limitations, particularly when it comes to cost and resource efficiency. Indeed, AI workloads differ from traditional applications in that they require a staggering amount and variety of compute resources, and their consumption is far less consistent than traditional workloads ... Considering the speed of AI innovation, teams cannot afford to be bogged down by these constant infrastructure concerns. A solution is needed ...

AI is the catalyst for significant investment in data teams as enterprises require higher-quality data to power their AI applications, according to the State of Analytics Engineering Report from dbt Labs ...

Misaligned architecture can lead to business consequences, with 93% of respondents reporting negative outcomes such as service disruptions, high operational costs and security challenges ...

A Gartner analyst recently suggested that GenAI tools could create 25% time savings for network operational teams. Where might these time savings come from? How are GenAI tools helping NetOps teams today, and what other tasks might they take on in the future as models continue improving? In general, these savings come from automating or streamlining manual NetOps tasks ...

IT and line-of-business teams are increasingly aligned in their efforts to close the data gap and drive greater collaboration to alleviate IT bottlenecks and offload growing demands on IT teams, according to The 2025 Automation Benchmark Report: Insights from IT Leaders on Enterprise Automation & the Future of AI-Driven Businesses from Jitterbit ...

A large majority (86%) of data management and AI decision makers cite protecting data privacy as a top concern, with 76% of respondents citing ROI on data privacy and AI initiatives across their organization, according to a new Harris Poll from Collibra ...

According to Gartner, Inc. the following six trends will shape the future of cloud over the next four years, ultimately resulting in new ways of working that are digital in nature and transformative in impact ...

2020 was the equivalent of a wedding with a top-shelf open bar. As businesses scrambled to adjust to remote work, digital transformation accelerated at breakneck speed. New software categories emerged overnight. Tech stacks ballooned with all sorts of SaaS apps solving ALL the problems — often with little oversight or long-term integration planning, and yes frequently a lot of duplicated functionality ... But now the music's faded. The lights are on. Everyone from the CIO to the CFO is checking the bill. Welcome to the Great SaaS Hangover ...

Regardless of OpenShift being a scalable and flexible software, it can be a pain to monitor since complete visibility into the underlying operations is not guaranteed ... To effectively monitor an OpenShift environment, IT administrators should focus on these five key elements and their associated metrics ...

The State of Cloud Costs 2024

Containers are a common theme of wasted spend among organizations, according to the State of Cloud Costs 2024 report from Datadog.

In fact, 83% of container costs were associated with idle resources. About 54% of this wasted spend was on cluster idle, which is the cost of overprovisioning cluster infrastructure, while 29% was associated with workload idle, which comes from resource requests that are larger than their workloads require. This wasted spend comes as organizations allocate more of their EC2 compute to running containers, up to 35% compared to 30% a year ago.

Other report findings include:

GPU Spend Increasing

The report found organizations that use graphics processing unit (GPU) instances have increased their average spending on those instances by 40% in the last year. This growth in spend on GPU instances comes as more companies are experimenting with AI and large language models (LLMs). GPUs' capacity for parallel processing makes them critical for training LLMs and executing other AI workloads, where they can be more than 200% faster than CPUs.

"Today, the most widely used type of GPU-based instance is also the least expensive. This suggests that many customers are still in the experimentation phase with AI and applying the GPU instance to their early efforts in adaptive AI, machine learning inference and small-scale training," said Yrieix Garnier, VP of Product at Datadog. "We expect that as organizations expand their AI activities and move them into production, they will be spending a larger proportion of their cloud compute budget as they use more expensive types of GPU-based instances."

Outdated Technologies Are Widely Used

AWS's current infrastructure offerings commonly both outperform their previous-generation versions and cost less, but 83% of organizations still spend an average of 17% of their EC2 budgets on previous-generation technologies.

Cross-AZ traffic makes up half of data transfer costs

The report states that, "On average, organizations spend almost as much on sending data from one availability zone (AZ) to another as they do on all other types of data transfer combined — including VPNs, gateways, ingress, and egress."

The report found that 98% of organizations are affected by cross-AZ charges, representing an opportunity to optimize cloud costs, such as by colocating related resources within a single AZ whenever availability requirements allow.

"In some cases, cloud providers have stopped charging for certain types of data transfer. It's difficult to predict how these changes might evolve, but if providers relax data transfer costs further, future cross-AZ traffic may become less of a factor in cloud cost efficiency," the report adds.

Fewer Organizations Taking Advantage of Discounts

Cloud service providers offer commitment-based discounts on many of their services — for example, AWS has discount programs for Amazon EC2, Amazon RDS, Amazon SageMaker and others — but only 67% of organizations are participating in these discounts, down from 72% last year.

Green Technology on the Rise

On average, organizations that use Arm-based instances spend 18% of their EC2 compute budget on them — twice as much as they did a year ago. Instance types based on the Arm processor use up to 60% less energy than similar EC2s and often provide better performance at a lower cost.

Hot Topics

The Latest

As businesses increasingly rely on high-performance applications to deliver seamless user experiences, the demand for fast, reliable, and scalable data storage systems has never been greater. Redis — an open-source, in-memory data structure store — has emerged as a popular choice for use cases ranging from caching to real-time analytics. But with great performance comes the need for vigilant monitoring ...

Kubernetes was not initially designed with AI's vast resource variability in mind, and the rapid rise of AI has exposed Kubernetes limitations, particularly when it comes to cost and resource efficiency. Indeed, AI workloads differ from traditional applications in that they require a staggering amount and variety of compute resources, and their consumption is far less consistent than traditional workloads ... Considering the speed of AI innovation, teams cannot afford to be bogged down by these constant infrastructure concerns. A solution is needed ...

AI is the catalyst for significant investment in data teams as enterprises require higher-quality data to power their AI applications, according to the State of Analytics Engineering Report from dbt Labs ...

Misaligned architecture can lead to business consequences, with 93% of respondents reporting negative outcomes such as service disruptions, high operational costs and security challenges ...

A Gartner analyst recently suggested that GenAI tools could create 25% time savings for network operational teams. Where might these time savings come from? How are GenAI tools helping NetOps teams today, and what other tasks might they take on in the future as models continue improving? In general, these savings come from automating or streamlining manual NetOps tasks ...

IT and line-of-business teams are increasingly aligned in their efforts to close the data gap and drive greater collaboration to alleviate IT bottlenecks and offload growing demands on IT teams, according to The 2025 Automation Benchmark Report: Insights from IT Leaders on Enterprise Automation & the Future of AI-Driven Businesses from Jitterbit ...

A large majority (86%) of data management and AI decision makers cite protecting data privacy as a top concern, with 76% of respondents citing ROI on data privacy and AI initiatives across their organization, according to a new Harris Poll from Collibra ...

According to Gartner, Inc. the following six trends will shape the future of cloud over the next four years, ultimately resulting in new ways of working that are digital in nature and transformative in impact ...

2020 was the equivalent of a wedding with a top-shelf open bar. As businesses scrambled to adjust to remote work, digital transformation accelerated at breakneck speed. New software categories emerged overnight. Tech stacks ballooned with all sorts of SaaS apps solving ALL the problems — often with little oversight or long-term integration planning, and yes frequently a lot of duplicated functionality ... But now the music's faded. The lights are on. Everyone from the CIO to the CFO is checking the bill. Welcome to the Great SaaS Hangover ...

Regardless of OpenShift being a scalable and flexible software, it can be a pain to monitor since complete visibility into the underlying operations is not guaranteed ... To effectively monitor an OpenShift environment, IT administrators should focus on these five key elements and their associated metrics ...