Skip to main content

The State of Cloud Costs 2024

Containers are a common theme of wasted spend among organizations, according to the State of Cloud Costs 2024 report from Datadog.

In fact, 83% of container costs were associated with idle resources. About 54% of this wasted spend was on cluster idle, which is the cost of overprovisioning cluster infrastructure, while 29% was associated with workload idle, which comes from resource requests that are larger than their workloads require. This wasted spend comes as organizations allocate more of their EC2 compute to running containers, up to 35% compared to 30% a year ago.

Other report findings include:

GPU Spend Increasing

The report found organizations that use graphics processing unit (GPU) instances have increased their average spending on those instances by 40% in the last year. This growth in spend on GPU instances comes as more companies are experimenting with AI and large language models (LLMs). GPUs' capacity for parallel processing makes them critical for training LLMs and executing other AI workloads, where they can be more than 200% faster than CPUs.

"Today, the most widely used type of GPU-based instance is also the least expensive. This suggests that many customers are still in the experimentation phase with AI and applying the GPU instance to their early efforts in adaptive AI, machine learning inference and small-scale training," said Yrieix Garnier, VP of Product at Datadog. "We expect that as organizations expand their AI activities and move them into production, they will be spending a larger proportion of their cloud compute budget as they use more expensive types of GPU-based instances."

Outdated Technologies Are Widely Used

AWS's current infrastructure offerings commonly both outperform their previous-generation versions and cost less, but 83% of organizations still spend an average of 17% of their EC2 budgets on previous-generation technologies.

Cross-AZ traffic makes up half of data transfer costs

The report states that, "On average, organizations spend almost as much on sending data from one availability zone (AZ) to another as they do on all other types of data transfer combined — including VPNs, gateways, ingress, and egress."

The report found that 98% of organizations are affected by cross-AZ charges, representing an opportunity to optimize cloud costs, such as by colocating related resources within a single AZ whenever availability requirements allow.

"In some cases, cloud providers have stopped charging for certain types of data transfer. It's difficult to predict how these changes might evolve, but if providers relax data transfer costs further, future cross-AZ traffic may become less of a factor in cloud cost efficiency," the report adds.

Fewer Organizations Taking Advantage of Discounts

Cloud service providers offer commitment-based discounts on many of their services — for example, AWS has discount programs for Amazon EC2, Amazon RDS, Amazon SageMaker and others — but only 67% of organizations are participating in these discounts, down from 72% last year.

Green Technology on the Rise

On average, organizations that use Arm-based instances spend 18% of their EC2 compute budget on them — twice as much as they did a year ago. Instance types based on the Arm processor use up to 60% less energy than similar EC2s and often provide better performance at a lower cost.

Hot Topics

The Latest

According to Auvik's 2025 IT Trends Report, 60% of IT professionals feel at least moderately burned out on the job, with 43% stating that their workload is contributing to work stress. At the same time, many IT professionals are naming AI and machine learning as key areas they'd most like to upskill ...

Businesses that face downtime or outages risk financial and reputational damage, as well as reducing partner, shareholder, and customer trust. One of the major challenges that enterprises face is implementing a robust business continuity plan. What's the solution? The answer may lie in disaster recovery tactics such as truly immutable storage and regular disaster recovery testing ...

IT spending is expected to jump nearly 10% in 2025, and organizations are now facing pressure to manage costs without slowing down critical functions like observability. To meet the challenge, leaders are turning to smarter, more cost effective business strategies. Enter stage right: OpenTelemetry, the missing piece of the puzzle that is no longer just an option but rather a strategic advantage ...

Amidst the threat of cyberhacks and data breaches, companies install several security measures to keep their business safely afloat. These measures aim to protect businesses, employees, and crucial data. Yet, employees perceive them as burdensome. Frustrated with complex logins, slow access, and constant security checks, workers decide to completely bypass all security set-ups ...

Image
Cloudbrink's Personal SASE services provide last-mile acceleration and reduction in latency

In MEAN TIME TO INSIGHT Episode 13, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses hybrid multi-cloud networking strategy ... 

In high-traffic environments, the sheer volume and unpredictable nature of network incidents can quickly overwhelm even the most skilled teams, hindering their ability to react swiftly and effectively, potentially impacting service availability and overall business performance. This is where closed-loop remediation comes into the picture: an IT management concept designed to address the escalating complexity of modern networks ...

In 2025, enterprise workflows are undergoing a seismic shift. Propelled by breakthroughs in generative AI (GenAI), large language models (LLMs), and natural language processing (NLP), a new paradigm is emerging — agentic AI. This technology is not just automating tasks; it's reimagining how organizations make decisions, engage customers, and operate at scale ...

In the early days of the cloud revolution, business leaders perceived cloud services as a means of sidelining IT organizations. IT was too slow, too expensive, or incapable of supporting new technologies. With a team of developers, line of business managers could deploy new applications and services in the cloud. IT has been fighting to retake control ever since. Today, IT is back in the driver's seat, according to new research by Enterprise Management Associates (EMA) ...

In today's fast-paced and increasingly complex network environments, Network Operations Centers (NOCs) are the backbone of ensuring continuous uptime, smooth service delivery, and rapid issue resolution. However, the challenges faced by NOC teams are only growing. In a recent study, 78% state network complexity has grown significantly over the last few years while 84% regularly learn about network issues from users. It is imperative we adopt a new approach to managing today's network experiences ...

Image
Broadcom

From growing reliance on FinOps teams to the increasing attention on artificial intelligence (AI), and software licensing, the Flexera 2025 State of the Cloud Report digs into how organizations are improving cloud spend efficiency, while tackling the complexities of emerging technologies ...

The State of Cloud Costs 2024

Containers are a common theme of wasted spend among organizations, according to the State of Cloud Costs 2024 report from Datadog.

In fact, 83% of container costs were associated with idle resources. About 54% of this wasted spend was on cluster idle, which is the cost of overprovisioning cluster infrastructure, while 29% was associated with workload idle, which comes from resource requests that are larger than their workloads require. This wasted spend comes as organizations allocate more of their EC2 compute to running containers, up to 35% compared to 30% a year ago.

Other report findings include:

GPU Spend Increasing

The report found organizations that use graphics processing unit (GPU) instances have increased their average spending on those instances by 40% in the last year. This growth in spend on GPU instances comes as more companies are experimenting with AI and large language models (LLMs). GPUs' capacity for parallel processing makes them critical for training LLMs and executing other AI workloads, where they can be more than 200% faster than CPUs.

"Today, the most widely used type of GPU-based instance is also the least expensive. This suggests that many customers are still in the experimentation phase with AI and applying the GPU instance to their early efforts in adaptive AI, machine learning inference and small-scale training," said Yrieix Garnier, VP of Product at Datadog. "We expect that as organizations expand their AI activities and move them into production, they will be spending a larger proportion of their cloud compute budget as they use more expensive types of GPU-based instances."

Outdated Technologies Are Widely Used

AWS's current infrastructure offerings commonly both outperform their previous-generation versions and cost less, but 83% of organizations still spend an average of 17% of their EC2 budgets on previous-generation technologies.

Cross-AZ traffic makes up half of data transfer costs

The report states that, "On average, organizations spend almost as much on sending data from one availability zone (AZ) to another as they do on all other types of data transfer combined — including VPNs, gateways, ingress, and egress."

The report found that 98% of organizations are affected by cross-AZ charges, representing an opportunity to optimize cloud costs, such as by colocating related resources within a single AZ whenever availability requirements allow.

"In some cases, cloud providers have stopped charging for certain types of data transfer. It's difficult to predict how these changes might evolve, but if providers relax data transfer costs further, future cross-AZ traffic may become less of a factor in cloud cost efficiency," the report adds.

Fewer Organizations Taking Advantage of Discounts

Cloud service providers offer commitment-based discounts on many of their services — for example, AWS has discount programs for Amazon EC2, Amazon RDS, Amazon SageMaker and others — but only 67% of organizations are participating in these discounts, down from 72% last year.

Green Technology on the Rise

On average, organizations that use Arm-based instances spend 18% of their EC2 compute budget on them — twice as much as they did a year ago. Instance types based on the Arm processor use up to 60% less energy than similar EC2s and often provide better performance at a lower cost.

Hot Topics

The Latest

According to Auvik's 2025 IT Trends Report, 60% of IT professionals feel at least moderately burned out on the job, with 43% stating that their workload is contributing to work stress. At the same time, many IT professionals are naming AI and machine learning as key areas they'd most like to upskill ...

Businesses that face downtime or outages risk financial and reputational damage, as well as reducing partner, shareholder, and customer trust. One of the major challenges that enterprises face is implementing a robust business continuity plan. What's the solution? The answer may lie in disaster recovery tactics such as truly immutable storage and regular disaster recovery testing ...

IT spending is expected to jump nearly 10% in 2025, and organizations are now facing pressure to manage costs without slowing down critical functions like observability. To meet the challenge, leaders are turning to smarter, more cost effective business strategies. Enter stage right: OpenTelemetry, the missing piece of the puzzle that is no longer just an option but rather a strategic advantage ...

Amidst the threat of cyberhacks and data breaches, companies install several security measures to keep their business safely afloat. These measures aim to protect businesses, employees, and crucial data. Yet, employees perceive them as burdensome. Frustrated with complex logins, slow access, and constant security checks, workers decide to completely bypass all security set-ups ...

Image
Cloudbrink's Personal SASE services provide last-mile acceleration and reduction in latency

In MEAN TIME TO INSIGHT Episode 13, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses hybrid multi-cloud networking strategy ... 

In high-traffic environments, the sheer volume and unpredictable nature of network incidents can quickly overwhelm even the most skilled teams, hindering their ability to react swiftly and effectively, potentially impacting service availability and overall business performance. This is where closed-loop remediation comes into the picture: an IT management concept designed to address the escalating complexity of modern networks ...

In 2025, enterprise workflows are undergoing a seismic shift. Propelled by breakthroughs in generative AI (GenAI), large language models (LLMs), and natural language processing (NLP), a new paradigm is emerging — agentic AI. This technology is not just automating tasks; it's reimagining how organizations make decisions, engage customers, and operate at scale ...

In the early days of the cloud revolution, business leaders perceived cloud services as a means of sidelining IT organizations. IT was too slow, too expensive, or incapable of supporting new technologies. With a team of developers, line of business managers could deploy new applications and services in the cloud. IT has been fighting to retake control ever since. Today, IT is back in the driver's seat, according to new research by Enterprise Management Associates (EMA) ...

In today's fast-paced and increasingly complex network environments, Network Operations Centers (NOCs) are the backbone of ensuring continuous uptime, smooth service delivery, and rapid issue resolution. However, the challenges faced by NOC teams are only growing. In a recent study, 78% state network complexity has grown significantly over the last few years while 84% regularly learn about network issues from users. It is imperative we adopt a new approach to managing today's network experiences ...

Image
Broadcom

From growing reliance on FinOps teams to the increasing attention on artificial intelligence (AI), and software licensing, the Flexera 2025 State of the Cloud Report digs into how organizations are improving cloud spend efficiency, while tackling the complexities of emerging technologies ...