2024 DataOps Predictions - Part 1

Industry experts offer predictions on how DataOps will evolve and impact IT and business in 2024.

Data observability becomes mandatory

Data observability will become mandatory as organizations seek to drive smarter automation and faster decision-making in 2024. As the volume of data has continued to double every two years, organizations are urgently seeking to ingest and analyze it faster and at a greater scale. However, the cost and risk of poor-quality data is greater than ever. In a recent survey, 57% of DevOps practitioners said the absence of data observability makes it difficult to drive automation in a compliant way. As a result, there will be an increased demand for solutions that provide data observability to enable organizations to rapidly and securely ingest high-quality and reliable data that is ready for analytics on demand. Increased data observability will enable users to understand not only the availability of data, but also the structure, distribution, relationships, and lineage of that data across all sources. This is essential to generating insights that users can trust, by ensuring its freshness, identifying anomalies, and eliminating duplicates that could lead to errors.
Bernd Greifeneder
CTO and Founder, Dynatrace

AI DRIVES Data Observability

In 2024 large enterprises will rapidly increase investment in AI and LLM technologies. This will create a greater need for data observability to validate that data feeding AI initiatives is accurate and complete. As a result, data observability vendors will be expected to expand support from predominantly cloud-native data environments to larger, more traditional enterprise data stacks and provide native solutions for LLM data pipeline monitoring and validation.
Kyle Kirwan
Co-Founder and CEO, Bigeye

AI DRIVES DATAOPS

2024 will see the impact of rapid AI/ML advances on DataOps — empowering developers and DevOps teams by making data analytics that much more accessible, accurate, and definitive. AI/ML and new data science techniques will enable new operating models, data analytics tools, and technologies that enable data-driven decisions. Specific benefits will manifest in everything from customer recommendation engines to insight-optimized business operations, including streamlined development practices and DevOps processes. In this way, DataOps will provide enterprises with decisive advantages over competitors who lack that data command and clarity.
Anil Inamdar
VP & Head of Data, Instaclustr, part of Spot by NetApp

AI DRIVES DATA INFRASTRUCTURE MODERNIZATION

The continuous and rapid adoption of AI will force organizations to modernize their data infrastructure in 2024. Enterprises are examining their data and it's pushing them to have a better handle on it so technologies like AI can be properly used. Organizations will double down on data management and data integrity to ensure third-party applications are seamlessly integrated. Data practitioners will look for solutions that continuously keep data clean to quickly act on workflows. Better data means better-trained models on less data, as well as a better ability to leverage that data in AI applications that incorporate retrieval.
Matt Wallace
Technical Advisor, Faction

AI WILL NOT REPLACE DATA ENGINEERS

Data engineering will evolve — and be highly valued — in an AI world. There's been a lot of chatter that the AI revolution will replace the role of data engineers. That's not the case, and in fact their data expertise will be more critical than ever — just in new and different ways. To keep up with the evolving landscape, data engineers will need to understand how generative AI adds value. The data pipelines built and managed by data engineers will be perhaps the first place to connect with large language models for organizations to unlock value. Data engineers will be the ones who understand how to consume a model and plug it into a data pipeline to automate the extraction of value. They will also be expected to oversee and understand the AI work.
Jeff Hollan
Director of Product Management, Snowflake

BREAKING DOWN DATA SILOS

The traditional silos between IT, legal, and business departments will crumble in 2024, giving way to a more collaborative approach. This cross-functional synergy will ensure that data, including unstructured data, is not just collected but strategically utilized to drive business value.
Rohit Choudhary
CEO, Acceldata

Data as Business Asset

In many senses, data is the new oil. It's a finite resource that needs to be mined and managed strategically, and its value is highly dependent on your ability to refine and manipulate it for specific applications. For this reason, we see 2024 as being a critical year in the transition of data from being 1s and 0s on a screen to an actual asset to be managed, tracked, and optimized within an enterprise.
Jackie McGuire
Senior Security Strategist, Cribl

Data management will evolve beyond mere data storage in 2024. Organizations will recognize the strategic value of weaving data, including unstructured data, into their business strategies. This shift will unlock a wealth of insights and redefine business decision-making.
Rohit Choudhary
CEO, Acceldata

In 2024, the technology landscape will witness a transformative shift as data evolves from being a valuable asset to the lifeblood of thriving enterprises. Organizations that overlook data quality, integrity, and lineage will be challenged to not only make informed decisions but also realize the full potential of generative AI, LLM and ML applications and use cases. As the year unfolds, I predict that organizations neglecting to craft robust data foundations and strategies will find it increasingly challenging to stay afloat in the swiftly evolving tech industry. Those who fail to adapt and prioritize data fundamentals will struggle to outpace their competitors and may even risk survival in this highly competitive environment.
Armon Petrossian
CEO and Co-Founder, Coalesce

Data as innovation asset

The rapid expansion of data will continue to be a dominant trend in 2024. However, the ability to efficiently gather, process, and utilize this data will become the critical factor limiting or accelerating innovation within organizations. The challenge will lie in developing methods to quickly and securely assimilate this growing data influx, converting it into actionable insights. Companies that can effectively manage this data deluge, turning it into a strategic asset for innovation, will gain a competitive edge in the increasingly data-driven business landscape.
David Boskovic
Founder and CEO, Flatfile

Data as a product

Until recently, only large companies had the expertise and resources needed to create reusable data assets that can be easily repurposed across different teams and applications. Thanks to advancements in the governance products required to build these assets, in 2024 more companies will be able to create reusable data products, greatly accelerating efficiency and data innovation. Multiple teams can benefit from having access to the same data to build a service or application. However, this data must be presented in a way that is secure, well-contextualized, and understandable for users who weren't involved in its production. As data moves farther away from its initial source, you have to do more checks, which becomes increasingly expensive. Starting the data governance process at the source is not only less expensive but is a better way to understand the data's source and how it's schematized. New data governance capabilities, pre-built into products such as cloud data warehouses, databases, and other data infrastructure services, can meet these needs. Developers no longer need to manually build the infrastructure to create and share reusable data products. As a result, reusable data products will no longer be restricted to companies with large, sophisticated data engineering teams. With more companies building reusable data products, in 2024, developers will increase the value of their data and spend more time building innovative data applications and services.
Andrew Sellers
Head of Technology Strategy, Confluent

Go to: 2024 DataOps Predictions - Part 2

Related Links

Hot Topics

The Latest

How AI Agents Are Reshaping DataOps for the Always-On Enterprise

Enterprises today operate in a real-time environment where uninterrupted access to trusted data has become a baseline expectation for users, applications and automated systems. Traditional DataOps models, built on manual effort and human triage, cannot keep pace with this always active demand. AI agents are emerging as the operational backbone, ensuring consistent data availability, reinforcing trustworthiness and enabling a level of scale that manual processes cannot achieve ...

AI Deepfakes: Rethinking Trust in the Workplace

For decades, trust in the digital workplace rested on familiar signals. We trusted faces on video calls, voices on the phone, and emails that appeared to come from people we knew. These cues felt human and intuitive. They anchored how decisions were made, approvals were granted, and access was authorized. AI-powered deepfakes have quietly broken that model ...

Nine in Ten Enterprises Plan Cloud Data Repatriation amid Rising Cloud Costs and Data Sovereignty Mandates

Cloud migration was supposed to be a one-way door. For most enterprises, it turns out it isn't. Cloud data repatriation is a real and growing trend. A new survey ... finds that 89% of organizations plan to expand their on-premises infrastructure footprint over the next two years — and 75% have already moved at least some workloads back from public cloud in the past 24 months. The findings point to a broad rethinking of where data belongs ...

Why ITOps Need Right-Sized AI, Not Bigger Models

Over the past few years, large language models (LLMs) have revolutionized the software industry. Given their ability to excel at multi-step reasoning, LLMs have helped enterprises streamline workflows and adapt to the unknown. However, employing such models comes with sky-high costs, latency issues, and limited flexibility. In the realm of IT operations, it is generally wiser to employ smaller, domain-specific models instead ...

The End of Reactive DevOps: AI-Driven Observability for Zero-Defect Digital Systems

For years, DevOps teams operated under a simple assumption: collect enough telemetry, and you can find and fix any problem. That assumption is breaking down. Modern enterprises now operate across microservices, hybrid cloud environments, APIs, Kubernetes, and highly automated delivery pipelines. Releases happen continuously, dependencies shift constantly, and failures spread faster than teams can diagnose them ...

5 Takeaways from the Observability Forecast for Media and Entertainment

New Relic surveyed IT and engineering leaders from the media and entertainment (M&E) sector to understand what's working — and where challenges persist with their observability practices. The findings reveal how M&E organizations are navigating rising platform complexity, audience expectations, and AI-driven change. Below are five takeaways that stand out ...

Why Moving Off the Cloud Is the Easy Part and What Comes Next Is Where Things Get Hard

Let me start with something I've seen play out more times than I can count. A team hits a wall with the cloud. Costs creep up, then spike. Performance starts to feel inconsistent. Someone in finance asks a simple question like "why did this double?" and nobody has a clean answer ... Maybe this isn't the right place for everything. That realization feels like a breakthrough, like you've identified the problem. In reality, you've just identified the starting line ...

MEAN TIME TO INSIGHT Podcast - Episode 24: Network Observability Tool Sprawl

In MEAN TIME TO INSIGHT Episode 24, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses network observability tool sprawl ... 

Capacity Isn't a Guess: Observability-Driven Sizing for On-Prem Databases

In cloud-native systems, scaling is often as simple as moving a slider. For on-premise databases, the stakes are different. Over-provisioning hardware is expensive. Under-provisioning leads to performance bottlenecks that are difficult to fix once the equipment is in the rack ...

5 Security Principles Every Entrepreneur Should Apply to Leadership

When most people think about cybersecurity, they picture firewalls, encryption, and access controls — technical tools designed to protect systems and data. But beneath the technology lies a deeper set of principles about trust, decision-making, and resilience ... The best leaders don't eliminate risk. They manage it intelligently. And in many ways, cybersecurity offers a surprisingly useful playbook for doing exactly that ...

2024 DataOps Predictions - Part 1

Industry experts offer predictions on how DataOps will evolve and impact IT and business in 2024.

Data observability becomes mandatory

Data observability will become mandatory as organizations seek to drive smarter automation and faster decision-making in 2024. As the volume of data has continued to double every two years, organizations are urgently seeking to ingest and analyze it faster and at a greater scale. However, the cost and risk of poor-quality data is greater than ever. In a recent survey, 57% of DevOps practitioners said the absence of data observability makes it difficult to drive automation in a compliant way. As a result, there will be an increased demand for solutions that provide data observability to enable organizations to rapidly and securely ingest high-quality and reliable data that is ready for analytics on demand. Increased data observability will enable users to understand not only the availability of data, but also the structure, distribution, relationships, and lineage of that data across all sources. This is essential to generating insights that users can trust, by ensuring its freshness, identifying anomalies, and eliminating duplicates that could lead to errors.
Bernd Greifeneder
CTO and Founder, Dynatrace

AI DRIVES Data Observability

In 2024 large enterprises will rapidly increase investment in AI and LLM technologies. This will create a greater need for data observability to validate that data feeding AI initiatives is accurate and complete. As a result, data observability vendors will be expected to expand support from predominantly cloud-native data environments to larger, more traditional enterprise data stacks and provide native solutions for LLM data pipeline monitoring and validation.
Kyle Kirwan
Co-Founder and CEO, Bigeye

AI DRIVES DATAOPS

2024 will see the impact of rapid AI/ML advances on DataOps — empowering developers and DevOps teams by making data analytics that much more accessible, accurate, and definitive. AI/ML and new data science techniques will enable new operating models, data analytics tools, and technologies that enable data-driven decisions. Specific benefits will manifest in everything from customer recommendation engines to insight-optimized business operations, including streamlined development practices and DevOps processes. In this way, DataOps will provide enterprises with decisive advantages over competitors who lack that data command and clarity.
Anil Inamdar
VP & Head of Data, Instaclustr, part of Spot by NetApp

AI DRIVES DATA INFRASTRUCTURE MODERNIZATION

The continuous and rapid adoption of AI will force organizations to modernize their data infrastructure in 2024. Enterprises are examining their data and it's pushing them to have a better handle on it so technologies like AI can be properly used. Organizations will double down on data management and data integrity to ensure third-party applications are seamlessly integrated. Data practitioners will look for solutions that continuously keep data clean to quickly act on workflows. Better data means better-trained models on less data, as well as a better ability to leverage that data in AI applications that incorporate retrieval.
Matt Wallace
Technical Advisor, Faction

AI WILL NOT REPLACE DATA ENGINEERS

Data engineering will evolve — and be highly valued — in an AI world. There's been a lot of chatter that the AI revolution will replace the role of data engineers. That's not the case, and in fact their data expertise will be more critical than ever — just in new and different ways. To keep up with the evolving landscape, data engineers will need to understand how generative AI adds value. The data pipelines built and managed by data engineers will be perhaps the first place to connect with large language models for organizations to unlock value. Data engineers will be the ones who understand how to consume a model and plug it into a data pipeline to automate the extraction of value. They will also be expected to oversee and understand the AI work.
Jeff Hollan
Director of Product Management, Snowflake

BREAKING DOWN DATA SILOS

The traditional silos between IT, legal, and business departments will crumble in 2024, giving way to a more collaborative approach. This cross-functional synergy will ensure that data, including unstructured data, is not just collected but strategically utilized to drive business value.
Rohit Choudhary
CEO, Acceldata

Data as Business Asset

In many senses, data is the new oil. It's a finite resource that needs to be mined and managed strategically, and its value is highly dependent on your ability to refine and manipulate it for specific applications. For this reason, we see 2024 as being a critical year in the transition of data from being 1s and 0s on a screen to an actual asset to be managed, tracked, and optimized within an enterprise.
Jackie McGuire
Senior Security Strategist, Cribl

Data management will evolve beyond mere data storage in 2024. Organizations will recognize the strategic value of weaving data, including unstructured data, into their business strategies. This shift will unlock a wealth of insights and redefine business decision-making.
Rohit Choudhary
CEO, Acceldata

In 2024, the technology landscape will witness a transformative shift as data evolves from being a valuable asset to the lifeblood of thriving enterprises. Organizations that overlook data quality, integrity, and lineage will be challenged to not only make informed decisions but also realize the full potential of generative AI, LLM and ML applications and use cases. As the year unfolds, I predict that organizations neglecting to craft robust data foundations and strategies will find it increasingly challenging to stay afloat in the swiftly evolving tech industry. Those who fail to adapt and prioritize data fundamentals will struggle to outpace their competitors and may even risk survival in this highly competitive environment.
Armon Petrossian
CEO and Co-Founder, Coalesce

Data as innovation asset

The rapid expansion of data will continue to be a dominant trend in 2024. However, the ability to efficiently gather, process, and utilize this data will become the critical factor limiting or accelerating innovation within organizations. The challenge will lie in developing methods to quickly and securely assimilate this growing data influx, converting it into actionable insights. Companies that can effectively manage this data deluge, turning it into a strategic asset for innovation, will gain a competitive edge in the increasingly data-driven business landscape.
David Boskovic
Founder and CEO, Flatfile

Data as a product

Until recently, only large companies had the expertise and resources needed to create reusable data assets that can be easily repurposed across different teams and applications. Thanks to advancements in the governance products required to build these assets, in 2024 more companies will be able to create reusable data products, greatly accelerating efficiency and data innovation. Multiple teams can benefit from having access to the same data to build a service or application. However, this data must be presented in a way that is secure, well-contextualized, and understandable for users who weren't involved in its production. As data moves farther away from its initial source, you have to do more checks, which becomes increasingly expensive. Starting the data governance process at the source is not only less expensive but is a better way to understand the data's source and how it's schematized. New data governance capabilities, pre-built into products such as cloud data warehouses, databases, and other data infrastructure services, can meet these needs. Developers no longer need to manually build the infrastructure to create and share reusable data products. As a result, reusable data products will no longer be restricted to companies with large, sophisticated data engineering teams. With more companies building reusable data products, in 2024, developers will increase the value of their data and spend more time building innovative data applications and services.
Andrew Sellers
Head of Technology Strategy, Confluent

Go to: 2024 DataOps Predictions - Part 2

Related Links

Hot Topics

The Latest

How AI Agents Are Reshaping DataOps for the Always-On Enterprise

Enterprises today operate in a real-time environment where uninterrupted access to trusted data has become a baseline expectation for users, applications and automated systems. Traditional DataOps models, built on manual effort and human triage, cannot keep pace with this always active demand. AI agents are emerging as the operational backbone, ensuring consistent data availability, reinforcing trustworthiness and enabling a level of scale that manual processes cannot achieve ...

AI Deepfakes: Rethinking Trust in the Workplace

For decades, trust in the digital workplace rested on familiar signals. We trusted faces on video calls, voices on the phone, and emails that appeared to come from people we knew. These cues felt human and intuitive. They anchored how decisions were made, approvals were granted, and access was authorized. AI-powered deepfakes have quietly broken that model ...

Nine in Ten Enterprises Plan Cloud Data Repatriation amid Rising Cloud Costs and Data Sovereignty Mandates

Cloud migration was supposed to be a one-way door. For most enterprises, it turns out it isn't. Cloud data repatriation is a real and growing trend. A new survey ... finds that 89% of organizations plan to expand their on-premises infrastructure footprint over the next two years — and 75% have already moved at least some workloads back from public cloud in the past 24 months. The findings point to a broad rethinking of where data belongs ...

Why ITOps Need Right-Sized AI, Not Bigger Models

Over the past few years, large language models (LLMs) have revolutionized the software industry. Given their ability to excel at multi-step reasoning, LLMs have helped enterprises streamline workflows and adapt to the unknown. However, employing such models comes with sky-high costs, latency issues, and limited flexibility. In the realm of IT operations, it is generally wiser to employ smaller, domain-specific models instead ...

The End of Reactive DevOps: AI-Driven Observability for Zero-Defect Digital Systems

For years, DevOps teams operated under a simple assumption: collect enough telemetry, and you can find and fix any problem. That assumption is breaking down. Modern enterprises now operate across microservices, hybrid cloud environments, APIs, Kubernetes, and highly automated delivery pipelines. Releases happen continuously, dependencies shift constantly, and failures spread faster than teams can diagnose them ...

5 Takeaways from the Observability Forecast for Media and Entertainment

New Relic surveyed IT and engineering leaders from the media and entertainment (M&E) sector to understand what's working — and where challenges persist with their observability practices. The findings reveal how M&E organizations are navigating rising platform complexity, audience expectations, and AI-driven change. Below are five takeaways that stand out ...

Why Moving Off the Cloud Is the Easy Part and What Comes Next Is Where Things Get Hard

Let me start with something I've seen play out more times than I can count. A team hits a wall with the cloud. Costs creep up, then spike. Performance starts to feel inconsistent. Someone in finance asks a simple question like "why did this double?" and nobody has a clean answer ... Maybe this isn't the right place for everything. That realization feels like a breakthrough, like you've identified the problem. In reality, you've just identified the starting line ...

MEAN TIME TO INSIGHT Podcast - Episode 24: Network Observability Tool Sprawl

In MEAN TIME TO INSIGHT Episode 24, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses network observability tool sprawl ... 

Capacity Isn't a Guess: Observability-Driven Sizing for On-Prem Databases

In cloud-native systems, scaling is often as simple as moving a slider. For on-premise databases, the stakes are different. Over-provisioning hardware is expensive. Under-provisioning leads to performance bottlenecks that are difficult to fix once the equipment is in the rack ...

5 Security Principles Every Entrepreneur Should Apply to Leadership

When most people think about cybersecurity, they picture firewalls, encryption, and access controls — technical tools designed to protect systems and data. But beneath the technology lies a deeper set of principles about trust, decision-making, and resilience ... The best leaders don't eliminate risk. They manage it intelligently. And in many ways, cybersecurity offers a surprisingly useful playbook for doing exactly that ...