2025 DataOps Predictions - Part 3

As part of APMdigest's 2025 Predictions Series, industry experts offer predictions on how DataOps and related technologies will evolve and impact business in 2025. Part 3 covers data technologies.

CONSOLIDATION OF DATA INFRASTRUCTURE

The Great Infrastructure Consolidation Will Accelerate: In 2025, we'll witness a significant consolidation of data infrastructure driven by economic pressures and the maturation of AI technologies. Organizations will move away from maintaining separate systems for streaming, batch processing, and AI workloads. Instead, they'll gravitate toward unified platforms that can handle multiple workloads efficiently. This shift isn't just about cost savings — it's about creating a more cohesive data ecosystem where real-time streaming, lakehouse storage, and AI processing work in harmony.
Sijie Guo
Founder and CEO, StreamNative

CONSOLIDATION OF DATA ASSETS

In 2025, organizations will focus on consolidating their data assets to build a unified foundation that powers future innovation, insights, and decision-making.
Ram Palaniappan
CTO, TEKsystems Global Services

CONTEXTUALIZING DATA

Contextualizing data will be the next frontier for data platforms. The evolution of the data platform is essential to the evolution of AI. Next year, we'll see breakthroughs that help LLMs better understand the data they're working with through the semantic layer. Today's data platforms are largely missing the semantic layer of data, which is the understanding of what the data means. For instance, when you have financial data in a table, it's typically the developer or the analyst who is tasked with understanding where that data came from, how it was calculated, and what it means — but this understanding should be baked directly into the data platforms. Having to rely on these additional stakeholders and build that understanding into every application you develop on top of your data is extremely burdensome. As a result, the semantic layer must be pushed down close to the data so that AI can understand the nature of it, and do a much better job at analyzing it. Users don't want to, and shouldn't have to, reinvent the semantic concepts for each application. They must push down to the data layer, that's the next evolution. 
Benoit Dageville
President and Co-Founder, Snowflake

SMART INFRASTRUCTURE

We'll see the emergence of "smart infrastructure" platforms that automatically optimize resource allocation and data movement based on workload patterns and cost constraints.
Sijie Guo
Founder and CEO, StreamNative

DATA LAKEHOUSE

The Data Lakehouse Becomes the Standard for Analytics: More companies are moving from traditional data warehouses to data lakehouses. By 2025, over half of all analytics workloads are expected to run on lakehouse architectures, driven by the cost savings and flexibility they offer. Currently, companies are shifting from cloud data warehouses to lakehouses, not just to save money but to simplify data access patterns and reduce the need for duplicate data storage. Large organizations have reported savings of over 50%, a major win for those with significant data processing needs.
Emmanuel Darras
CEO and Co-Founder, Kestra

HYBRID LAKEHOUSE

The Rise of the Hybrid Lakehouse: The resurgence of on-prem data architectures will see lakehouses expanding into hybrid environments, merging cloud and on-premises data storage seamlessly. The hybrid lakehouse model offers scalability of cloud storage and secure control of on-premises, delivering flexibility and scalability within a unified, accessible framework.
Justin Borgman
Co-Founder and CEO, Starburst

SQL RETURNS TO THE DATA LAKE

SQL is experiencing a comeback in the data lake as table formats like Apache Iceberg simplify data access, enabling SQL engines to outpace Spark. SQL's renewed popularity democratizes data across organizations, fostering data-driven decision-making and expanding data literacy across teams. SQL's accessibility will make data insights widely available, supporting data empowerment.
Justin Borgman
Co-Founder and CEO, Starburst

OPEN TABLE FORMATS

Open table formats, particularly Apache Iceberg, are quickly gaining popularity. Iceberg's flexibility and compatibility with various data processing engines make it a preferred choice. Iceberg provides a standardized table format and integrates it with SQL engines as well as with data platforms, enabling SQL queries to run efficiently on both data lakes and data warehouses. Relying on open table formats allows companies to manage and query large datasets without relying solely on traditional data warehouses. With organizations planning to adopt Iceberg over other formats, its role in big data management is expected to expand, thanks to its strong focus on vendor-agnostic data access patterns, schema evolution, and interoperability.
Emmanuel Darras
CEO and Co-Founder, Kestra

POSTGRESQL: EVERYTHING DATABASE

In 2025, PostgreSQL will solidify its position as the go-to "Everything Database" — the first to fully integrate AI functionality like embeddings directly within its core ecosystem. This will streamline data workflows, eliminate the need for external processing tools, and enable businesses to manage complex data types in one place. With its unique extension capabilities, PostgreSQL is leading the charge toward a future where companies no longer have to rely on standalone or specialized databases.
Avthar Sewrathan
AI Product Lead, Timescale

DATA MESH

Data Mesh Gains Momentum Across Organizations: Data mesh is now more than an IT-driven strategy. It's increasingly led by business units themselves, with data mesh initiatives coming from non-IT teams focused on improving data quality and governance. Data mesh promotes decentralized data ownership, enabling business units to manage their data independently. This setup brings faster decision-making, more agility, and better data access. 
Emmanuel Darras
CEO and Co-Founder, Kestra

DATA FABRIC

A data fabric will become accepted as a pre-cursor to using AI at scale: As businesses increasingly adopt AI to drive innovation, one key challenge remains: ensuring that AI is reliable, responsible, and relevant. AI solutions must be trained on real, company-specific data — not synthetic or generalized data — to deliver accurate, actionable insights. To make this a reality, more and more organizations will adopt a data fabric as their data strategy as it provides the semantics and rich business context that AI requires to be used in real business cases. 
Daniel Yu
SVP, SAP Data and Analytics

MULTI-CLOUD NETWORKING

Enhanced Multi-Cloud Networking for Regulatory Compliance: By 2025, companies will increasingly rely on multi-cloud networking solutions, a capability required to meet diverse data sovereignty and industry-specific regulatory requirements. These advanced solutions will enable seamless connectivity and secure data transfer across cloud environments through robust encryption and access controls and they must also possess the critical ability to identify and remediate risks, threats, and vulnerabilities. CIOs and network architects will prioritize network designs that facilitate secure, efficient data flows, actively minimize regulatory risk, and maintain data integrity across cloud platforms.
Ali Shaikh
Chief Product Officer and Chief Operating Officer, Graphiant

STREAMING-FIRST APPROACH

Streaming-first approach grows with AI: Pressure will grow for more AI and applications to respond to real-time information to drive automation and meet the expectations of consumers. Organizations will adopt a "streaming-first" approach when architecting new applications.  These applications are Event-Driven and will replace traditional application architectures that process data at rest and to a large extent involve request/response communication. This will also facilitate more data sharing of real-time data between totally different domains of a business than previously.
Guillaume Aymé
CEO, Lenses.io

STREAMING DATA PLATFORMS: OBSERVABILITY AND SECURITY

In 2025, streaming data platforms will become indispensable for managing the exponential growth of observability and security data. Organizations will increasingly adopt streaming data platforms to process vast volumes of logs, metrics, and events in real-time, enabling faster threat detection, anomaly resolution, and system optimization to meet the demands of ever-evolving infrastructure and cyber threats.
Bipin Singh
Senior Director of Product Marketing, Redpanda

STREAMING DATA PLATFORMS: AI

In 2025, streaming data platforms will serve as the backbone for agentic AI, RAG AI and sovereign AI applications, providing the low-latency, high-throughput capabilities required to power autonomous decision-making systems and ensuring compliance with data sovereignty requirements.
Bipin Singh
Senior Director of Product Marketing, Redpanda

REAL-TIME DATA STREAMING FABRIC

Businesses will look to accelerate hyper-connecting applications and architectures across all parts of their business through real-time data streams. This "streaming fabric" across a business will blur the lines between previously isolated different AI, analytics and software architectures and allow connecting systems across business lines such as finance, ecommerce, manufacturing, distribution, supply chain. This connectivity will allow applications to be built that offer new digital consumer-facing services as well as ones that provide new levels of automation within a business.
Guillaume Aymé
CEO, Lenses.io

Related Links

Hot Topics

The Latest

Why Cloud Reliability Comes Down to People and How to Improve It

Cloud outages are usually described as technical failures. When a service goes down, a dependency breaks, or a region has issues, the focus immediately shifts to infrastructure. But if you look closely at how these incidents actually unfold, the root cause is rarely the technology itself. It is almost always tied to decisions made earlier, during design, implementation, or day-to-day operations. The system behaves the way it was built. The real question is how it was built ...

Why IT Keeps Getting Handed an AI Training Problem It Can't Solve Alone

77% of leaders say their teams need AI skills urgently. 64% say their organization plans to train current employees rather than hire new ones. So far, so reasonable. The part that surprised me is who's been put in charge: 34% of those leaders say IT and engineering own the AI skills mandate. Learning and Development or HR own it at 7% of organizations. That's roughly five-to-one in favor of the people who understand the tools, over the people whose actual job is teaching adults how to learn new ones ...

Bringing IBM i Into the Modern Observability Stack: The Missing Piece of Enterprise AIOps

In the ever-evolving digital landscape, enterprises are increasingly focused on enhancing their observability stacks to gain deeper insights into their IT environments. Observability has become a cornerstone of modern IT operations, enabling organizations to monitor, diagnose, and optimize their systems with unprecedented precision. However, a critical piece of the puzzle often goes unnoticed in this transformation: IBM i ...

89% of Frontend Teams Use AI in Their Workflows, but Few Use It for Observability

We just surveyed 300 frontend and mobile engineers across 16 countries, and the finding that keeps sticking with me isn't the one about AI. It's this: 74% of engineering teams rate themselves in the "middle" of the observability maturity scale. Not reactive, not strategic. Stuck in the middle. They have dashboards, they have tracing, they have alerts. And yet when something goes wrong, they still can't tell you why ...

MEAN TIME TO INSIGHT Podcast - Episode 25: AI's Impact on WAN

In MEAN TIME TO INSIGHT Episode 25, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses  AI's impact on the Wide Area Network (WAN) ... 

6 Signs Your APM Dashboard Looks Healthy but Your App Is Not

Application performance monitoring (APM) dashboards are only as useful as what they are configured to measure. The default setup covers obvious failure modes such as downtime, error spikes, and latency breaches, but it does not cover everything. Some failures produce no alerts or anomalies. The dashboard stays green while users experience a broken product. Here are six signs that is happening ...

AI Innovation Was the First Phase. Governance and Accountability Are the Next ...

The race to deploy AI is largely over. Most enterprises have entered it. The question now is not whether artificial intelligence is running inside the organization. The question is whether anyone is genuinely responsible for what it does. That is not a technical question. It is a leadership one. And most organizations are not yet structured to answer it honestly ...

Your AI Agent's Biggest Problem Isn't the Model - It's the Data

A new analysis of 250 real-world queries across common retail tasks, such as product pricing, availability, ratings, shipping and specifications, reveals systemic inefficiency at the heart of web-based AI agents. On average, 97.9% of the data retrieved by agents from live web pages is irrelevant to the query being answered. Specifically, the average page ingested ran nearly 9,000 characters, while the average answer was just 32 characters, resulting in a noise-to-signal ratio of 278:1. Price queries were the most extreme outlier, with noise rates approaching 99.5%. That's not a rounding error. That's a structural problem ...

Observability Is the New Control Plane for Enterprise Transformation

The enterprises that will define the next decade are not the ones that deployed the most technology. They are the ones who understood what their technology was actually doing. That distinction is not a philosophical point. It is the central operational challenge facing every organization that has spent the last five years modernizing at speed ...

When AI Becomes the Corporate OS

AI is becoming the operating system of the enterprise. It acts as an invisible coordination layer that understands intent, connects systems, and executes work across complex SaaS environments. Previously, employees had to click through multiple systems — CRM, ERP, support tools, collaboration platforms — to complete a single task. Now, instead of navigating each application manually, they can simply state what they need to accomplish ...

2025 DataOps Predictions - Part 3

As part of APMdigest's 2025 Predictions Series, industry experts offer predictions on how DataOps and related technologies will evolve and impact business in 2025. Part 3 covers data technologies.

CONSOLIDATION OF DATA INFRASTRUCTURE

The Great Infrastructure Consolidation Will Accelerate: In 2025, we'll witness a significant consolidation of data infrastructure driven by economic pressures and the maturation of AI technologies. Organizations will move away from maintaining separate systems for streaming, batch processing, and AI workloads. Instead, they'll gravitate toward unified platforms that can handle multiple workloads efficiently. This shift isn't just about cost savings — it's about creating a more cohesive data ecosystem where real-time streaming, lakehouse storage, and AI processing work in harmony.
Sijie Guo
Founder and CEO, StreamNative

CONSOLIDATION OF DATA ASSETS

In 2025, organizations will focus on consolidating their data assets to build a unified foundation that powers future innovation, insights, and decision-making.
Ram Palaniappan
CTO, TEKsystems Global Services

CONTEXTUALIZING DATA

Contextualizing data will be the next frontier for data platforms. The evolution of the data platform is essential to the evolution of AI. Next year, we'll see breakthroughs that help LLMs better understand the data they're working with through the semantic layer. Today's data platforms are largely missing the semantic layer of data, which is the understanding of what the data means. For instance, when you have financial data in a table, it's typically the developer or the analyst who is tasked with understanding where that data came from, how it was calculated, and what it means — but this understanding should be baked directly into the data platforms. Having to rely on these additional stakeholders and build that understanding into every application you develop on top of your data is extremely burdensome. As a result, the semantic layer must be pushed down close to the data so that AI can understand the nature of it, and do a much better job at analyzing it. Users don't want to, and shouldn't have to, reinvent the semantic concepts for each application. They must push down to the data layer, that's the next evolution. 
Benoit Dageville
President and Co-Founder, Snowflake

SMART INFRASTRUCTURE

We'll see the emergence of "smart infrastructure" platforms that automatically optimize resource allocation and data movement based on workload patterns and cost constraints.
Sijie Guo
Founder and CEO, StreamNative

DATA LAKEHOUSE

The Data Lakehouse Becomes the Standard for Analytics: More companies are moving from traditional data warehouses to data lakehouses. By 2025, over half of all analytics workloads are expected to run on lakehouse architectures, driven by the cost savings and flexibility they offer. Currently, companies are shifting from cloud data warehouses to lakehouses, not just to save money but to simplify data access patterns and reduce the need for duplicate data storage. Large organizations have reported savings of over 50%, a major win for those with significant data processing needs.
Emmanuel Darras
CEO and Co-Founder, Kestra

HYBRID LAKEHOUSE

The Rise of the Hybrid Lakehouse: The resurgence of on-prem data architectures will see lakehouses expanding into hybrid environments, merging cloud and on-premises data storage seamlessly. The hybrid lakehouse model offers scalability of cloud storage and secure control of on-premises, delivering flexibility and scalability within a unified, accessible framework.
Justin Borgman
Co-Founder and CEO, Starburst

SQL RETURNS TO THE DATA LAKE

SQL is experiencing a comeback in the data lake as table formats like Apache Iceberg simplify data access, enabling SQL engines to outpace Spark. SQL's renewed popularity democratizes data across organizations, fostering data-driven decision-making and expanding data literacy across teams. SQL's accessibility will make data insights widely available, supporting data empowerment.
Justin Borgman
Co-Founder and CEO, Starburst

OPEN TABLE FORMATS

Open table formats, particularly Apache Iceberg, are quickly gaining popularity. Iceberg's flexibility and compatibility with various data processing engines make it a preferred choice. Iceberg provides a standardized table format and integrates it with SQL engines as well as with data platforms, enabling SQL queries to run efficiently on both data lakes and data warehouses. Relying on open table formats allows companies to manage and query large datasets without relying solely on traditional data warehouses. With organizations planning to adopt Iceberg over other formats, its role in big data management is expected to expand, thanks to its strong focus on vendor-agnostic data access patterns, schema evolution, and interoperability.
Emmanuel Darras
CEO and Co-Founder, Kestra

POSTGRESQL: EVERYTHING DATABASE

In 2025, PostgreSQL will solidify its position as the go-to "Everything Database" — the first to fully integrate AI functionality like embeddings directly within its core ecosystem. This will streamline data workflows, eliminate the need for external processing tools, and enable businesses to manage complex data types in one place. With its unique extension capabilities, PostgreSQL is leading the charge toward a future where companies no longer have to rely on standalone or specialized databases.
Avthar Sewrathan
AI Product Lead, Timescale

DATA MESH

Data Mesh Gains Momentum Across Organizations: Data mesh is now more than an IT-driven strategy. It's increasingly led by business units themselves, with data mesh initiatives coming from non-IT teams focused on improving data quality and governance. Data mesh promotes decentralized data ownership, enabling business units to manage their data independently. This setup brings faster decision-making, more agility, and better data access. 
Emmanuel Darras
CEO and Co-Founder, Kestra

DATA FABRIC

A data fabric will become accepted as a pre-cursor to using AI at scale: As businesses increasingly adopt AI to drive innovation, one key challenge remains: ensuring that AI is reliable, responsible, and relevant. AI solutions must be trained on real, company-specific data — not synthetic or generalized data — to deliver accurate, actionable insights. To make this a reality, more and more organizations will adopt a data fabric as their data strategy as it provides the semantics and rich business context that AI requires to be used in real business cases. 
Daniel Yu
SVP, SAP Data and Analytics

MULTI-CLOUD NETWORKING

Enhanced Multi-Cloud Networking for Regulatory Compliance: By 2025, companies will increasingly rely on multi-cloud networking solutions, a capability required to meet diverse data sovereignty and industry-specific regulatory requirements. These advanced solutions will enable seamless connectivity and secure data transfer across cloud environments through robust encryption and access controls and they must also possess the critical ability to identify and remediate risks, threats, and vulnerabilities. CIOs and network architects will prioritize network designs that facilitate secure, efficient data flows, actively minimize regulatory risk, and maintain data integrity across cloud platforms.
Ali Shaikh
Chief Product Officer and Chief Operating Officer, Graphiant

STREAMING-FIRST APPROACH

Streaming-first approach grows with AI: Pressure will grow for more AI and applications to respond to real-time information to drive automation and meet the expectations of consumers. Organizations will adopt a "streaming-first" approach when architecting new applications.  These applications are Event-Driven and will replace traditional application architectures that process data at rest and to a large extent involve request/response communication. This will also facilitate more data sharing of real-time data between totally different domains of a business than previously.
Guillaume Aymé
CEO, Lenses.io

STREAMING DATA PLATFORMS: OBSERVABILITY AND SECURITY

In 2025, streaming data platforms will become indispensable for managing the exponential growth of observability and security data. Organizations will increasingly adopt streaming data platforms to process vast volumes of logs, metrics, and events in real-time, enabling faster threat detection, anomaly resolution, and system optimization to meet the demands of ever-evolving infrastructure and cyber threats.
Bipin Singh
Senior Director of Product Marketing, Redpanda

STREAMING DATA PLATFORMS: AI

In 2025, streaming data platforms will serve as the backbone for agentic AI, RAG AI and sovereign AI applications, providing the low-latency, high-throughput capabilities required to power autonomous decision-making systems and ensuring compliance with data sovereignty requirements.
Bipin Singh
Senior Director of Product Marketing, Redpanda

REAL-TIME DATA STREAMING FABRIC

Businesses will look to accelerate hyper-connecting applications and architectures across all parts of their business through real-time data streams. This "streaming fabric" across a business will blur the lines between previously isolated different AI, analytics and software architectures and allow connecting systems across business lines such as finance, ecommerce, manufacturing, distribution, supply chain. This connectivity will allow applications to be built that offer new digital consumer-facing services as well as ones that provide new levels of automation within a business.
Guillaume Aymé
CEO, Lenses.io

Related Links

Hot Topics

The Latest

Why Cloud Reliability Comes Down to People and How to Improve It

Cloud outages are usually described as technical failures. When a service goes down, a dependency breaks, or a region has issues, the focus immediately shifts to infrastructure. But if you look closely at how these incidents actually unfold, the root cause is rarely the technology itself. It is almost always tied to decisions made earlier, during design, implementation, or day-to-day operations. The system behaves the way it was built. The real question is how it was built ...

Why IT Keeps Getting Handed an AI Training Problem It Can't Solve Alone

77% of leaders say their teams need AI skills urgently. 64% say their organization plans to train current employees rather than hire new ones. So far, so reasonable. The part that surprised me is who's been put in charge: 34% of those leaders say IT and engineering own the AI skills mandate. Learning and Development or HR own it at 7% of organizations. That's roughly five-to-one in favor of the people who understand the tools, over the people whose actual job is teaching adults how to learn new ones ...

Bringing IBM i Into the Modern Observability Stack: The Missing Piece of Enterprise AIOps

In the ever-evolving digital landscape, enterprises are increasingly focused on enhancing their observability stacks to gain deeper insights into their IT environments. Observability has become a cornerstone of modern IT operations, enabling organizations to monitor, diagnose, and optimize their systems with unprecedented precision. However, a critical piece of the puzzle often goes unnoticed in this transformation: IBM i ...

89% of Frontend Teams Use AI in Their Workflows, but Few Use It for Observability

We just surveyed 300 frontend and mobile engineers across 16 countries, and the finding that keeps sticking with me isn't the one about AI. It's this: 74% of engineering teams rate themselves in the "middle" of the observability maturity scale. Not reactive, not strategic. Stuck in the middle. They have dashboards, they have tracing, they have alerts. And yet when something goes wrong, they still can't tell you why ...

MEAN TIME TO INSIGHT Podcast - Episode 25: AI's Impact on WAN

In MEAN TIME TO INSIGHT Episode 25, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses  AI's impact on the Wide Area Network (WAN) ... 

6 Signs Your APM Dashboard Looks Healthy but Your App Is Not

Application performance monitoring (APM) dashboards are only as useful as what they are configured to measure. The default setup covers obvious failure modes such as downtime, error spikes, and latency breaches, but it does not cover everything. Some failures produce no alerts or anomalies. The dashboard stays green while users experience a broken product. Here are six signs that is happening ...

AI Innovation Was the First Phase. Governance and Accountability Are the Next ...

The race to deploy AI is largely over. Most enterprises have entered it. The question now is not whether artificial intelligence is running inside the organization. The question is whether anyone is genuinely responsible for what it does. That is not a technical question. It is a leadership one. And most organizations are not yet structured to answer it honestly ...

Your AI Agent's Biggest Problem Isn't the Model - It's the Data

A new analysis of 250 real-world queries across common retail tasks, such as product pricing, availability, ratings, shipping and specifications, reveals systemic inefficiency at the heart of web-based AI agents. On average, 97.9% of the data retrieved by agents from live web pages is irrelevant to the query being answered. Specifically, the average page ingested ran nearly 9,000 characters, while the average answer was just 32 characters, resulting in a noise-to-signal ratio of 278:1. Price queries were the most extreme outlier, with noise rates approaching 99.5%. That's not a rounding error. That's a structural problem ...

Observability Is the New Control Plane for Enterprise Transformation

The enterprises that will define the next decade are not the ones that deployed the most technology. They are the ones who understood what their technology was actually doing. That distinction is not a philosophical point. It is the central operational challenge facing every organization that has spent the last five years modernizing at speed ...

When AI Becomes the Corporate OS

AI is becoming the operating system of the enterprise. It acts as an invisible coordination layer that understands intent, connects systems, and executes work across complex SaaS environments. Previously, employees had to click through multiple systems — CRM, ERP, support tools, collaboration platforms — to complete a single task. Now, instead of navigating each application manually, they can simply state what they need to accomplish ...