3 Critical Steps for Implementing a Secure Data Mesh
January 31, 2024

Claude Zwicker

Share this

As data insights continue to be the key factor in driving business innovation and growth, organizations are constantly refining their data strategies, exploring frameworks like data mesh to give more users self-service access in producing and consuming data. Recent data shows that nearly half of data leaders identified data mesh as a primary area of investment for 2023.

Data mesh has been revolutionary for many data-driven organizations. Before data mesh, teams had to coordinate data access and use through centralized IT bottlenecks, leading to long waiting times to access data and ultimately resulting in frustration for both data producers and consumers. With data mesh, teams can independently develop and manage their own data products through the decentralized data ownership and the enablement of domain experts. Not only does this allow organizations to boost their data-driven initiatives, but it also helps them enhance everything from data democratization to alignment between business operations and data resources, and sustain growth at scale.

Whilst the distribution and delegation of responsibilities promoted through data mesh has many benefits, achieving an implementation that fulfills data security standards is not without its challenges. Many organizations run into issues around data access, governance, and privacy. Let's explore these issues further and some steps organizations can take to help overcome them.

Data Mesh Security Challenges

The need to secure data mesh is only becoming more urgent as regulations increase and the US federal government continues to roll out more data privacy actions. What makes this a complex process is the fledgling nature of the data mesh combined with its distributed composition. Data security must be applied in a way that simultaneously protects individual domains and the entire ecosystem, without hindering data accessibility and innovation.

As data mesh implementations become more regular, there are three main challenges I see organizations experiencing when it comes to implementing and securing the architecture:

Decentralized ownership and access control. While the decentralized data ownership that comes with data mesh offers a range of benefits, it can also be difficult to keep track of who owns what when it comes to data collection, processing, sharing and use. Systems that are built to enable cross-domain data discovery, access and sharing can help address this challenge, but can also lead to a larger attack surface for bad actors. On top of this, it becomes more challenging for rules to stay consistent across data products, risking the security of the data.

Data governance. Along the same lines, it's no surprise that centralized ecosystems are easier to protect when it comes to data governance and compliance with regulations as opposed to decentralized environments. With distributed domains and data locations, security requirements become much more complex, requiring additional governance policies for each domain, and a way to oversee the security and compliance of the entire domain-based framework. Access and governance requirements are also federated, making it more difficult to consistently and effectively protect data.

Privacy in a self-service environment. Less oversight into data access and outdated/inadequate controls can also increase the likelihood of data misuse across domains. This is a growing concern as today's organizations collect increasing volumes of sensitive, personal data in order to provide consumers with more personalized products, experiences and services. Because this personal data — if exposed or accessed by an unauthorized party — can easily harm the data subject, modern data rules and regulations are requiring stricter privacy protections be enforced on data ecosystems.

Three Steps That Can Help

There are a few best practices organizations can follow to help achieve an efficient, secure, and distributed data ecosystem.

1. Maintaining consistent metadata. The first step in securing any data mesh architecture should be creating and applying a consistent metadata identification and tagging system. Why? Because users and administrators must have a thorough understanding of the resources at their disposal in order to protect and secure any data ecosystem.

Metadata allows them to identify and understand these ecosystem parts, from data sets to data users, by providing critical contextual information about resources or users that is vital to the system's operation. This helps with efficient access management, analytics, monitoring and compliance — all critical elements of data mesh. However, metadata is ineffective unless it can be consistently attributed and understood across domains.

To maintain consistent metadata, organizations can leverage tools that offer sensitive data discovery (SDD) capabilities, enabling teams to assess their data and ensure that it is tagged and classified appropriately. This helps data teams gain a better, holistic view of the resources across their distributed data mesh for enhanced data security.

2. Employing global & local policy management. Establishing a balance of both horizontal (global) and vertical (local) policies is crucial to data mesh security and governance. In the distributed domains of data mesh architectures, policies can be applied locally within specific domains. But these rely solely on domain-based policies, limiting consistency across the data ecosystem and requiring great manual effort and time to maintain. Applying policies globally across domains is not a perfect solution either — it improves consistency, but overlooks the unique requirements of each domain's purpose, users and specific data resources.

That's why finding the right balance of horizontal and vertical policy management, and maintaining them as scale, is key. A federated governance framework can help create, apply, and maintain policies at both the global and local level. Within this framework, domain-level policy management is delegated to the teams that own the data, and the responsibility of global policies remains with security and governance teams.

All teams must maintain rigorous activity monitoring across domains so that they can have complete oversight of global policy application and local policy enforcement within specific domains. This helps them respond to and manage security incidents as quickly as possible, and effectively manage both global and local data security.

3. Foster organizational alignment. Adopting a data mesh framework is an organizational change. In order to achieve true success, data teams, engineers and leaders must also be aligned and behind the initiative as well.

A large part of this involves organizations identifying internal data mesh champions to lead the charge and help teams adopt a data mesh mindset. Leaders must also learn to effectively collaborate and communicate with one another. Once technical, security, business and compliance stakeholders are aligned, virtually any organization can establish an effective and secure data mesh framework.

At the end of the day, every organization will have a different approach to data mesh depending on their industry, business needs, and data demands. But security remains a critical component across all data mesh strategies, at any stage. By following these steps, businesses can effectively jumpstart their data mesh strategy.

Claude Zwicker is Senior Product Manager at Immuta
Share this

The Latest

February 21, 2024

Generative AI will usher in advantages within various industries. However, the technology is still nascent, and according to the recent Dynatrace survey there are many challenges and risks that organizations need to overcome to use this technology effectively ...

February 20, 2024

In today's digital era, monitoring and observability are indispensable in software and application development. Their efficacy lies in empowering developers to swiftly identify and address issues, enhance performance, and deliver flawless user experiences. Achieving these objectives requires meticulous planning, strategic implementation, and consistent ongoing maintenance. In this blog, we're sharing our five best practices to fortify your approach to application performance monitoring (APM) and observability ...

February 16, 2024

In MEAN TIME TO INSIGHT Episode 3, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at Enterprise Management Associates (EMA) discusses network security with Chris Steffen, VP of Research Covering Information Security, Risk, and Compliance Management at EMA ...

February 15, 2024

In a time where we're constantly bombarded with new buzzwords and technological advancements, it can be challenging for businesses to determine what is real, what is useful, and what they truly need. Over the years, we've witnessed the rise and fall of various tech trends, such as the promises (and fears) of AI becoming sentient and replacing humans to the declaration that data is the new oil. At the end of the day, one fundamental question remains: How can companies navigate through the tech buzz and make informed decisions for their future? ...

February 14, 2024

We increasingly see companies using their observability data to support security use cases. It's not entirely surprising given the challenges that organizations have with legacy SIEMs. We wanted to dig into this evolving intersection of security and observability, so we surveyed 500 security professionals — 40% of whom were either CISOs or CSOs — for our inaugural State of Security Observability report ...

February 13, 2024

Cloud computing continues to soar, with little signs of slowing down ... But, as with any new program, companies are seeing substantial benefits in the cloud but are also navigating budgetary challenges. With an estimated 94% of companies using cloud services today, priorities for IT teams have shifted from purely adoption-based to deploying new strategies. As they explore new territories, it can be a struggle to exploit the full value of their spend and the cloud's transformative capabilities ...

February 12, 2024

What will the enterprise of the future look like? If we asked this question three years ago, I doubt most of us would have pictured today as we know it: a future where generative AI has become deeply integrated into business and even our daily lives ...

February 09, 2024

With a focus on GenAI, industry experts offer predictions on how AI will evolve and impact IT and business in 2024. Part 5, the final installment in this series, covers the advantages AI will deliver: Generative AI will become increasingly important for resolving complicated data integration challenges, essentially providing a natural-language intermediary between data endpoints ...

February 08, 2024

With a focus on GenAI, industry experts offer predictions on how AI will evolve and impact IT and business in 2024. Part 4 covers the challenges of AI: In the short term, the rapid development and adoption of AI tools and products leveraging AI services will lead to an increase in biased outputs ...

February 07, 2024

With a focus on GenAI, industry experts offer predictions on how AI will evolve and impact IT and business in 2024. Part 3 covers the technologies that will drive AI: The question on every leader's mind in 2023 was - how soon will I see the return on my AI investment? The answer may lie in quantum computing ...