Elastic Introduces Cross-Cluster Replication
April 09, 2019
Share this

Elastic, the company behind Elasticsearch and the Elastic Stack, announced the general availability of cross-cluster replication (CCR) in version 6.7 of the Elastic Stack.

CCR enables native replication of data between Elasticsearch clusters, whether they are located in the same datacenter or across the world. Cross-datacenter replication has been a requirement for mission-critical applications on the Elastic Stack and was previously solved with additional technologies. With the latest release of the Elastic Stack, no additional technologies are needed to replicate data across datacenters, geographies, or Elasticsearch clusters.

"Cross-cluster replication in Elasticsearch has been the most asked-for feature by our customers, who have been putting more and more business-critical data into their Elasticsearch clusters,” said Shay Banon, founder and CEO of Elastic. "Built-in cross-cluster replication helps solve for disaster recovery, geo data locality, and centralized data location. It is a critical step in Elasticsearch's evolution as a data store."

The native cross-cluster replication feature is efficient, scalable, and reliable and enables a variety of mission-critical use cases for Elasticsearch and the Elastic Stack.

- Disaster recovery (DR) / high availability (HA): Tolerance to withstand a datacenter or region outage has become a requirement for many mission-critical applications. Replicating data across Elasticsearch clusters ensures that if a cluster is unavailable due to a datacenter or infrastructure outage, one or more copies of the data are available in additional locations to ensure business continuity.

- Data locality / geo-proximity: CCR can be used to replicate data from a central location to geographically distributed locations to maintain copies of data that are closer to end users. For example, a product catalog or reference dataset may be replicated to 20 or more datacenters around the world to minimize the distance between the data and the end user. Another example of geographically replicating data may be a stock trading firm with offices in London and New York. All trades in the London office are written locally and replicated to the New York office, and all trades in the New York office are written locally and replicated to London. Both offices then have a global view of all trades.

- Centralized reporting: Replicating data from a large number of dispersed clusters back to a centralized reporting cluster is another valuable use case for CCR. This is useful when it may not be efficient to send cross-cluster queries across a large network. For example, a large global bank may have 100 Elasticsearch clusters around the world, each within a different bank branch. CCR can be used to replicate events from all 100 banks around the world back to a central cluster where they can be analyzed and aggregated locally.

With CCR natively integrated into Elasticsearch, users are freed of the burden and drawbacks of managing complicated solutions, and also enjoy additional advantages that are not available in existing workarounds (e.g., comprehensive error handling). CCR includes both APIs within Elasticsearch and UIs in Kibana for managing and monitoring replication jobs.

The CCR feature in Elasticsearch is designed and architected with government-grade security controls at its heart. All replicated data is encrypted in transit across Elasticsearch clusters, and specific access controls can be specified at the cluster, index, document, or field level. Furthermore, CCR bakes in support for centralized authentication systems, which is a requirement for some organizations.

Replication management and administration are important to ensure CCR architectures are correctly configured and working as designed. Elastic has released a robust management and monitoring UI within Kibana specifically for Elasticsearch CCR. Connections across Elasticsearch clusters can be configured and monitored and data replication can be initiated all with a few clicks.

Share this

The Latest

October 16, 2019

Modern enterprises are generating data at an unprecedented rate but aren't taking advantage of all the data available to them in order to drive real-time, actionable insights. According to a recent study commissioned by Actian, more than half of enterprises today are unable to efficiently manage nor effectively use data to drive decision-making ...

October 15, 2019

According to a study by Forrester Research, an enhanced UX design can increase the conversion rate by 400%. If UX has become the ultimate arbiter in determining the success or failure of a product or service, let us first understand what UX is all about ...

October 10, 2019

The requirements of an APM tool are now much more complex than they've ever been. Not only do they need to trace a user transaction across numerous microservices on the same system, but they also need to happen pretty fast ...

October 09, 2019

Performance monitoring is an old problem. As technology has advanced, we've had to evolve how we monitor applications. Initially, performance monitoring largely involved sending ICMP messages to start troubleshooting a down or slow application. Applications have gotten much more complex, so this is no longer enough. Now we need to know not just whether an application is broken, but why it broke. So APM has had to evolve over the years for us to get there. But how did this evolution take place, and what happens next? Let's find out ...

October 08, 2019

There are some IT organizations that are using DevOps methodology but are wary of getting bogged down in ITSM procedures. But without at least some ITSM controls in place, organizations lose their focus on systematic customer engagement, making it harder for them to scale ...

October 07, 2019
OK, I admit it. "Service modeling" is an awkward term, especially when you're trying to frame three rather controversial acronyms in the same overall place: CMDB, CMS and DDM. Nevertheless, that's exactly what we did in EMA's most recent research: <span style="font-style: italic;">Service Modeling in the Age of Cloud and Containers</span>. The goal was to establish a more holistic context for looking at the synergies and differences across all these areas ...
October 03, 2019

If you have deployed a Java application in production, you've probably encountered a situation where the application suddenly starts to take up a large amount of CPU. When this happens, application response becomes sluggish and users begin to complain about slow response. Often the solution to this problem is to restart the application and, lo and behold, the problem goes away — only to reappear a few days later. A key question then is: how to troubleshoot high CPU usage of a Java application? ...

October 02, 2019

Operations are no longer tethered tightly to a main office, as the headquarters-centric model has been retired in favor of a more decentralized enterprise structure. Rather than focus the business around a single location, enterprises are now comprised of a web of remote offices and individuals, where network connectivity has broken down the geographic barriers that in the past limited the availability of talent and resources. Key to the success of the decentralized enterprise model is a new generation of collaboration and communication tools ...

October 01, 2019

To better understand the AI maturity of businesses, Dotscience conducted a survey of 500 industry professionals. Research findings indicate that although enterprises are dedicating significant time and resources towards their AI deployments, many data science and ML teams don't have the adequate tools needed to properly collaborate on, build and deploy AI models efficiently ...

September 30, 2019

Digital transformation, migration to the enterprise cloud and increasing customer demands are creating a surge in IT complexity and the associated costs of managing it. Technical leaders around the world are concerned about the effect this has on IT performance and ultimately, their business according to a new report from Dynatrace, based on an independent global survey of 800 CIOs, Top Challenges for CIOs in a Software-Driven, Hybrid, Multi-Cloud World ...