Elastic Brings Real-Time Graph Analytics to the Elastic Stack
March 31, 2016
Share this

Elastic unveiled Graph, a new extension for Elasticsearch and Kibana that allows anyone to uncover, understand, and explore the relationships that live in their data.

By combining the speed and relevance-ranking of search with graph exploration, Graph opens up a whole host of new use-cases with the Elastic Stack.

"We built Graph to help you ask new types of questions about the data you store in Elasticsearch," said Steve Kearns, Sr. Director of Product Management at Elastic. "By looking at the relationships in your data through the lens of relevance, it becomes easy to answer questions that previously would involve multiple systems, batch jobs or machine learning."

When you store data in Elasticsearch — products, users, documents, logs — this data often contains references or properties that represent connections between objects, entities, people, or machines. The best way to explore these connections is to see them, which Graph provides via a Kibana plugin. Like everything at Elastic, this UI is built on a simple, but powerful API that leverages Elastic’s experience in relevance ranking to surface the most meaningful connections that live in your data. This approach to graph exploration opens a wide range of new use-cases for the Elastic Stack, without requiring new index formats, by allowing users to query their existing data in new ways.

Graph makes it easy to answer complex questions and address use-cases such as behavioral analysis, fraud, cybersecurity, drug discovery, personalized medicine, and to build personalized recommendations based on continuous real-time data. Graph automatically identifies the most important connections, separating the signal from the noise by employing relevance ranking specific to each query. Because it is built on Elasticsearch, Graph benefits from high scalability and near-real-time data availability, enabling answers that evolve as your data changes.

When data is added to Elasticsearch, the indexing process tracks and counts the values in each field of the document, updates global frequencies, and prepares the data for a wide range of queries. These statistics are used to perform relevance ranking for search and efficiently summarize the data in Elasticsearch aggregations. With Graph, the Elastic Stack uses these statistics in new ways — first to identify relationships within and across sets of documents, and then to prioritize the most relevant relationships for the given query.

In contrast, traditional graph techniques are based on a simple count of the frequency of a given relationship. This approach has the drawback that elements with the most connections — the Shawshank Redemption in movie recommendation data or Starbucks in credit card purchase data — are returned as the most important. With Graph in Elasticsearch, relevance is calculated by correlating the significance of each relationship in comparison to global averages, bringing back the important results, and avoiding frequent connections.

"Graph is an excellent example of the limitless possibilities within our products and how we strive to make it as easy as possible for our users to leverage the power of the Elastic Stack," said Shay Banon, Co-Founder and CTO of Elastic. "I am proud to see our company continue to innovate, and I can’t wait to see the new ways in which our customers will adopt Graph to solve really challenging problems and use cases."

Share this

The Latest

June 25, 2020

I've had the opportunity to work with a number of organizations embarking on their AIOps journey. I always advise them to start by evaluating their needs and the possibilities AIOps can bring to them through five different levels of AIOps maturity. This is a strategic approach that allows enterprises to achieve complete automation for long-term success ...

June 24, 2020

Sumo Logic recently commissioned an independent market research study to understand the industry momentum behind continuous intelligence — and the necessity for digital organizations to embrace a cloud-native, real-time continuous intelligence platform to support the speed and agility of business for faster decision-making, optimizing security, driving new innovation and delivering world-class customer experiences. Some of the key findings include ...

June 23, 2020

When it comes to viruses, it's typically those of the computer/digital variety that IT is concerned about. But with the ongoing pandemic, IT operations teams are on the hook to maintain business functions in the midst of rapid and massive change. One of the biggest challenges for businesses is the shift to remote work at scale. Ensuring that they can continue to provide products and services — and satisfy their customers — against this backdrop is challenging for many ...

June 22, 2020

Teams tasked with developing and delivering software are under pressure to balance the business imperative for speed with high customer expectations for quality. In the course of trying to achieve this balance, engineering organizations rely on a variety of tools, techniques and processes. The 2020 State of Software Quality report provides a snapshot of the key challenges organizations encounter when it comes to delivering quality software at speed, as well as how they are approaching these hurdles. This blog introduces its key findings ...

June 18, 2020

For IT teams, run-the-business, commodity areas such as employee help desks, device support and communication platforms are regularly placed in the crosshairs for cost takeout, but these areas are also highly visible to employees. Organizations can improve employee satisfaction and business performance by building unified functions that are measured by employee experience rather than price. This approach will ultimately fund transformation, as well as increase productivity and innovation ...

June 17, 2020

In the agile DevOps framework, there is a vital piece missing; something that previous approaches to application development did well, but has since fallen by the wayside. That is, the post-delivery portion of the toolchain. Without continuous cloud optimization, the CI/CD toolchain still produces massive inefficiencies and overspend ...

June 16, 2020

The COVID-19 pandemic has exponentially accelerated digital transformation projects. To better understand where IT professionals are turning for help, we analyzed the online behaviors of IT decision-makers. Our research found an increase in demand for resources related to APM, microservices and dependence on cloud services ...

June 15, 2020

The rush to the public cloud has now slowed as organizations realized that it is not a "one size fits all" solution. The main issue is the lack of deep visibility into the performance of applications provided by the host. Our own research has recently revealed that 32% of public cloud resources are currently under-utilized, and without proper direction and guidance, this will remain the case ...

June 11, 2020

The global shift to working from home (WFH) enforced by COVID-19 stay-at-home orders has had a massive impact on everyone's working lives, not just in the way they remotely interact with their teams and IT systems, but also in how they spend their working days. With both governments and businesses committed to slowly opening up offices, it's increasingly clear that a high prevalence of remote work will continue throughout 2020 and beyond. This situation begets important questions ...

June 10, 2020
In recent years, with the emergence of newer technologies ranging from the cloud to machine learning, IT modernization has evolved from a replacement of end-of-life infrastructure to an enabler of innovation and business value. It is a complex process that can take months or even years, but a recent survey shows that the effort begins to deliver measurable results almost as soon as an organization executes the first steps on its roadmap ...