Elastic Brings Real-Time Graph Analytics to the Elastic Stack
March 31, 2016
Share this

Elastic unveiled Graph, a new extension for Elasticsearch and Kibana that allows anyone to uncover, understand, and explore the relationships that live in their data.

By combining the speed and relevance-ranking of search with graph exploration, Graph opens up a whole host of new use-cases with the Elastic Stack.

"We built Graph to help you ask new types of questions about the data you store in Elasticsearch," said Steve Kearns, Sr. Director of Product Management at Elastic. "By looking at the relationships in your data through the lens of relevance, it becomes easy to answer questions that previously would involve multiple systems, batch jobs or machine learning."

When you store data in Elasticsearch — products, users, documents, logs — this data often contains references or properties that represent connections between objects, entities, people, or machines. The best way to explore these connections is to see them, which Graph provides via a Kibana plugin. Like everything at Elastic, this UI is built on a simple, but powerful API that leverages Elastic’s experience in relevance ranking to surface the most meaningful connections that live in your data. This approach to graph exploration opens a wide range of new use-cases for the Elastic Stack, without requiring new index formats, by allowing users to query their existing data in new ways.

Graph makes it easy to answer complex questions and address use-cases such as behavioral analysis, fraud, cybersecurity, drug discovery, personalized medicine, and to build personalized recommendations based on continuous real-time data. Graph automatically identifies the most important connections, separating the signal from the noise by employing relevance ranking specific to each query. Because it is built on Elasticsearch, Graph benefits from high scalability and near-real-time data availability, enabling answers that evolve as your data changes.

When data is added to Elasticsearch, the indexing process tracks and counts the values in each field of the document, updates global frequencies, and prepares the data for a wide range of queries. These statistics are used to perform relevance ranking for search and efficiently summarize the data in Elasticsearch aggregations. With Graph, the Elastic Stack uses these statistics in new ways — first to identify relationships within and across sets of documents, and then to prioritize the most relevant relationships for the given query.

In contrast, traditional graph techniques are based on a simple count of the frequency of a given relationship. This approach has the drawback that elements with the most connections — the Shawshank Redemption in movie recommendation data or Starbucks in credit card purchase data — are returned as the most important. With Graph in Elasticsearch, relevance is calculated by correlating the significance of each relationship in comparison to global averages, bringing back the important results, and avoiding frequent connections.

"Graph is an excellent example of the limitless possibilities within our products and how we strive to make it as easy as possible for our users to leverage the power of the Elastic Stack," said Shay Banon, Co-Founder and CTO of Elastic. "I am proud to see our company continue to innovate, and I can’t wait to see the new ways in which our customers will adopt Graph to solve really challenging problems and use cases."

Share this

The Latest

May 23, 2019

The first word in APM technology is "Application" ... yet for mobile, apps are entirely different. As the mobile app ecosystem is evolving and expanding from pure entertainment to more utilitarian uses, there's a rising need for the next generation of APM technology to stay ahead of the issues that can cause apps to fail ...

May 22, 2019

For application performance monitoring (APM), many in IT tend to focus a significant amount of their time on the tool that performs the analysis. Unfortunately for them, the battle is won or lost at the data access level. If you don’t have the right data, you can’t fix the problem correctly ...

May 21, 2019

Findings of the Digital Employee Experience survey from VMware show correlation between enabling employees with a positive digital experience (i.e., device choice/flexibility, seamless access to apps, remote work capabilities) and an organization's competitive position, revenue growth and employee sentiment ...

May 20, 2019

In today's competitive landscape, businesses must have the ability and process in place to face new challenges and find ways to successfully tackle them in a proactive manner. For years, this has been placed on the shoulders of DevOps teams within IT departments. But, as automation takes over manual intervention to increase speed and efficiency, these teams are facing what we know as IT digitization. How has this changed the way companies function over the years, and what do we have to look forward to in the coming years? ...

May 16, 2019

Although the vast majority of IT organizations have implemented a broad variety of systems and tools to modernize, simplify and streamline data center operations, many are still burdened by inefficiencies, security risks and performance gaps in their IT infrastructure as well as the excessive time it takes to manage legacy infrastructure, according to the State of IT Transformation, a report from Datrium ...

May 15, 2019

When it comes to network visibility, there are a lot of discussions about packet broker technology and the various features these solutions provide to network architects and IT managers. Packet brokers allow organizations to aggregate the data required for a variety of monitoring solutions including network performance monitoring and diagnostic (NPMD) platforms and unified threat management (UTM) appliances. But, when it comes to ensuring these solutions provide the insights required by NetOps and security teams, IT can spend an exorbitant amount of time dealing with issues around adds, moves and changes. This can have a dramatic impact on budgets and tool availability. Why does this happen? ...

May 14, 2019

Data may be pouring into enterprises but IT professionals still find most of it stuck in siloed departments and weeks away from being able to drive any valued action. Coupled with the ongoing concerns over security responsiveness, IT teams have to push aside other important performance-oriented data in order to ensure security data, at least, gets prominent attention. A new survey by Ivanti shows the disconnect between enterprise departments struggling to improve operations like automation while being challenged with a siloed structure and a data onslaught ...

May 13, 2019

A subtle, deliberate shift has occurred within the software industry which, at present, only the most innovative organizations have seized upon for competitive advantage. Although primarily driven by Artificial Intelligence (AI), this transformation strikes at the core of the most pervasive IT resources including cloud computing and predictive analytics ...

May 09, 2019

When asked who is mandated with developing and delivering their organization's digital competencies, 51% of respondents say their IT departments have a leadership role. The critical question is whether IT departments are prepared to take on a leadership role in which collaborating with other functions and disseminating knowledge and digital performance data are requirements ...

May 08, 2019

The Economist Intelligence Unit just released a new study commissioned by Riverbed that explores nine digital competencies that help organizations improve their digital performance and, ultimately, achieve their objectives. Here's a brief summary of 7 key research findings you'll find covered in detail in the report ...