Monte Carlo Launches Incident IQ
July 14, 2021
Share this

Monte Carlo released Incident IQ, a new suite of capabilities that help data engineers better pinpoint, address, and resolve data downtime at scale through the Monte Carlo Data Observability Platform.

Incident IQ automatically generates rich insights about critical data issues through root cause analysis, giving teams unprecedented visibility into the end-to-end health and trust of their data beyond the scope of traditional data quality solutions.

On average, companies lose over $15 million per year on bad data, with data engineers spending upwards of 40 percent - or 120 hours per week - of their time tackling broken data pipelines. In the same way that New Relic, DataDog, and other Application Performance Management (APM) solutions ensure reliable software and keep application downtime at bay, Data Observability solves the costly problem of data downtime, in other words, periods of time when data is missing, inaccurate, or otherwise unreliable.

To help companies eliminate data downtime, Monte Carlo built Incident IQ, the first end-to-end solution that conducts root cause analysis for data issues at each stage of the pipeline, from ingestion in the data warehouse or lake to analytics in your business intelligence dashboards. Incident IQ automatically generates historical insights about your data to identify patterns in query logs, trigger investigative follow-on query results, and monitor upstream dependency changes to pin-point exactly what caused the issue to occur, reducing the amount of data incidents by 90 percent at each stage of the pipeline.

Developed after reviewing thousands of real data incidents from our customers, Incident IQ gives data engineers access to insights about their code, their data, and their operational environment that allows them to quickly and collaboratively get to the root cause of data problems -- all in a single UI.

With Incident IQ, everything related to the data issue is captured in an elegant timeline with easy commenting, documentation, and collaboration features to create rich post-mortems. This level of detail, common in software engineering and DevOps tooling, helps data teams learn from past incidents and determine where to allocate future investment. Additionally, Incident IQ makes it easy to create and share high-level incident reporting with CTOs and CDOs, fostering greater data trust and ownership across the company.

Core capabilities of Incident IQ include:

- Central UI that connects the dots between correlated causes of data incidents, and surfaces a historical collection of data incidents for quick comparison.

- Access to example queries that pull sample data, as well as rich query logs, historical incidents, and quick links to Monte Carlo’s Lineage and Catalog features, making it easy to identify, root cause, and fix data issues all from the same interface.

- Automatic insights based on the statistical correlation between table fields in anomalous records (for instance, Incident IQ can surface if an increase in order_id null values correlates with a specific order source).

- Automatic, end-to-end lineage that maps impacted downstream BI dashboards to the furthest upstream tables, helping teams narrow the focus of root cause investigations.

- Automatic runbooks and workflows to make the incident resolution and triaging process easy, fast, and collaborative between data engineers and analysts.

- Comprehensive query logs that reveal periodic vs. ad hoc queries, changes in query patterns, and more.

“As companies become more data driven, it’s fundamental that organizations not only understand the health of their data, but also have the data observability necessary to trust it from end to end,” said Lior Gavish, CTO, Monte Carlo. “As the data stack fragments to incorporate new tools, it’s becoming increasingly difficult to identify when data pipelines break and take action to fix them. With Incident IQ, data practitioners and leaders alike can holistically understand and respond to issues faster, before they become a serious problem for the business. We believe these features will help customers eliminate hundreds of hours of data downtime and thousands to millions of dollars in savings each month, as well as enable data platform teams to scale with rich post-mortems that track performance and facilitate greater learning.”

Monte Carlo is a Data Observability partner for the FinTech, e-commerce, media, B2B software, and retail industries, counting data teams at Fox, Vimeo, ThredUp, and PagerDuty among their customers.

In February 2021, the company announced their $25M Series B funding, led by Redpoint Ventures and GGV Capital, and was named one of the 2021 Enterprise Tech 30.

Share this

The Latest

July 28, 2021

Business leaders are in the unique position of having immediate access to huge amounts of data in today's smartphone and laptop-dominated world. They are also under pressure to make data-driven decisions and mobile business intelligence can one of the most valuable decision making tools in their arsenal ...

July 27, 2021

Unlike some AI initiatives, AIOps doesn't always necessitate the use of a data scientist, so don't let hiring expenses put your AIOps initiatives on hold. It is always nice to have IT team members with AI skills, but this becomes less critical as more intelligent solutions come into prominence that offer AIOps features out of the box, a readily deployable option for IT ...

July 26, 2021

AIOps is rapidly becoming a de-facto option for enterprises' IT strategies, with nearly immeasurable benefits to be provided. However, AIOps is still a relatively new discipline and misconceptions surrounding the technology's capabilities and uses have caused bottlenecks and roadblocks in its widespread adoption. So, what should organizations expect from AIOps? How can organizations that want to digitally transform their IT pursue AIOps for maximum benefit? ...

July 22, 2021

In response to the global pandemic, companies have given their workforce the tools they need to work remote. And research shows it has increased their engagement and productivity. But these gains are on the brink of being wiped out. According to a new study from Citrix Systems, Inc., employees feel they've been given too many tools and not enough efficient ways to execute. And it's hindering their ability to get things done ...

July 21, 2021

The third installment of Aptum's four-part Cloud Impact Study, A Bright Forecast on Cloud, presents data showing the benefits organizations gain from cloud computing, as well as mistakes to avoid during migration. As organizations migrate workloads to different cloud platforms, they often run into unexpected challenges due to a lack of proactive planning. Here are a few key findings from Part 3 of the Cloud Impact Study ...

July 20, 2021

Currently, (and most likely well into the future) the overwhelming majority of organizations still need to monitor and maintain enterprise applications. Moreover, where these are complex systems developed, debugged and refined over years, often decades, around a business's core processes, there can also be very strong practical arguments for viewing them as classics. They can offer a valuable legacy, one best left where it is, doing what it does, how it always has done ...

July 19, 2021

Anti-patterns involve realizing a problem and implementing a non-optimal solution that is broadly embraced as the go-to method for solving that problem. This solution sounds good in theory, but for one reason or another it is not the best means of solving the problem. Anti-patterns are common across IT as well, especially around application monitoring and observability. One that is particularly prevalent is in response to the increasing complexity of cloud-native infrastructure and applications ...

July 15, 2021

The introduction of the latest technology — such as AI and machine learning — can be seen as a way for organizations to accelerate growth, increase efficiency, and improve customer service. However, the truth is that the technology alone will do little to deliver on these business outcomes. AI for IT operations (AIOps) is one area where the application of technology, if not matched with organizational maturity readiness, will fail to deliver all the promised benefits ...

July 14, 2021

SREs that fail to deliver customer value run the risk of being stuck in an operational toil rut. Conversely, businesses failing to recognize the bi-modal nature and importance of SRE activities run the risk of losing talented employees and their competitive edge ...

July 13, 2021

As part of digital transformation initiatives, IT teams are quickly adopting AIOps solutions to accommodate a new multifaceted infrastructure. However, there are still several roadblocks IT leaders must overcome when adopting AIOps — namely, understanding how to showcase ROI and changing their team's cultural mindset around adopting a new strategy ...