AIOps Across 17 Vendors: What the Data Shows
September 17, 2020

Dennis Drogseth

Share this

One of the benefits of doing the EMA Radar Report: AIOps- A Guide for Investing in Innovation was getting data from all 17 vendors on critical areas ranging from deployment and adoption challenges, to cost and pricing, to architectural and functionality insights across everything from heuristics, to automation, and data assimilation.

Listen to EMA's Dennis Drogseth on the AI+ITOPS Podcast

Administration and Deployment

In the area of deployment and administration, EMA found that on average AIOps vendors indicated between 1-1.5 full-time employees (FTE) were required for ongoing administration in an enterprise with about 10,000 employees. This didn’t include initial deployment or any significant extension in breadth of coverage or functionality.

In 31 interviews, these estimates were generally borne out. Three vendors at the high end estimated between 2.5 and 3 FTEs, whereas the three vendors at the low end estimated between less than 0.5 FTEs.


The great majority of AIOps platforms have heuristics that can "learn" their environments dynamically, without added administrative intervention. On average, they can do this in a little more than one week for 5,000 managed entities.

EMA then asked vendors to weight their AI/ML heuristics on a scale from 0-2, with 2 being a featured heuristic value, 1 being present, and 0 being absent. The top 10 heuristics getting a 2 weighting were:

1. Correlators

2. Anomaly detection

3. Machine learning and baselining for event pattern recognition

4. Topology-based analytics

5. Prescriptive analytics

6. Predictive algorithms

7. Comparators

8. Streaming analytics

9. Optimization algorithms

10. Object-based modeling

Data Assimilation

On average, AIOps vendors could assimilate between 1 million and 10 million metrics within five minutes. When we asked about what data types were in play, we saw:

1. Events (performance related)

2. Time Series

3. Log files

4. Events/ Time Series (security related)

5. Transaction (application performance)

6. Configuration/topology

7. Unstructured data

8. Agent data (systems)

9. Byte code instrumentation

10. Comma delimited files /CSV files

Third-party toolset integration

Significantly, all 17 vendors have some level of third-party toolset integration out of the box, or in parallel, none claim to do "all their own monitoring." In fact, the average AIOps platform has supported integrations for more than 50 different third-party toolsets, with four vendors indicating 100 or more.

These integrations can have powerful political and practical advantages, easing stakeholder reluctance by eliminating the need to break away from their existing tools completely. Additional values include toolset consolidation as IT organizations begin to observe redundancies while also realizing which toolsets are most valuable.

The most common toolset integrations were application performance monitoring (APM) tools tied with CMDBs or extended configuration management systems. Service desk integration for trouble ticketing followed and third-party event management systems came in fourth. Automation integrations were also key, with IT process automation (runbook), and workflow across IT in the lead.

A few use-case views

We had three use-case scenarios. And for each use case we examined a number of factors ranging from domain reach, stakeholders supported, real-time data currency, and heuristics to enable not only awareness of anomalies, but predictive and prescriptive recommendations. Vendors were positioned separately on a per-use-case basis.

When we asked about the top benefits for incident, availability and performance management all vendors led with the following six items, which were also born out in deployment interviews:

■ Faster time to repair problems

■ Proactive ability to prevent problems

■ Improved OpEx efficiencies within IT

■ Less time spent writing rules

■ Real-time insights and historical trends on IT services

■ Reduction/consolidation, minimalization of tools

When we asked what changes each vendor could trace for change impact and capacity optimization, we got the following top five:

■ System configuration service impact analysis

■ Application release changes

■ Service impact analysis (in general)

■ Virtualized infrastructure service impact analysis

■ Containers and microservices service impact analysis

For business impact and IT-to-business alignment, we asked about relevant data sources and saw these as the top five:

■ Enterprise operations data

■ IT warehouse for advanced trending

■ Business application owner data

■ Executive dashboard

■ Security/audit compliance systems

To wrap up

This is just a taste of the data that emerged from our AIOps Radar research. The report contains considerably more detail, while still being a condensation of 105 data-rich slides.

Doing this has been an adventure for me, for EMA as a whole, and I believe for the vendors involved, as well. I do hope you can check out the report and see for yourself as to why.

Dennis Drogseth is VP at Enterprise Management Associates (EMA)
Share this

The Latest

June 24, 2021
E-commerce metrics give insights on how the business is doing and operating an e-commerce platform without tracking the metrics is akin to driving a car with eyes closed. To sustain and be successful, e-commerce businesses should understand their performance and compare their progress over time. The metrics can be harnessed to derive meaningful insights on the store’s performance, the average value of items sold at any given time, and the total sales clocked in a day, week, and month, among others. To help you decide the retail e-commerce metrics to monitor, we have compiled this list ...
June 23, 2021

More than half (61%) of respondents reported that their teams are practicing observability, an 8% increase from 2020, signaling that overall adoption is on the rise, according to a 2021 survey from Honeycomb. However, the majority of respondents indicated their teams are at the earliest stages of observability maturity ...

June 22, 2021

Your employees aren't coming back to the office, at least not in the traditional sense. The pandemic shifted almost all industries into remote work. And according to the results of Ivanti's Everywhere Workplace survey, they're not interested in going back to the way things once were ...

June 21, 2021

Respondents to an OpsRamp survey are moving forward with digital transformation, but many are re-evaluating the number and type of tools they're using. There are three main takeaways from the survey ...

June 17, 2021

More and more mainframe decision makers are becoming aware that the traditional way of handling mainframe operations will soon fall by the wayside. The ever-growing demand for newer, faster digital services has placed increased pressure on data centers to keep up as new applications come online, the volume of data handled continually increases, and workloads become increasingly unpredictable. In a recent Forrester Consulting AIOps survey, commissioned by BMC, the majority of respondents cited that they spend too much time reacting to incidents and not enough time finding ways to prevent them ...

June 16, 2021

In the age of digital transformation, enterprises are migrating to open source software (OSS) in droves to streamline operations and improve customer and employee experiences. However, to unlock the deluge of OSS benefits, it's not enough for organizations to simply implement the software. They must take the necessary steps to build an intentional OSS strategy rooted in ongoing third-party support and training ...

June 15, 2021

In Part 1 of this series, we explored the top pain points associated with managing Internet-based WANs today. This second installment will focus on today's most prevalent SD-WAN deployment challenges specifically and what you can do to better manage modern WANs overall ...

June 14, 2021

Enterprise wide-area networks (WANs) have undergone an incredible transformation over the past several years. More often than not, they're hybrid, offering multiple connection paths between WANs. This provides many benefits but also makes them more challenging to manage than ever before. In Part 1 of this series, we'll explore the top pain points associated with Internet-based WANs ...

June 10, 2021

As we have seen during this digital transformation boom during the pandemic, technologists are managing more applications and data than ever before, which has led three quarters of technologists to be concerned with increased IT complexity. Even more significant, 89% admitted to feeling under immense pressure to keep up with the churn, according to the recent AppDynamics Agents of Transformation report. It's clear that the pandemic has pushed many technologists to their breaking point. To help tackle IT burnout, tech professionals need a "canary" to help them streamline and catch the anomalies before they cause any major performance issues ...

June 09, 2021

An hour-long outage this Tuesday ground the Internet to a halt after popular Content Delivery Network (CDN) provider, Fastly, experienced a glitch that downed Reddit, Spotify, HBO Max, Shopify, Stripe and the BBC, to name just a few of properties affected ...