AIOps Across 17 Vendors: What the Data Shows
September 17, 2020

Dennis Drogseth

Share this

One of the benefits of doing the EMA Radar Report: AIOps- A Guide for Investing in Innovation was getting data from all 17 vendors on critical areas ranging from deployment and adoption challenges, to cost and pricing, to architectural and functionality insights across everything from heuristics, to automation, and data assimilation.

Listen to EMA's Dennis Drogseth on the AI+ITOPS Podcast

Administration and Deployment

In the area of deployment and administration, EMA found that on average AIOps vendors indicated between 1-1.5 full-time employees (FTE) were required for ongoing administration in an enterprise with about 10,000 employees. This didn’t include initial deployment or any significant extension in breadth of coverage or functionality.

In 31 interviews, these estimates were generally borne out. Three vendors at the high end estimated between 2.5 and 3 FTEs, whereas the three vendors at the low end estimated between less than 0.5 FTEs.


The great majority of AIOps platforms have heuristics that can "learn" their environments dynamically, without added administrative intervention. On average, they can do this in a little more than one week for 5,000 managed entities.

EMA then asked vendors to weight their AI/ML heuristics on a scale from 0-2, with 2 being a featured heuristic value, 1 being present, and 0 being absent. The top 10 heuristics getting a 2 weighting were:

1. Correlators

2. Anomaly detection

3. Machine learning and baselining for event pattern recognition

4. Topology-based analytics

5. Prescriptive analytics

6. Predictive algorithms

7. Comparators

8. Streaming analytics

9. Optimization algorithms

10. Object-based modeling

Data Assimilation

On average, AIOps vendors could assimilate between 1 million and 10 million metrics within five minutes. When we asked about what data types were in play, we saw:

1. Events (performance related)

2. Time Series

3. Log files

4. Events/ Time Series (security related)

5. Transaction (application performance)

6. Configuration/topology

7. Unstructured data

8. Agent data (systems)

9. Byte code instrumentation

10. Comma delimited files /CSV files

Third-party toolset integration

Significantly, all 17 vendors have some level of third-party toolset integration out of the box, or in parallel, none claim to do "all their own monitoring." In fact, the average AIOps platform has supported integrations for more than 50 different third-party toolsets, with four vendors indicating 100 or more.

These integrations can have powerful political and practical advantages, easing stakeholder reluctance by eliminating the need to break away from their existing tools completely. Additional values include toolset consolidation as IT organizations begin to observe redundancies while also realizing which toolsets are most valuable.

The most common toolset integrations were application performance monitoring (APM) tools tied with CMDBs or extended configuration management systems. Service desk integration for trouble ticketing followed and third-party event management systems came in fourth. Automation integrations were also key, with IT process automation (runbook), and workflow across IT in the lead.

A few use-case views

We had three use-case scenarios. And for each use case we examined a number of factors ranging from domain reach, stakeholders supported, real-time data currency, and heuristics to enable not only awareness of anomalies, but predictive and prescriptive recommendations. Vendors were positioned separately on a per-use-case basis.

When we asked about the top benefits for incident, availability and performance management all vendors led with the following six items, which were also born out in deployment interviews:

■ Faster time to repair problems

■ Proactive ability to prevent problems

■ Improved OpEx efficiencies within IT

■ Less time spent writing rules

■ Real-time insights and historical trends on IT services

■ Reduction/consolidation, minimalization of tools

When we asked what changes each vendor could trace for change impact and capacity optimization, we got the following top five:

■ System configuration service impact analysis

■ Application release changes

■ Service impact analysis (in general)

■ Virtualized infrastructure service impact analysis

■ Containers and microservices service impact analysis

For business impact and IT-to-business alignment, we asked about relevant data sources and saw these as the top five:

■ Enterprise operations data

■ IT warehouse for advanced trending

■ Business application owner data

■ Executive dashboard

■ Security/audit compliance systems

To wrap up

This is just a taste of the data that emerged from our AIOps Radar research. The report contains considerably more detail, while still being a condensation of 105 data-rich slides.

Doing this has been an adventure for me, for EMA as a whole, and I believe for the vendors involved, as well. I do hope you can check out the report and see for yourself as to why.

Dennis Drogseth is VP at Enterprise Management Associates (EMA)
Share this

The Latest

October 28, 2020

The "APM" solutions we've come to love over the last 2 decades can't handle Serverless Functions or deliver the same performance and operational details that they deliver for other architectural constructs — including App Servers, Frameworks, Cloud, even Containers. And the reason is that they're methodologies for collecting performance data simply won't operate with the same characteristics as it would in persistent code ...

October 27, 2020

I asked myself this important question: Can APM tools Manage Serverless Workloads? And the answer is "No, not really." It is true that every monitoring solution in the world claims support for monitoring serverless platforms (at least one of them). What I mean by my answer is that the "APM" solutions we've come to love over the last 2 decades can't handle Serverless Functions or deliver the same performance and operational details that they deliver for other architectural constructs ...

October 26, 2020

In Episode 11, Andrew Tunall, GM, New Relic Serverless & Emerging Cloud Services, joins the AI+ITOPS Podcast to discuss the challenges and advantages of distributed tracing ...

October 22, 2020

IT teams critically require better visibility into the network driven by a number of factors, including tremendous disruption from the COVID-19 pandemic, relentless technological advances, remote working reaching an all-time high and the expanding security threatscape, according to State of the Network 2020, a study conducted by VIAVI Solutions ...

October 21, 2020

Mobile commerce offers several benefits for retailers. But all this potential can only be fully realized if retailers can manage the associated challenges that mobile commerce introduces. Anyone involved in the development, operation or troubleshooting of a mobile shopping app needs to be aware of the three following technical obstacles and plan accordingly ...

October 20, 2020

Although cost control/expense management remains top of mind, organizations are realizing the necessity of technology solutions to enable them to steer the business during these turbulent times, according to IDG's CIO Pandemic Business Impact Study ...

October 19, 2020

The COVID-19 pandemic has compressed six years of modernization projects into 6 months. According to a recent report, IT leaders have accelerated projects aimed at increasing productivity and business agility, improving application performance and end-user experience, and driving additional revenue through existing channels ...

October 15, 2020

There is no doubt that automation has become the key aspect of modern IT management. The end-user computing market is no exception. With a large and complex technology stack and a huge number of applications, EUC specialists need to handle an ever-increasing number of changes at an ever-increasing rate. Many IT organizations are starting to realize that they can no longer control the flow of changes. It is time to think about how to facilitate change ...

October 14, 2020

Starting this September, the lifespan of an SSL/TLS certificate has been limited to 398 days, a reduction from the previous maximum certificate lifetime of 825 days. With this change, everyone needs to more carefully monitor SSL certificate expiration and server characteristics ...

October 13, 2020

Nearly 6 in 10 responding organizations have accelerated their digital transformations due to the COVID-19 pandemic, according to The IBM Institute for Business Value study COVID-19 and the Future of Business ...