AIOps Across 17 Vendors: What the Data Shows
September 17, 2020

Dennis Drogseth

Share this

One of the benefits of doing the EMA Radar Report: AIOps- A Guide for Investing in Innovation was getting data from all 17 vendors on critical areas ranging from deployment and adoption challenges, to cost and pricing, to architectural and functionality insights across everything from heuristics, to automation, and data assimilation.

Listen to EMA's Dennis Drogseth on the AI+ITOPS Podcast

Administration and Deployment

In the area of deployment and administration, EMA found that on average AIOps vendors indicated between 1-1.5 full-time employees (FTE) were required for ongoing administration in an enterprise with about 10,000 employees. This didn’t include initial deployment or any significant extension in breadth of coverage or functionality.

In 31 interviews, these estimates were generally borne out. Three vendors at the high end estimated between 2.5 and 3 FTEs, whereas the three vendors at the low end estimated between less than 0.5 FTEs.


The great majority of AIOps platforms have heuristics that can "learn" their environments dynamically, without added administrative intervention. On average, they can do this in a little more than one week for 5,000 managed entities.

EMA then asked vendors to weight their AI/ML heuristics on a scale from 0-2, with 2 being a featured heuristic value, 1 being present, and 0 being absent. The top 10 heuristics getting a 2 weighting were:

1. Correlators

2. Anomaly detection

3. Machine learning and baselining for event pattern recognition

4. Topology-based analytics

5. Prescriptive analytics

6. Predictive algorithms

7. Comparators

8. Streaming analytics

9. Optimization algorithms

10. Object-based modeling

Data Assimilation

On average, AIOps vendors could assimilate between 1 million and 10 million metrics within five minutes. When we asked about what data types were in play, we saw:

1. Events (performance related)

2. Time Series

3. Log files

4. Events/ Time Series (security related)

5. Transaction (application performance)

6. Configuration/topology

7. Unstructured data

8. Agent data (systems)

9. Byte code instrumentation

10. Comma delimited files /CSV files

Third-party toolset integration

Significantly, all 17 vendors have some level of third-party toolset integration out of the box, or in parallel, none claim to do "all their own monitoring." In fact, the average AIOps platform has supported integrations for more than 50 different third-party toolsets, with four vendors indicating 100 or more.

These integrations can have powerful political and practical advantages, easing stakeholder reluctance by eliminating the need to break away from their existing tools completely. Additional values include toolset consolidation as IT organizations begin to observe redundancies while also realizing which toolsets are most valuable.

The most common toolset integrations were application performance monitoring (APM) tools tied with CMDBs or extended configuration management systems. Service desk integration for trouble ticketing followed and third-party event management systems came in fourth. Automation integrations were also key, with IT process automation (runbook), and workflow across IT in the lead.

A few use-case views

We had three use-case scenarios. And for each use case we examined a number of factors ranging from domain reach, stakeholders supported, real-time data currency, and heuristics to enable not only awareness of anomalies, but predictive and prescriptive recommendations. Vendors were positioned separately on a per-use-case basis.

When we asked about the top benefits for incident, availability and performance management all vendors led with the following six items, which were also born out in deployment interviews:

■ Faster time to repair problems

■ Proactive ability to prevent problems

■ Improved OpEx efficiencies within IT

■ Less time spent writing rules

■ Real-time insights and historical trends on IT services

■ Reduction/consolidation, minimalization of tools

When we asked what changes each vendor could trace for change impact and capacity optimization, we got the following top five:

■ System configuration service impact analysis

■ Application release changes

■ Service impact analysis (in general)

■ Virtualized infrastructure service impact analysis

■ Containers and microservices service impact analysis

For business impact and IT-to-business alignment, we asked about relevant data sources and saw these as the top five:

■ Enterprise operations data

■ IT warehouse for advanced trending

■ Business application owner data

■ Executive dashboard

■ Security/audit compliance systems

To wrap up

This is just a taste of the data that emerged from our AIOps Radar research. The report contains considerably more detail, while still being a condensation of 105 data-rich slides.

Doing this has been an adventure for me, for EMA as a whole, and I believe for the vendors involved, as well. I do hope you can check out the report and see for yourself as to why.

Dennis Drogseth is VP at Enterprise Management Associates (EMA)
Share this

The Latest

April 15, 2021

A growing need for process automation as a result of the confluence of digital transformation initiatives with the remote/hybrid work policies brought on by the pandemic was uncovered by an independent survey of over 500 IT Operations, DevOps, and Site Reliability Engineering (SRE) professionals commissioned by Transposit for its inaugural State of DevOps Automation Report ...

April 14, 2021

As the Covid-19 pandemic forces a global reset of how we gather and work, 60% of organizations are looking forward to increased spending in 2021 to deploy new technologies, according to the 14th annual State of the Network global study of enterprise networking and security challenges released by VIAVI Solutions ...

April 13, 2021

Complexity breaks correlation. Intelligence brings cohesion. This simple principle is what makes real-time asset intelligence a must-have for AIOps that is meant to diffuse complexity. To further create a context for the user, it is critical to understand service dependencies and correlate alerts across the stack to resolve incidents ...

April 12, 2021

We're all familiar with the process of QA within the software development cycle. Developers build a product and send it to QA engineers, who test and bless it before pushing it into the world. After release, a different team of SREs with their own toolset then monitor for issues and bugs. Now, a new level of customer expectations for speed and reliability have pushed businesses further toward delivering rapid product iterations and innovations to keep up with customer demands. This leaves little time to run the traditional development process ...

April 08, 2021

On Wednesday January 27, 2021, Microsoft Office 365 experienced an outage affected a number of its services with a prolonged outage affecting Exchange Online. Despite Microsoft indicating that it was just Exchange Online affected during this outage, some monitoring tools detected that Azure Active Directory and dependent services like SharePoint and OneDrive were also affected at the time. The outage information indicated a rollout and rollback but we wouldn't expect to see such a widescale outage and slowdown just affecting some of the schema unless everything had to be taken offline ...

April 07, 2021

Application availability depends on the availability of other elements in a system, for example, network, server, operating system and so on, which support the application. Concentrating solely on the availability of any one block will not produce optimum availability of the application for the end user ...

April 06, 2021

A hybrid work environment will persist after the pandemic recedes, with over 80% stating that they expect over a quarter of workers to remain remote, and over two-thirds desiring flexibility between on-premises and remote deployments according to the 2021 State of the WAN report released by Aryaka ...

April 05, 2021

As vaccinations rise and businesses plan for a post-covid future, more than 80% of knowledge workers in the US would like their long-term work environment to include some element of remote work ...

April 01, 2021

With so many of us working from home, IT leaders and executives are now more than ever interested in ensuring that the cloud services their team relies on are available. But instead of accessing popular business-critical applications such as Salesforce, G Suite, Office 365, Microsoft 365, and so on through the company's data center, employees now get these services directly from the Internet. Experience and productivity at each location vary by internet, ISP, gateway, proxy, etc. ...

March 31, 2021

Integration challenges continue to be a major roadblock for digital transformation initiatives, according to MuleSoft’s 2021 Connectivity Benchmark Report. As digital initiatives accelerate, integration has emerged as a critical factor in determining the success and speed of digital transformation across industries ...