The State of AI Development and Operations in 2019
October 01, 2019

Mark Coleman
Dotscience

Share this

The use of AI is booming across the modern enterprise. In fact, according to Gartner's 2019 CIO Survey, the number of enterprises implementing AI grew 270% in the past four years and tripled in the past year. However, many enterprises will be unable to realize the full potential of their initiatives until they find more efficient means of tracking data, code, models and metrics across the entire AI lifecycle.

To better understand the AI maturity of businesses, Dotscience conducted a survey of 500 industry professionals in its inaugural State of Development and Operations of AI Applications 2019 report.

Research findings indicate that although enterprises are dedicating significant time and resources towards their AI deployments, many data science and ML teams don't have the adequate tools needed to properly collaborate on, build and deploy AI models efficiently.

AI Goes Mainstream

AI has moved beyond the experimentation stage and is now seen as a critical and impactful function for many businesses. Enterprises are becoming increasingly reliant on AI for its ability to deliver greater operational efficiency, streamline complex business processes, and support cost control and profit potential. This is evidenced by the survey results, which indicate that the top three drivers of AI adoption are efficiency gains (47%), growth initiatives (46%) and digital transformation (44%). Furthermore, over 88% of respondents at organizations where AI is in production indicated that AI has either been impactful or highly impactful to their company's competitive advantage. The exponential growth of AI's value and influence is also reflected in the large investments organizations are making in AI. Nearly a third of respondents (30%) are budgeting between 1 and 10 million dollars for AI tools, platforms and services.

Unfortunately, it's not all rainbows and sunshine in the world of enterprise AI. The study also found that despite this level of financial commitment, data science and ML teams continue to experience issues, including duplicating their work (33%), rewriting models after team members leave (28%), justifying the value of their projects to the wider business (27%), and slow and unpredictable AI projects (25%).

Manual Tools and Processes

Despite providing an impactful competitive advantage for enterprises, AI deployments today are largely slow and inefficient. The manual tools and processes primarily in use to operationalize ML and AI don't support the scaling and governance demanded of many AI initiatives.

The top two ways that ML engineers and data scientists collaborate with each other are by using a manually updated shared spreadsheet for metrics (44%) and sitting in the same office and working closely together (38%). These methods of collaboration ultimately disrupt efficiency and limit AI's potential. Machine learning has many moving parts, and teams require version control for their training and test data, their code and their environment, as well as metrics and hyperparameters in order to collaborate efficiently. Survey findings show that over 35% of organizations don't use any version control for their training and test data. However, of those who don't currently have any version control, over 60% would like to.

These limitations are compounded by the fact that nearly 90% of respondents either manually track model provenance (a complete record of all the steps taken to create an AI model) or do not track provenance at all. And of those that manually track model provenance, more than half (52%) do their tracking in a spreadsheet or wiki, a cumbersome and error-prone approach.

Challenges in Scaling AI Initiatives

Despite significant investment in AI, many companies are still struggling to stabilize and scale their AI initiatives. The manual tools and processes being used by many for AI model development are insufficient and do not support the required scaling and governance.

While 63% of businesses reported they are spending between $500,000 and $10 million on their AI efforts, 61% of respondents continue to experience a variety of operational challenges. This is evidenced by the fact that 64% of organizations deploying AI said that it is taking between 7 to 18 months to get their AI workloads from idea into production, illustrating the slow, unpredictable nature of AI projects today. Meanwhile, for nearly another 20%, the anticipated timeline is 19+ months to production.

DevOps Like It's 1999

The challenges faced by data science and ML teams today are reminiscent of the same challenges facing software engineers in the late 1990s. Then came DevOps, which transformed the way software engineers deliver applications by making it possible to collaborate, test and deliver software continuously.

With ML and AI projects today, collaboration is even more challenging when compared to basic software engineering. Normal software development tools focus on versions or commits of code whereas ML has many more moving parts. ML teams require version control for both training and test data, their code and their environment, as well as metrics and hyperparameters for each training run.

While ML and AI are understood as powerful technologies with the potential to reinvent the global economy, operationalizing AI still remains a major hurdle for many organizations. To simplify, accelerate and control every stage of the AI model lifecycle, the same DevOps-like principles of collaboration, fast feedback and continuous delivery should be applied to AI. Only then can enterprises realize the full potential of their AI deployments across the organization.

Mark Coleman is VP of Product and Marketing at Dotscience
Share this

The Latest

June 29, 2022

When it comes to AIOps predictions, there's no question of AI's value in predictive intelligence and faster problem resolution for IT teams. In fact, Gartner has reported that there is no future for IT Operations without AIOps. So, where is AIOps headed in five years? Here's what the vendors and thought leaders in the AIOps space had to share ...

June 27, 2022

A new study by OpsRamp on the state of the Managed Service Providers (MSP) market concludes that MSPs face a market of bountiful opportunities but must prepare for this growth by embracing complex technologies like hybrid cloud management, root cause analysis and automation ...

June 27, 2022

Hybrid work adoption and the accelerated pace of digital transformation are driving an increasing need for automation and site reliability engineering (SRE) practices, according to new research. In a new survey almost half of respondents (48.2%) said automation is a way to decrease Mean Time to Resolution/Repair (MTTR) and improve service management ...

June 23, 2022

Digital businesses don't invest in monitoring for monitoring's sake. They do it to make the business run better. Every dollar spent on observability — every hour your team spends using monitoring tools or responding to what they reveal — should tie back directly to business outcomes: conversions, revenues, brand equity. If they don't? You might be missing the forest for the trees ...

June 22, 2022

Every day, companies are missing customer experience (CX) "red flags" because they don't have the tools to observe CX processes or metrics. Even basic errors or defects in automated customer interactions are left undetected for days, weeks or months, leading to widespread customer dissatisfaction. In fact, poor CX and digital technology investments are costing enterprises billions of dollars in lost potential revenue ...

June 21, 2022

Organizations are moving to microservices and cloud native architectures at an increasing pace. The primary incentive for these transformation projects is typically to increase the agility and velocity of software release and product innovation. These dynamic systems, however, are far more complex to manage and monitor, and they generate far higher data volumes ...

June 16, 2022

Global IT teams adapted to remote work in 2021, resolving employee tickets 23% faster than the year before as overall resolution time for IT tickets went down by 7 hours, according to the Freshservice Service Management Benchmark Report from Freshworks ...

June 15, 2022

Once upon a time data lived in the data center. Now data lives everywhere. All this signals the need for a new approach to data management, a next-gen solution ...

June 14, 2022

Findings from the 2022 State of Edge Messaging Report from Ably and Coleman Parkes Research show that most organizations (65%) that have built edge messaging capabilities in house have experienced an outage or significant downtime in the last 12-18 months. Most of the current in-house real-time messaging services aren't cutting it ...

June 13, 2022
Today's users want a complete digital experience when dealing with a software product or system. They are not content with the page load speeds or features alone but want the software to perform optimally in an omnichannel environment comprising multiple platforms, browsers, devices, and networks. This calls into question the role of load testing services to check whether the given software under testing can perform optimally when subjected to peak load ...