Skip to main content

The State of AI Development and Operations in 2019

Mark Coleman
Dotscience

The use of AI is booming across the modern enterprise. In fact, according to Gartner's 2019 CIO Survey, the number of enterprises implementing AI grew 270% in the past four years and tripled in the past year. However, many enterprises will be unable to realize the full potential of their initiatives until they find more efficient means of tracking data, code, models and metrics across the entire AI lifecycle.

To better understand the AI maturity of businesses, Dotscience conducted a survey of 500 industry professionals in its inaugural State of Development and Operations of AI Applications 2019 report.

Research findings indicate that although enterprises are dedicating significant time and resources towards their AI deployments, many data science and ML teams don't have the adequate tools needed to properly collaborate on, build and deploy AI models efficiently.

AI Goes Mainstream

AI has moved beyond the experimentation stage and is now seen as a critical and impactful function for many businesses. Enterprises are becoming increasingly reliant on AI for its ability to deliver greater operational efficiency, streamline complex business processes, and support cost control and profit potential. This is evidenced by the survey results, which indicate that the top three drivers of AI adoption are efficiency gains (47%), growth initiatives (46%) and digital transformation (44%). Furthermore, over 88% of respondents at organizations where AI is in production indicated that AI has either been impactful or highly impactful to their company's competitive advantage. The exponential growth of AI's value and influence is also reflected in the large investments organizations are making in AI. Nearly a third of respondents (30%) are budgeting between 1 and 10 million dollars for AI tools, platforms and services.

Unfortunately, it's not all rainbows and sunshine in the world of enterprise AI. The study also found that despite this level of financial commitment, data science and ML teams continue to experience issues, including duplicating their work (33%), rewriting models after team members leave (28%), justifying the value of their projects to the wider business (27%), and slow and unpredictable AI projects (25%).

Manual Tools and Processes

Despite providing an impactful competitive advantage for enterprises, AI deployments today are largely slow and inefficient. The manual tools and processes primarily in use to operationalize ML and AI don't support the scaling and governance demanded of many AI initiatives.

The top two ways that ML engineers and data scientists collaborate with each other are by using a manually updated shared spreadsheet for metrics (44%) and sitting in the same office and working closely together (38%). These methods of collaboration ultimately disrupt efficiency and limit AI's potential. Machine learning has many moving parts, and teams require version control for their training and test data, their code and their environment, as well as metrics and hyperparameters in order to collaborate efficiently. Survey findings show that over 35% of organizations don't use any version control for their training and test data. However, of those who don't currently have any version control, over 60% would like to.

These limitations are compounded by the fact that nearly 90% of respondents either manually track model provenance (a complete record of all the steps taken to create an AI model) or do not track provenance at all. And of those that manually track model provenance, more than half (52%) do their tracking in a spreadsheet or wiki, a cumbersome and error-prone approach.

Challenges in Scaling AI Initiatives

Despite significant investment in AI, many companies are still struggling to stabilize and scale their AI initiatives. The manual tools and processes being used by many for AI model development are insufficient and do not support the required scaling and governance.

While 63% of businesses reported they are spending between $500,000 and $10 million on their AI efforts, 61% of respondents continue to experience a variety of operational challenges. This is evidenced by the fact that 64% of organizations deploying AI said that it is taking between 7 to 18 months to get their AI workloads from idea into production, illustrating the slow, unpredictable nature of AI projects today. Meanwhile, for nearly another 20%, the anticipated timeline is 19+ months to production.

DevOps Like It's 1999

The challenges faced by data science and ML teams today are reminiscent of the same challenges facing software engineers in the late 1990s. Then came DevOps, which transformed the way software engineers deliver applications by making it possible to collaborate, test and deliver software continuously.

With ML and AI projects today, collaboration is even more challenging when compared to basic software engineering. Normal software development tools focus on versions or commits of code whereas ML has many more moving parts. ML teams require version control for both training and test data, their code and their environment, as well as metrics and hyperparameters for each training run.

While ML and AI are understood as powerful technologies with the potential to reinvent the global economy, operationalizing AI still remains a major hurdle for many organizations. To simplify, accelerate and control every stage of the AI model lifecycle, the same DevOps-like principles of collaboration, fast feedback and continuous delivery should be applied to AI. Only then can enterprises realize the full potential of their AI deployments across the organization.

Mark Coleman is VP of Product and Marketing at Dotscience

Hot Topics

The Latest

Industry experts offer predictions on how AI will evolve and impact technology and business in 2025. Part 3 covers AI's impact on employees and their roles ...

Industry experts offer predictions on how AI will evolve and impact technology and business in 2025. Part 2 covers the challenges presented by AI, as well as solutions to those problems ...

In the final part of APMdigest's 2025 Predictions Series, industry experts offer predictions on how AI will evolve and impact technology and business in 2025 ...

E-commerce is set to skyrocket with a 9% rise over the next few years ... To thrive in this competitive environment, retailers must identify digital resilience as their top priority. In a world where savvy shoppers expect 24/7 access to online deals and experiences, any unexpected downtime to digital services can lead to significant financial losses, damage to brand reputation, abandoned carts with designer shoes, and additional issues ...

Efficiency is a highly-desirable objective in business ... We're seeing this scenario play out in enterprises around the world as they continue to struggle with infrastructures and remote work models with an eye toward operational efficiencies. In contrast to that goal, a recent Broadcom survey of global IT and network professionals found widespread adoption of these strategies is making the network more complex and hampering observability, leading to uptime, performance and security issues. Let's look more closely at these challenges ...

Image
Broadcom

The 2025 Catchpoint SRE Report dives into the forces transforming the SRE landscape, exploring both the challenges and opportunities ahead. Let's break down the key findings and what they mean for SRE professionals and the businesses relying on them ...

Image
Catchpoint

The pressure on IT teams has never been greater. As data environments grow increasingly complex, resource shortages are emerging as a major obstacle for IT leaders striving to meet the demands of modern infrastructure management ... According to DataStrike's newly released 2025 Data Infrastructure Survey Report, more than half (54%) of IT leaders cite resource limitations as a top challenge, highlighting a growing trend toward outsourcing as a solution ...

Image
Datastrike

Gartner revealed its top strategic predictions for 2025 and beyond. Gartner's top predictions explore how generative AI (GenAI) is affecting areas where most would assume only humans can have lasting impact ...

The adoption of artificial intelligence (AI) is accelerating across the telecoms industry, with 88% of fixed broadband service providers now investigating or trialing AI automation to enhance their fixed broadband services, according to new research from Incognito Software Systems and Omdia ...

 

AWS is a cloud-based computing platform known for its reliability, scalability, and flexibility. However, as helpful as its comprehensive infrastructure is, disparate elements and numerous siloed components make it difficult for admins to visualize the cloud performance in detail. It requires meticulous monitoring techniques and deep visibility to understand cloud performance and analyze operational efficiency in detail to ensure seamless cloud operations ...

The State of AI Development and Operations in 2019

Mark Coleman
Dotscience

The use of AI is booming across the modern enterprise. In fact, according to Gartner's 2019 CIO Survey, the number of enterprises implementing AI grew 270% in the past four years and tripled in the past year. However, many enterprises will be unable to realize the full potential of their initiatives until they find more efficient means of tracking data, code, models and metrics across the entire AI lifecycle.

To better understand the AI maturity of businesses, Dotscience conducted a survey of 500 industry professionals in its inaugural State of Development and Operations of AI Applications 2019 report.

Research findings indicate that although enterprises are dedicating significant time and resources towards their AI deployments, many data science and ML teams don't have the adequate tools needed to properly collaborate on, build and deploy AI models efficiently.

AI Goes Mainstream

AI has moved beyond the experimentation stage and is now seen as a critical and impactful function for many businesses. Enterprises are becoming increasingly reliant on AI for its ability to deliver greater operational efficiency, streamline complex business processes, and support cost control and profit potential. This is evidenced by the survey results, which indicate that the top three drivers of AI adoption are efficiency gains (47%), growth initiatives (46%) and digital transformation (44%). Furthermore, over 88% of respondents at organizations where AI is in production indicated that AI has either been impactful or highly impactful to their company's competitive advantage. The exponential growth of AI's value and influence is also reflected in the large investments organizations are making in AI. Nearly a third of respondents (30%) are budgeting between 1 and 10 million dollars for AI tools, platforms and services.

Unfortunately, it's not all rainbows and sunshine in the world of enterprise AI. The study also found that despite this level of financial commitment, data science and ML teams continue to experience issues, including duplicating their work (33%), rewriting models after team members leave (28%), justifying the value of their projects to the wider business (27%), and slow and unpredictable AI projects (25%).

Manual Tools and Processes

Despite providing an impactful competitive advantage for enterprises, AI deployments today are largely slow and inefficient. The manual tools and processes primarily in use to operationalize ML and AI don't support the scaling and governance demanded of many AI initiatives.

The top two ways that ML engineers and data scientists collaborate with each other are by using a manually updated shared spreadsheet for metrics (44%) and sitting in the same office and working closely together (38%). These methods of collaboration ultimately disrupt efficiency and limit AI's potential. Machine learning has many moving parts, and teams require version control for their training and test data, their code and their environment, as well as metrics and hyperparameters in order to collaborate efficiently. Survey findings show that over 35% of organizations don't use any version control for their training and test data. However, of those who don't currently have any version control, over 60% would like to.

These limitations are compounded by the fact that nearly 90% of respondents either manually track model provenance (a complete record of all the steps taken to create an AI model) or do not track provenance at all. And of those that manually track model provenance, more than half (52%) do their tracking in a spreadsheet or wiki, a cumbersome and error-prone approach.

Challenges in Scaling AI Initiatives

Despite significant investment in AI, many companies are still struggling to stabilize and scale their AI initiatives. The manual tools and processes being used by many for AI model development are insufficient and do not support the required scaling and governance.

While 63% of businesses reported they are spending between $500,000 and $10 million on their AI efforts, 61% of respondents continue to experience a variety of operational challenges. This is evidenced by the fact that 64% of organizations deploying AI said that it is taking between 7 to 18 months to get their AI workloads from idea into production, illustrating the slow, unpredictable nature of AI projects today. Meanwhile, for nearly another 20%, the anticipated timeline is 19+ months to production.

DevOps Like It's 1999

The challenges faced by data science and ML teams today are reminiscent of the same challenges facing software engineers in the late 1990s. Then came DevOps, which transformed the way software engineers deliver applications by making it possible to collaborate, test and deliver software continuously.

With ML and AI projects today, collaboration is even more challenging when compared to basic software engineering. Normal software development tools focus on versions or commits of code whereas ML has many more moving parts. ML teams require version control for both training and test data, their code and their environment, as well as metrics and hyperparameters for each training run.

While ML and AI are understood as powerful technologies with the potential to reinvent the global economy, operationalizing AI still remains a major hurdle for many organizations. To simplify, accelerate and control every stage of the AI model lifecycle, the same DevOps-like principles of collaboration, fast feedback and continuous delivery should be applied to AI. Only then can enterprises realize the full potential of their AI deployments across the organization.

Mark Coleman is VP of Product and Marketing at Dotscience

Hot Topics

The Latest

Industry experts offer predictions on how AI will evolve and impact technology and business in 2025. Part 3 covers AI's impact on employees and their roles ...

Industry experts offer predictions on how AI will evolve and impact technology and business in 2025. Part 2 covers the challenges presented by AI, as well as solutions to those problems ...

In the final part of APMdigest's 2025 Predictions Series, industry experts offer predictions on how AI will evolve and impact technology and business in 2025 ...

E-commerce is set to skyrocket with a 9% rise over the next few years ... To thrive in this competitive environment, retailers must identify digital resilience as their top priority. In a world where savvy shoppers expect 24/7 access to online deals and experiences, any unexpected downtime to digital services can lead to significant financial losses, damage to brand reputation, abandoned carts with designer shoes, and additional issues ...

Efficiency is a highly-desirable objective in business ... We're seeing this scenario play out in enterprises around the world as they continue to struggle with infrastructures and remote work models with an eye toward operational efficiencies. In contrast to that goal, a recent Broadcom survey of global IT and network professionals found widespread adoption of these strategies is making the network more complex and hampering observability, leading to uptime, performance and security issues. Let's look more closely at these challenges ...

Image
Broadcom

The 2025 Catchpoint SRE Report dives into the forces transforming the SRE landscape, exploring both the challenges and opportunities ahead. Let's break down the key findings and what they mean for SRE professionals and the businesses relying on them ...

Image
Catchpoint

The pressure on IT teams has never been greater. As data environments grow increasingly complex, resource shortages are emerging as a major obstacle for IT leaders striving to meet the demands of modern infrastructure management ... According to DataStrike's newly released 2025 Data Infrastructure Survey Report, more than half (54%) of IT leaders cite resource limitations as a top challenge, highlighting a growing trend toward outsourcing as a solution ...

Image
Datastrike

Gartner revealed its top strategic predictions for 2025 and beyond. Gartner's top predictions explore how generative AI (GenAI) is affecting areas where most would assume only humans can have lasting impact ...

The adoption of artificial intelligence (AI) is accelerating across the telecoms industry, with 88% of fixed broadband service providers now investigating or trialing AI automation to enhance their fixed broadband services, according to new research from Incognito Software Systems and Omdia ...

 

AWS is a cloud-based computing platform known for its reliability, scalability, and flexibility. However, as helpful as its comprehensive infrastructure is, disparate elements and numerous siloed components make it difficult for admins to visualize the cloud performance in detail. It requires meticulous monitoring techniques and deep visibility to understand cloud performance and analyze operational efficiency in detail to ensure seamless cloud operations ...