The State of AI Development and Operations in 2019
October 01, 2019

Mark Coleman
Dotscience

Share this

The use of AI is booming across the modern enterprise. In fact, according to Gartner's 2019 CIO Survey, the number of enterprises implementing AI grew 270% in the past four years and tripled in the past year. However, many enterprises will be unable to realize the full potential of their initiatives until they find more efficient means of tracking data, code, models and metrics across the entire AI lifecycle.

To better understand the AI maturity of businesses, Dotscience conducted a survey of 500 industry professionals in its inaugural State of Development and Operations of AI Applications 2019 report.

Research findings indicate that although enterprises are dedicating significant time and resources towards their AI deployments, many data science and ML teams don't have the adequate tools needed to properly collaborate on, build and deploy AI models efficiently.

AI Goes Mainstream

AI has moved beyond the experimentation stage and is now seen as a critical and impactful function for many businesses. Enterprises are becoming increasingly reliant on AI for its ability to deliver greater operational efficiency, streamline complex business processes, and support cost control and profit potential. This is evidenced by the survey results, which indicate that the top three drivers of AI adoption are efficiency gains (47%), growth initiatives (46%) and digital transformation (44%). Furthermore, over 88% of respondents at organizations where AI is in production indicated that AI has either been impactful or highly impactful to their company's competitive advantage. The exponential growth of AI's value and influence is also reflected in the large investments organizations are making in AI. Nearly a third of respondents (30%) are budgeting between 1 and 10 million dollars for AI tools, platforms and services.

Unfortunately, it's not all rainbows and sunshine in the world of enterprise AI. The study also found that despite this level of financial commitment, data science and ML teams continue to experience issues, including duplicating their work (33%), rewriting models after team members leave (28%), justifying the value of their projects to the wider business (27%), and slow and unpredictable AI projects (25%).

Manual Tools and Processes

Despite providing an impactful competitive advantage for enterprises, AI deployments today are largely slow and inefficient. The manual tools and processes primarily in use to operationalize ML and AI don't support the scaling and governance demanded of many AI initiatives.

The top two ways that ML engineers and data scientists collaborate with each other are by using a manually updated shared spreadsheet for metrics (44%) and sitting in the same office and working closely together (38%). These methods of collaboration ultimately disrupt efficiency and limit AI's potential. Machine learning has many moving parts, and teams require version control for their training and test data, their code and their environment, as well as metrics and hyperparameters in order to collaborate efficiently. Survey findings show that over 35% of organizations don't use any version control for their training and test data. However, of those who don't currently have any version control, over 60% would like to.

These limitations are compounded by the fact that nearly 90% of respondents either manually track model provenance (a complete record of all the steps taken to create an AI model) or do not track provenance at all. And of those that manually track model provenance, more than half (52%) do their tracking in a spreadsheet or wiki, a cumbersome and error-prone approach.

Challenges in Scaling AI Initiatives

Despite significant investment in AI, many companies are still struggling to stabilize and scale their AI initiatives. The manual tools and processes being used by many for AI model development are insufficient and do not support the required scaling and governance.

While 63% of businesses reported they are spending between $500,000 and $10 million on their AI efforts, 61% of respondents continue to experience a variety of operational challenges. This is evidenced by the fact that 64% of organizations deploying AI said that it is taking between 7 to 18 months to get their AI workloads from idea into production, illustrating the slow, unpredictable nature of AI projects today. Meanwhile, for nearly another 20%, the anticipated timeline is 19+ months to production.

DevOps Like It's 1999

The challenges faced by data science and ML teams today are reminiscent of the same challenges facing software engineers in the late 1990s. Then came DevOps, which transformed the way software engineers deliver applications by making it possible to collaborate, test and deliver software continuously.

With ML and AI projects today, collaboration is even more challenging when compared to basic software engineering. Normal software development tools focus on versions or commits of code whereas ML has many more moving parts. ML teams require version control for both training and test data, their code and their environment, as well as metrics and hyperparameters for each training run.

While ML and AI are understood as powerful technologies with the potential to reinvent the global economy, operationalizing AI still remains a major hurdle for many organizations. To simplify, accelerate and control every stage of the AI model lifecycle, the same DevOps-like principles of collaboration, fast feedback and continuous delivery should be applied to AI. Only then can enterprises realize the full potential of their AI deployments across the organization.

Mark Coleman is VP of Product and Marketing at Dotscience
Share this

The Latest

June 25, 2020

I've had the opportunity to work with a number of organizations embarking on their AIOps journey. I always advise them to start by evaluating their needs and the possibilities AIOps can bring to them through five different levels of AIOps maturity. This is a strategic approach that allows enterprises to achieve complete automation for long-term success ...

June 24, 2020

Sumo Logic recently commissioned an independent market research study to understand the industry momentum behind continuous intelligence — and the necessity for digital organizations to embrace a cloud-native, real-time continuous intelligence platform to support the speed and agility of business for faster decision-making, optimizing security, driving new innovation and delivering world-class customer experiences. Some of the key findings include ...

June 23, 2020

When it comes to viruses, it's typically those of the computer/digital variety that IT is concerned about. But with the ongoing pandemic, IT operations teams are on the hook to maintain business functions in the midst of rapid and massive change. One of the biggest challenges for businesses is the shift to remote work at scale. Ensuring that they can continue to provide products and services — and satisfy their customers — against this backdrop is challenging for many ...

June 22, 2020

Teams tasked with developing and delivering software are under pressure to balance the business imperative for speed with high customer expectations for quality. In the course of trying to achieve this balance, engineering organizations rely on a variety of tools, techniques and processes. The 2020 State of Software Quality report provides a snapshot of the key challenges organizations encounter when it comes to delivering quality software at speed, as well as how they are approaching these hurdles. This blog introduces its key findings ...

June 18, 2020

For IT teams, run-the-business, commodity areas such as employee help desks, device support and communication platforms are regularly placed in the crosshairs for cost takeout, but these areas are also highly visible to employees. Organizations can improve employee satisfaction and business performance by building unified functions that are measured by employee experience rather than price. This approach will ultimately fund transformation, as well as increase productivity and innovation ...

June 17, 2020

In the agile DevOps framework, there is a vital piece missing; something that previous approaches to application development did well, but has since fallen by the wayside. That is, the post-delivery portion of the toolchain. Without continuous cloud optimization, the CI/CD toolchain still produces massive inefficiencies and overspend ...

June 16, 2020

The COVID-19 pandemic has exponentially accelerated digital transformation projects. To better understand where IT professionals are turning for help, we analyzed the online behaviors of IT decision-makers. Our research found an increase in demand for resources related to APM, microservices and dependence on cloud services ...

June 15, 2020

The rush to the public cloud has now slowed as organizations realized that it is not a "one size fits all" solution. The main issue is the lack of deep visibility into the performance of applications provided by the host. Our own research has recently revealed that 32% of public cloud resources are currently under-utilized, and without proper direction and guidance, this will remain the case ...

June 11, 2020

The global shift to working from home (WFH) enforced by COVID-19 stay-at-home orders has had a massive impact on everyone's working lives, not just in the way they remotely interact with their teams and IT systems, but also in how they spend their working days. With both governments and businesses committed to slowly opening up offices, it's increasingly clear that a high prevalence of remote work will continue throughout 2020 and beyond. This situation begets important questions ...

June 10, 2020
In recent years, with the emergence of newer technologies ranging from the cloud to machine learning, IT modernization has evolved from a replacement of end-of-life infrastructure to an enabler of innovation and business value. It is a complex process that can take months or even years, but a recent survey shows that the effort begins to deliver measurable results almost as soon as an organization executes the first steps on its roadmap ...