The State of AI Development and Operations in 2019
October 01, 2019

Mark Coleman
Dotscience

Share this

The use of AI is booming across the modern enterprise. In fact, according to Gartner's 2019 CIO Survey, the number of enterprises implementing AI grew 270% in the past four years and tripled in the past year. However, many enterprises will be unable to realize the full potential of their initiatives until they find more efficient means of tracking data, code, models and metrics across the entire AI lifecycle.

To better understand the AI maturity of businesses, Dotscience conducted a survey of 500 industry professionals in its inaugural State of Development and Operations of AI Applications 2019 report.

Research findings indicate that although enterprises are dedicating significant time and resources towards their AI deployments, many data science and ML teams don't have the adequate tools needed to properly collaborate on, build and deploy AI models efficiently.

AI Goes Mainstream

AI has moved beyond the experimentation stage and is now seen as a critical and impactful function for many businesses. Enterprises are becoming increasingly reliant on AI for its ability to deliver greater operational efficiency, streamline complex business processes, and support cost control and profit potential. This is evidenced by the survey results, which indicate that the top three drivers of AI adoption are efficiency gains (47%), growth initiatives (46%) and digital transformation (44%). Furthermore, over 88% of respondents at organizations where AI is in production indicated that AI has either been impactful or highly impactful to their company's competitive advantage. The exponential growth of AI's value and influence is also reflected in the large investments organizations are making in AI. Nearly a third of respondents (30%) are budgeting between 1 and 10 million dollars for AI tools, platforms and services.

Unfortunately, it's not all rainbows and sunshine in the world of enterprise AI. The study also found that despite this level of financial commitment, data science and ML teams continue to experience issues, including duplicating their work (33%), rewriting models after team members leave (28%), justifying the value of their projects to the wider business (27%), and slow and unpredictable AI projects (25%).

Manual Tools and Processes

Despite providing an impactful competitive advantage for enterprises, AI deployments today are largely slow and inefficient. The manual tools and processes primarily in use to operationalize ML and AI don't support the scaling and governance demanded of many AI initiatives.

The top two ways that ML engineers and data scientists collaborate with each other are by using a manually updated shared spreadsheet for metrics (44%) and sitting in the same office and working closely together (38%). These methods of collaboration ultimately disrupt efficiency and limit AI's potential. Machine learning has many moving parts, and teams require version control for their training and test data, their code and their environment, as well as metrics and hyperparameters in order to collaborate efficiently. Survey findings show that over 35% of organizations don't use any version control for their training and test data. However, of those who don't currently have any version control, over 60% would like to.

These limitations are compounded by the fact that nearly 90% of respondents either manually track model provenance (a complete record of all the steps taken to create an AI model) or do not track provenance at all. And of those that manually track model provenance, more than half (52%) do their tracking in a spreadsheet or wiki, a cumbersome and error-prone approach.

Challenges in Scaling AI Initiatives

Despite significant investment in AI, many companies are still struggling to stabilize and scale their AI initiatives. The manual tools and processes being used by many for AI model development are insufficient and do not support the required scaling and governance.

While 63% of businesses reported they are spending between $500,000 and $10 million on their AI efforts, 61% of respondents continue to experience a variety of operational challenges. This is evidenced by the fact that 64% of organizations deploying AI said that it is taking between 7 to 18 months to get their AI workloads from idea into production, illustrating the slow, unpredictable nature of AI projects today. Meanwhile, for nearly another 20%, the anticipated timeline is 19+ months to production.

DevOps Like It's 1999

The challenges faced by data science and ML teams today are reminiscent of the same challenges facing software engineers in the late 1990s. Then came DevOps, which transformed the way software engineers deliver applications by making it possible to collaborate, test and deliver software continuously.

With ML and AI projects today, collaboration is even more challenging when compared to basic software engineering. Normal software development tools focus on versions or commits of code whereas ML has many more moving parts. ML teams require version control for both training and test data, their code and their environment, as well as metrics and hyperparameters for each training run.

While ML and AI are understood as powerful technologies with the potential to reinvent the global economy, operationalizing AI still remains a major hurdle for many organizations. To simplify, accelerate and control every stage of the AI model lifecycle, the same DevOps-like principles of collaboration, fast feedback and continuous delivery should be applied to AI. Only then can enterprises realize the full potential of their AI deployments across the organization.

Mark Coleman is VP of Product and Marketing at Dotscience
Share this

The Latest

October 20, 2021

Over three quarters (79%) of database professionals are now using either a paid-for or in-house monitoring tool, according to a new survey from Redgate Software ...

October 19, 2021

Gartner announced the top strategic technology trends that organizations need to explore in 2022. With CEOs and Boards striving to find growth through direct digital connections with customers, CIOs' priorities must reflect the same business imperatives, which run through each of Gartner's top strategic tech trends for 2022 ...

October 18, 2021

Distributed tracing has been growing in popularity as a primary tool for investigating performance issues in microservices systems. Our recent DevOps Pulse survey shows a 38% increase year-over-year in organizations' tracing use. Furthermore, 64% of those respondents who are not yet using tracing indicated plans to adopt it in the next two years ...

October 14, 2021

Businesses are embracing artificial intelligence (AI) technologies to improve network performance and security, according to a new State of AIOps Study, conducted by ZK Research and Masergy ...

October 13, 2021

What may have appeared to be a stopgap solution in the spring of 2020 is now clearly our new workplace reality: It's impossible to walk back so many of the developments in workflow we've seen since then. The question is no longer when we'll all get back to the office, but how the companies that are lagging in their technological ability to facilitate remote work can catch up ...

October 12, 2021

The pandemic accelerated organizations' journey to the cloud to enable agile, on-demand, flexible access to resources, helping them align with a digital business's dynamic needs. We heard from many of our customers at the start of lockdown last year, saying they had to shift to a remote work environment, seemingly overnight, and this effort was heavily cloud-reliant. However, blindly forging ahead can backfire ...

October 07, 2021

SmartBear recently released the results of its 2021 State of Software Quality | Testing survey. I doubt you'll be surprised to hear that a "lack of time" was reported as the number one challenge to doing more testing, especially as release frequencies continue to increase. However, it was disheartening to see that a lack of time was also the number one response when we asked people to identify the biggest blocker to professional development ...

October 06, 2021

The role of the CIO is evolving with an increased focus on unlocking customer connections through service innovation, according to the 2021 Global CIO Survey. The study reveals the shift in the role of the CIO with the majority of CIO respondents stating innovation, operational efficiency, and customer experience as their top priorities ...

October 05, 2021

The perception of IT support has dramatically improved thanks to the successful response of service desks to the pandemic, lockdowns and working from home, according to new research from the Service Desk Institute (SDI), sponsored by Sunrise Software ...

October 04, 2021

Is your company trying to use artificial intelligence (AI) for business purposes like sales and marketing, finance or customer experience? If not, why not? If so, has it struggled to start AI projects and get them to work effectively? ...