Machine Learning (ML) technologies have evolved at an incredible pace over the past few years, and yet multiple studies suggest that most ML projects fail in the real world. Despite the availability of high-quality technologies, there still exist challenges in using these technologies to create and deliver complete solutions, which can be attributed to several factors.
The main causes of failure can be grouped into four categories:
■ failure to frame the ML problem from a business perspective
■ failure to build a team with the right talent, in the right roles
■ failure to select the right data and ML infrastructure
■ failure to properly manage the AI solution in production
Let's dive into each of these areas in more detail.
1. Failure to frame the ML problem from a business perspective
Firstly, failure to frame the ML problem from a business challenge or opportunity perspective is a common issue. Many companies approach ML with unrealistic expectations, or they are simply following the trend to implement ML, without a clear business need or opportunity. This can lead to wasted resources and disappointment when the project fails to deliver the expected results. To avoid this, it is crucial for the ML problem to be clearly defined, with close collaboration between business leaders and experienced engineers. This ensures that both the business and technical aspects of the problem are considered and that the solution is tailored to the specific needs of the company.
2. Failure to build a team with the right talent, in the right roles
The second factor of AI project failure is the failure to put the right talent in the right roles on the team. When a company has a problem to solve, it is important to get the right talent to work on it. However, this can be a challenging task, as it requires the ability to recognize genuine expertise and skill, which in turn requires the presence of that talent within the organization. To address this, companies should invest in training and development programs to develop talent with the necessary skills within the organization. They should also look for external experts who can bring in specialized knowledge.
3. Failure to select the right data and ML infrastructure
The third cause of failure is not having the right data and ML infrastructure. Even with the right talent, a project can still fail if the appropriate data and infrastructure are not in place. Data is the backbone of any ML project, and without quality data, the model cannot deliver accurate results. Infrastructure is also crucial for the success of the project. This includes hardware and software used for data processing, storage, and model training. Without the right infrastructure, the project will be unable to scale and deliver the expected results.
4. Failure to properly manage the AI solution in production
The final major reason for failure is the failure to properly maintain the AI solution in production. This is the final step of any ML project, and it is where many companies stumble. Once the model has been trained and tested, it needs to be integrated into the current business systems, and work at the scale of the business. This requires talent with yet another expert skillset, and it can be challenging to manage the model in production. This includes monitoring the model, updating it as necessary, and addressing any issues that arise.
Essential Capabilities for ML Infrastructure
These four horsemen of AI project failure are common issues that companies face when implementing ML solutions.
The first two issues are not so much technical as organizational. Clearly, when starting such initiatives, the company's leadership should closely watch for any discrepancies in the organizational structure and processes.
The last two factors that often contribute to an ML project’s failure can be attributed to MLOps and can be resolved by an appropriate implementation.
MLOps, or Machine Learning Operations, is a highly fragmented space, and it can be overwhelming to keep up with all the frameworks and platforms available. But there are certain capabilities that are essential for any real-world ML infrastructure solution. One of the most important is scalability. Organizations and use cases often need to be able to scale up and down, to adjust to the usage patterns of end users. Without scalability, an ML solution may be unable to meet the demands of a production environment.
Another important capability is reproducibility. The platform should be able to successfully reproduce an experiment from a month ago, which requires versioning of everything: data, ML code, pipeline configuration, infrastructure code, experiments, and more. This capability ensures that the results are consistent and can be trusted.
Security and observability are also key capabilities for an ML platform. Properly configured security ensures that the data and models are protected from unauthorized access. In its turn, observability ensures that the platform has full visibility into everything, including data, models, infrastructure, code, and users. This allows for a better understanding and management of the solution.
In conclusion, while ML technologies have advanced rapidly in recent years, the implementation of ML solutions in real-world environments remains a challenge. To overcome challenges, companies should clearly define the ML problem through collaboration between business leaders and experienced engineers. They should invest in training and development programs to build the necessary skills within the organization and seek external experts to bring in specialized knowledge.
Additionally, organizations should focus on building a robust ML infrastructure that includes key capabilities, including scalability, reproducibility, security, and observability.
With a well-defined problem, and the right talent, data, and infrastructure in place, companies can increase their chances of success in implementing ML solutions in the real world.
Developers need a tool that can be portable and vendor agnostic, given the advent of microservices. It may be clear an issue is occurring; what may not be clear is if it's part of a distributed system or the app itself. Enter OpenTelemetry, commonly referred to as OTel, an open-source framework that provides a standardized way of collecting and exporting telemetry data (logs, metrics, and traces) from cloud-native software ...
As SLOs grow in popularity their usage is becoming more mature. For example, 82% of respondents intend to increase their use of SLOs, and 96% have mapped SLOs directly to their business operations or already have a plan to, according to The State of Service Level Objectives 2023 from Nobl9 ...
Observability has matured beyond its early adopter position and is now foundational for modern enterprises to achieve full visibility into today's complex technology environments, according to The State of Observability 2023, a report released by Splunk in collaboration with Enterprise Strategy Group ...
Before network engineers even begin the automation process, they tend to start with preconceived notions that oftentimes, if acted upon, can hinder the process. To prevent that from happening, it's important to identify and dispel a few common misconceptions currently out there and how networking teams can overcome them. So, let's address the three most common network automation myths ...
Many IT organizations apply AI/ML and AIOps technology across domains, correlating insights from the various layers of IT infrastructure and operations. However, Enterprise Management Associates (EMA) has observed significant interest in applying these AI technologies narrowly to network management, according to a new research report, titled AI-Driven Networks: Leveling Up Network Management with AI/ML and AIOps ...
When it comes to system outages, AIOps solutions with the right foundation can help reduce the blame game so the right teams can spend valuable time restoring the impacted services rather than improving their MTTI score (mean time to innocence). In fact, much of today's innovation around ChatGPT-style algorithms can be used to significantly improve the triage process and user experience ...
Gartner identified the top 10 data and analytics (D&A) trends for 2023 that can guide D&A leaders to create new sources of value by anticipating change and transforming extreme uncertainty into new business opportunities ...
The only way for companies to stay competitive is to modernize applications, yet there's no denying that bringing apps into the modern era can be challenging ... Let's look at a few ways to modernize applications and consider what new obstacles and opportunities 2023 presents ...
As online penetration grows, retailers' profits are shrinking — with the cost of serving customers anytime, anywhere, at any speed not bringing in enough topline growth to best monetize even existing investments in technology, systems, infrastructure, and people, let alone new investments, according to Digital-First Retail: Turning Profit Destruction into Customer and Shareholder Value, a new report from AlixPartners and World Retail Congress ...