In 2018, the concept of MLOps emerged in professional circles and at conferences dedicated to AI, quickly establishing itself in the industry and now evolving as an independent field. In the future, MLOps could become one of the most sought-after areas in IT. What is it and how is it applied? Let's explore.
What is MLOps
MLOps (the integration of machine learning technologies and processes with approaches to implementing developed models in business processes) is a new way of collaboration between business representatives, scientists, mathematicians, machine learning specialists, and IT engineers in creating artificial intelligence systems.
In other words, it is a method of transforming machine learning techniques and technologies into useful tools for addressing business challenges.
It's important to understand that the productivity chain begins long before model development. Its first step is defining the business task, hypothesizing the value that can be extracted from data, and the business idea for its application.
The term MLOps originated as an analogy to the DevOps concept applied to machine learning models and technologies. DevOps is an approach to software development that allows for faster implementation of individual changes while maintaining flexibility and reliability through a range of techniques, including continuous development, dividing functions into several independent microservices, automated testing and deployment of individual changes, global performance monitoring, and operational response systems for identified failures.
DevOps defined the software lifecycle, and within the community of specialists arose the idea of using the same methodology for big data. DataOps is an attempt to adapt and extend the methodology considering the specifics of storing, transferring, and processing large datasets across diverse and interrelated platforms.
With the emergence of a certain critical mass of machine learning models integrated into business processes, a strong similarity between the life cycle of machine learning models and the software life cycle has been observed. The only difference is that the algorithms of the models are created using tools and methods of machine learning. Therefore, the idea naturally arose to apply and adapt well-known software development approaches for machine learning models. Thus, the life cycle of machine learning models can be divided into the following key stages:
- defining the business idea;
- training the model;
- testing and deploying the model into the business process;
- operating the model.
When the need arises to modify or retrain the model on new data during operation, the cycle starts anew — the model is refined, tested, and a new version is deployed.
Digression. Why retrain and not re-train? The term 'retraining the model' has a dual interpretation: among specialists, it refers to a defect in the model, where the model predicts well, actually replicating the forecasted parameter on the training dataset, but performs much worse on external data samples. Naturally, such a model is considered defective, as this flaw prevents its application.
In this life cycle, the use of DevOps tools seems logical: automated testing, deployment, and monitoring, organizing the calculation of models as separate microservices. However, there are also several features that hinder the direct application of these tools without additional ML infrastructure.
How to make models work and generate profit.
As an example, we will demonstrate the application of the MLOps approach by taking the classic task of automating a bank's (or any other) product support chat. Typically, the business process for chat support looks like this: the client enters a message with a question in the chat and receives a response from a specialist within a predefined dialogue tree. The task of automating such a chat is usually solved using expert-defined sets of rules, which are very labor-intensive to develop and maintain. The effectiveness of such automation may range from 20% to 30%, depending on the complexity of the task. Naturally, the idea arises that it may be more beneficial to implement an artificial intelligence module — a model developed using machine learning, which:
- is capable of processing a greater number of requests without operator involvement (depending on the topic, in some cases, effectiveness can reach 70% to 80%);
- better adapts to non-standard phrasing in the dialogue — it can determine intent and the actual desire of the user from a vaguely formulated request;
- can determine when the model's response is adequate and when there are doubts about the 'awareness' of that response, necessitating an additional clarifying question or handing off to an operator;
- can be retrained automatically (instead of a team of developers constantly adapting and correcting response scripts, the model is retrained by a Data Science specialist using relevant machine learning libraries).
How to make such an advanced model work?
As with any other task, before developing such a module, it is necessary to define the business process and formally describe the specific task we will be solving using the machine learning method. This is where the operationalization process begins, indicated by the abbreviation Ops.
The next step involves the Data Science specialist, in collaboration with the data engineer, verifying the availability and sufficiency of data and the business hypothesis regarding the viability of the business idea. This includes developing a model prototype and assessing its actual effectiveness. Only after confirmation from the business can the transition from model development to its integration into systems that execute specific business processes begin. Comprehensive planning for implementation, along with a deep understanding at each stage of how the model will be utilized and what economic impact it will yield, is a fundamental aspect of implementing MLOps approaches within the company's technological landscape.
With the advancement of AI technologies, the number and diversity of tasks that can be solved through machine learning are increasing exponentially. Each of these business processes represents a cost saving for the company due to the automation of labor-intensive roles (such as call centers, document verification, and sorting), expansion of the client base through the addition of new attractive and convenient features, efficient use of funds through optimal resource allocation, and much more. Ultimately, every process is focused on creating value and, consequently, must generate a specific economic effect. Here, it is crucial to clearly articulate the business idea and estimate the projected profit from implementing the model within the overall value creation structure of the company. There are situations where the implementation of the model does not justify itself, and the time spent by machine learning specialists may cost significantly more than the position of an operator performing the task. This is why it is essential to identify such cases at the early stages of AI system development.
Thus, the profits from the model only begin to materialize when the business task was accurately defined during the MLOps process, priorities were established, and the model implementation process was articulated in the early stages of development.
New processes bring new challenges.
A comprehensive answer to the fundamental business question of how applicable ML models are for solving tasks, the overarching issue of trust in AI — is one of the key challenges in the development and implementation of MLOps approaches. Initially, businesses are skeptical about integrating machine learning into processes — it's hard to rely on models in areas where humans have typically worked. For businesses, programs seem like a 'black box,' the relevance of whose answers still needs to be proven. Additionally, in banking, telecommunications, and other sectors, there are strict requirements from government regulators. All systems and algorithms integrated into banking processes undergo audits. To resolve this issue and prove to businesses and regulators the justification and correctness of the answers provided by artificial intelligence, monitoring tools are implemented alongside the model. Furthermore, there is an independent validation procedure required for regulatory models that meets the requirements of the Central Bank. An independent expert group audits the results obtained from the model based on input data.
The second challenge is the assessment and consideration of model risks when implementing a machine learning model. If even a human cannot answer the question with absolute certainty whether that dress was white or blue, then artificial intelligence has the right to make mistakes as well. Additionally, it is important to consider that data can change over time, and models need to be retrained to yield sufficiently accurate results. To ensure that the business process does not suffer, it is necessary to manage model risks and monitor model performance, regularly retraining it on new data.
However, after the initial stage of distrust, a reverse effect begins to manifest. The more models are successfully implemented into processes, the greater the business appetite for using artificial intelligence grows — new tasks continually emerge that can be addressed using machine learning methods. Each task initiates a whole process requiring specific competencies:
- data engineers prepare and process the data;
- data scientists apply machine learning tools and develop the model;
- IT integrates the model into the system;
- The ML engineer determines how to correctly embed this model into the process and which IT tools to use depending on the requirements for model application, taking into account the flow of requests, response time, etc.
- The ML architect designs how the software product can be physically implemented in an industrial system.
The entire cycle requires a large number of highly qualified specialists. At a certain point in the development and penetration of ML models into business processes, it becomes expensive and inefficient to linearly scale the number of specialists in proportion to the growth in the number of tasks. Therefore, the question of automating the MLOps process arises — defining several standard classes of machine learning tasks, developing typical pipelines for data processing and model retraining. In an ideal scenario for solving such tasks, professionals are needed who equally possess competencies at the intersection of Big Data, Data Science, DevOps, and IT. Thus, the biggest problem in the Data Science industry and the greatest challenge in organizing MLOps processes is the lack of such competence in the existing labor market. Specialists who meet these requirements are currently rare in the job market and are valued highly.
Regarding competencies
Theoretically, all MLOps tasks can be solved with classic DevOps tools without resorting to specialized role model extensions. However, as we noted above, the data scientist must be not only a mathematician and data analytics specialist but also the guru of the entire pipeline — responsible for developing architecture, programming models in multiple languages depending on architecture, preparing data showcases, and deploying the application itself. However, creating the technological framework implemented in the end-to-end MLOps process takes up to 80% of the labor costs, meaning that a qualified mathematician, who is a high-quality Data Scientist, will only spend 20% of their time on their specialty. Therefore, delineating the roles of specialists involved in the process of implementing machine learning models becomes critically necessary.
The extent to which roles should be clearly defined depends on the scale of the enterprise. It's one thing when a startup has a single specialist, a jack-of-all-trades handling engineering, architecture, and DevOps tasks. It's quite another when a large company has all model development processes concentrated among a few high-level Data Science specialists, while programmers or database specialists—who are more common and less costly in the job market—can take on much of the routine tasks.
Thus, the boundary in choosing specialists for facilitating the MLOps process and how the operationalization of developed models is organized directly affects the speed and quality of the models being developed, the performance of the team, and the work environment within it.
What our team has accomplished so far
We have recently begun to build the structure for competencies and MLOps processes. But already, our projects for managing the model lifecycle and for deploying models as a service are in the MVP testing stage.
We have also identified the optimal competency structure and organizational framework for interaction among all participants in the process for a large enterprise. Agile teams have been organized to address the needs of all business clients, and a process for collaboration with project teams for creating platforms and infrastructure—which is the foundation of the emerging MLOps structure—has been established.
Questions for the future
MLOps is an evolving field that is currently experiencing a shortage of expertise and will gain momentum in the future. For now, it is best to build on the developments and practices established by DevOps. The main goal of MLOps is to use ML models more effectively for solving business tasks. However, several questions arise:
- How can we reduce the time to deploy models into production?
- How can we minimize bureaucratic friction between teams of different competencies and enhance collaboration?
- How can we track models, manage versions, and organize effective monitoring?
- How can we create a truly cyclical lifecycle for modern ML models?
- How can we standardize the machine learning process?
The answers to these questions will largely determine how quickly MLOps will fully realize its potential.
Source: habr.com
