The future has arrivedāartificial intelligence and machine learning technologies are already successfully used by your favorite stores, transportation companies, and even turkey farms.
If something exists, then there is information about it on the internet... an open project! See how Open Data Hub helps scale new technologies and avoids difficulties in their implementation.
Despite the advantages of artificial intelligence (AI) and machine learning (ML), organizations often face challenges in scaling these technologies. The main issues typically include:
- Information Sharing and Collaboration ā sharing information without unnecessary effort and collaborating in rapid iterations is practically impossible.
- Access to Data ā for each task, it needs to be built from scratch and manually, which takes a lot of time.
- On-Demand Access ā there is no way to obtain on-demand access to machine learning tools and platforms, as well as to computing infrastructure.
- Production ā models remain at the prototype stage and are not brought to production.
- Tracking and Explaining AI Results ā reproducibility, tracking, and explaining AI/ML results are challenging.
If these problems are left unaddressed, they negatively affect the speed, efficiency, and productivity of valuable data processing and analysis specialists. This leads to their frustration, disappointment in their work, and ultimately, business expectations regarding AI/ML go up in smoke.
The responsibility for solving these issues falls on IT specialists, who must provide data analysts ā correctly, something akin to the cloud. In broader terms, what is required is a platform that offers freedom of choice and has convenient, straightforward access. At the same time, it must be fast, easily reconfigurable, scalable on demand, and fault-tolerant. Building such a platform based on open-source technologies helps avoid vendor lock-in and maintain a long-term strategic advantage in terms of cost control.
A few years ago, something similar happened in application development, leading to the emergence of microservices, hybrid cloud environments, IT automation, and agile processes. To manage all of this, IT professionals began using containers, Kubernetes, and open hybrid clouds.
Now, this experience is being applied to respond to AI challenges. Therefore, IT specialists are creating platforms based on containers that enable the development of AI/ML services within agile processes, accelerate innovation, and are built with a focus on hybrid clouds.
We will start building such a platform with Red Hat OpenShift, our container Kubernetes platform for hybrid clouds, which has a rapidly growing ecosystem of software and hardware ML solutions (NVIDIA, H2O.ai, Starburst, PerceptiLabs, etc.). Some of Red Hat's clients, such as BMW Group, ExxonMobil, and others, have already deployed containerized ML toolchains and DevOps processes based on this platform and its ecosystem to bring their ML architectures into production mode and speed up the work of data analysts.
Another reason we launched the Open Data Hub project is to demonstrate an architecture based on several open-source projects and show how to implement the entire lifecycle of an ML solution on the OpenShift platform.
Open Data Hub Project
This is an open-source project developed within the corresponding development community that implements the complete cycle of operationsāfrom loading and transforming raw data to generating, training, and deploying the modelāwhen tackling AI/ML tasks using containers and Kubernetes on the OpenShift platform. This project can be viewed as a reference implementation, an example of how to build an open solution for 'AI/ML as a service' based on OpenShift and associated open-source tools like TensorFlow, JupyterHub, Spark, and others. Importantly, Red Hat itself uses this project to provide its AI/ML services. Additionally, OpenShift integrates with key software and hardware ML solutions from NVIDIA, Seldon, Starburst, and other vendors, making it easier to build and launch your own machine learning systems.
The Open Data Hub project targets the following user categories and use cases:
- A data analyst in need of a cloud-based solution for implementing ML projects with self-service features.
- A data analyst requiring the maximum selection from a wide variety of the latest open-source AI/ML tools and platforms.
- A data analyst who needs access to data sources while training models.
- A data analyst who needs access to computing resources (CPU, GPU, memory).
- A data analyst who requires the ability to collaborate and share work results with colleagues, receive feedback, and implement improvements through rapid iterations.
- A data analyst looking to engage with developers (and devops teams) to ensure that their ML models and work results are production-ready.
- A data engineer needing to provide data analysts with access to a variety of data sources while adhering to compliance and security requirements.
- An IT systems administrator/operator requiring the ability to effortlessly manage the lifecycle (installation, configuration, updates) of components and open-source technologies, along with the necessary management and quota tools.
The Open Data Hub project combines a range of open-source tools to enable the complete cycle of AI/ML operations. Jupyter Notebook serves as the main working tool for data analysts here. This toolkit is widely popular among data processing and analysis specialists, and Open Data Hub allows them to easily create and manage Jupyter Notebook workspaces using the integrated JupyterHub. In addition to creating and importing Jupyter notebooks, the Open Data Hub project also features a number of pre-built notebooks in the form of an AI Library.
This library is a collection of open-source machine learning components and solutions for common scenarios that facilitate rapid prototyping. JupyterHub is integrated with the OpenShift RBAC access model, allowing the use of existing OpenShift accounts and implementing single sign-on. Additionally, JupyterHub features a user-friendly interface called spawner, which enables users to easily configure the amount of computational resources (CPU cores, memory, GPU) for their selected Jupyter Notebook.
Once the data analyst creates and configures the notebook, all further management is handled by the Kubernetes scheduler, which is part of OpenShift. Users are left to conduct their experiments, save, and share their results. Moreover, advanced users can directly access the OpenShift CLI shell from Jupyter notebooks to leverage Kubernetes primitives like Jobs or OpenShift features such as Tekton or Knative. Alternatively, the user-friendly OpenShift GUI, known as the "OpenShift web console," can be utilized for this purpose.
Moving on to the next stage, Open Data Hub enables data pipeline management. This is achieved using a Ceph object, provided as an S3-compatible object data store. Apache Spark facilitates data streaming from external sources or the embedded Ceph S3 storage, as well as performing preliminary data transformations. Apache Kafka offers enhanced data pipeline management (allowing for multiple loads, as well as transformation, analysis, and storage operations).
So, the data analyst has accessed the data and built a model. Now there is a desire to share the results with colleagues or application developers, providing them with the model as a service. This requires an inference server, and Open Data Hub has such a server, named Seldon, which allows for model publication as a RESTful service.
At some point, there are several such models on the Seldon server, and the need arises to monitor how they are being used. For this purpose, Open Data Hub offers a collection of relevant metrics and a reporting engine based on widely used open-source monitoring tools Prometheus and Grafana. As a result, we obtain feedback for monitoring the usage of AI models, particularly in a production environment.
Thus, Open Data Hub provides a cloud-like approach throughout the entire AI/ML operations lifecycle, starting from data access and preparation to model training and production deployment.
Bringing it all together
Now the question arises of how to organize all this for the OpenShift administrator. This is where a special Kubernetes operator for Open Data Hub projects comes into play.
This operator manages the installation, configuration, and lifecycle of the Open Data Hub project, including the deployment of the aforementioned tools like JupyterHub, Ceph, Spark, Kafka, Seldon, Prometheus, and Grafana. The Open Data Hub project can be found in the OpenShift web console under the community operators section. Thus, the OpenShift administrator can designate that the relevant OpenShift projects fall under the category of 'Open Data Hub project.' This is done once. After that, the data analyst logs into the web console of OpenShift into their project space and sees that the corresponding Kubernetes operator is installed and available for their projects. They can then create an instance of the Open Data Hub project with a single click and immediately gain access to the tools described above. All of this can be set up for high availability and fault tolerance.
If you want to try the Open Data Hub project yourself, start with . Technical details of the Open Data Hub architecture can be found , project development plans - . Future plans include additional integration with Kubeflow, addressing a number of data regulation and security issues, and organizing integration with rule-based systems like Drools and Optaplanner. You can express your opinion and become a participant in the project on the page .
To summarize: serious scaling issues hinder organizations from fully realizing the potential of artificial intelligence and machine learning. Red Hat OpenShift has long been successfully used to address similar challenges in the software industry. The Open Data Hub project, implemented within the open-source development community, offers a reference architecture for organizing the full cycle of AI/ML operations based on the OpenShift hybrid cloud. We have a clear and well-thought-out development plan for this project, and we are seriously committed to creating an active and fruitful open AI solutions development community on the OpenShift platform.
Source: habr.com
