Continuously improving and deploying machine learning applications –MLOps

Current situation

Like all software, AI applications are constantly evolving. It is hardly ever the case that the scope of application, desired functionality, and all of the boundary conditions that apply are known in detail at the start of the project. New requirements as well as requests for adjustments arise over time, especially once the system is live. In the case of machine learning applications, additional challenges make it especially important to pursue a structured approach both during initial development – the PAISE^® (Process Model for AI Systems Engineering) – methodology is useful here – and in live operation, during the evolution of the software components and continuous training of machine learning models. This is the only way to ensure that further developments beyond the prototype stage can ultimately be taken live into productive environments.

Objectives of MLOps

This structured approach is the subject of the MLOps paradigm (which is named by analogy to the DevOps concept in software development). A project of the same name aims to harness our experience across a whole host of AI application domains to realize MLOps with maximum effectiveness and automate it as far as possible. To that end, we have developed a software suite that incorporates a wide range of tools for creating machine learning applications and training the ML models. Our toolkit makes versioning of various model releases possible, including the associated quality criteria, while also supporting the transition to productive operation and monitoring the application and its results, for example to detect model drifts. The tool collection also includes components developed at Fraunhofer IOSB for sensor data management (FROST^®),video data annotation (ANTONN), and explainability of AI methods (XAI-Toolbox).