After around three years, the funded project KARLI came to an end with the presentation of its results in the Arena2036. KARLI stands for Artificial Intelligence (AI) for Adaptive, Responsive and Level-compliant Interaction in the Vehicle of the Future. On Wednesday, September 18, 2024, 120 participants from science, industry, politics and the press gathered in Stuttgart-Vaihingen. The twelve consortium partners involved in the project presented their research results on applications in passenger cars, which specifically benefit from a new AI approach. In addition to numerous presentations and an extensive poster exhibition covering all aspects of the project’s work, attendees were able to test the AI applications in practice: in demonstrators, in stationary vehicles, and even while driving. The focus of the project was to design the interaction between the vehicle and its users with the help of AI, taking into account the changing requirements of different automation levels while driving.
“The automotive industry is in the midst of a technological upheaval. On the one hand, of course, in terms of digitalization and drive technology. Particularly with regard to the further development of automated driving,” said Nina Stock, spokeswoman for the Digitalization, Industry 4.0 Department IV A 3 at the Federal Ministry of Economics and Climate Protection (BMWK), in her welcoming speech. Stock continues: “Fully automated driving is not yet a reality. But it seems to be within reach, as driver assistants and partial automation are already in use today. In order to achieve the desired breakthrough, it is necessary to further increase safety and confidence in autonomous driving. Initiatives such as the KARLI project make a decisive contribution to this.
Increasing safety and comfort
Dr. Frederik Diederichs, technical project manager of KARLI and group leader at Fraunhofer IOSB, introduces the project: “Automated driving on public roads today and in the short to medium term only temporarily relieves the user of the need to control the vehicle. For example, depending on the level of automation, a passive role can be assumed at low speeds, in good weather, or when traffic is light, allowing the user to engage in non-driving activities. However, changing external conditions can make it necessary to take active control at any time. Diederichs explains the research project: “In KARLI, we have dedicated ourselves to making this transition between active and passive user roles as safe and comfortable as possible. To do this, we are using the possibilities that AI offers us today. Essentially, it is a matter of correctly interpreting the situation in the vehicle and designing the necessary human-machine interaction accordingly and individually – and all this in compliance with ethical aspects, data security and data protection according to European standards”.
In particular, the project focused on three AI applications for the so-called SAE levels two to four: partial driving automation (2), conditional driving automation (3), and highly automated driving (4). SAE levels are used to classify assisted, automated, and autonomous driving according to SAE standard J3016. Each level has a different role for the human involved. “KARLI uses AI to keep the user constantly aware of his changing tasks,” explains Diederichs.
Switching between active and passive usage behavior
The “Level-compliant Behavior” team developed solutions for driver-vehicle interaction that consider the need to switch roles at different levels of automation. For example, the driver usually has only a few seconds to switch from reading a book to actively controlling the vehicle. As soon as the vehicle detects a behavior that is not allowed at the corresponding automation level, the multimodal HMI (Human Machine Interface) in the vehicle interacts with the user. An AI agent selects the type of signal (audio, visual, haptic, or a combination) based on the driver’s state.
Designing human-machine interaction
The “AI Interaction” work package dealt with questions concerning the design of the HMI and thus the perception of the AI in the vehicle. How should the general activity of an AI be made visible in the vehicle? What should an AI avatar look like so that it is both trusted and understood for what it is trying to communicate? And how can the AI semantically understand the driver and the situation? Generated user interfaces allow the vehicle to address users individually and in a situationally appropriate manner. Interior sensors record the condition of the passengers, and the vehicle asks about usage preferences at the appropriate time. This data, combined with large language models, enables the vehicle to recognize context and interact with users without distraction. Distracting messages are reduced by referencing history and context, customizing them, and delivering them at more appropriate times.
Preventing motion sickness
The research topic “Recognizing and Avoiding Motion Sickness” was dedicated to the question of how the use of AI can not only recognize the onset of motion sickness during non-driving activities, but also prevent it. To do this, the AI must learn to anticipate motion sickness and compare it to the current interior conditions. With this information, the AI can then interact with passengers to provide recommendations for behavioral adjustments to avoid motion sickness.
Christoph Wannemacher from the consortium leader Continental Automotive Technologies GmbH addresses the sponsors and funding agencies: “We would like to thank the German Federal Ministry for Economic Affairs and Climate Action and our funding agency, TÜV Rheinland Consulting GmbH, for their confidence in our consortium and in the sustainability of our research project. We are convinced that over the past three years, we have achieved results that are not only relevant for the future of automated driving, but can also be implemented in real industry.
The twelve partners involved in the KARLI consortium were: Continental Automotive Technologies GmbH, Ford-Werke GmbH, Audi AG, INVENSITY GmbH, semvox GmbH, TWT GmbH Science & Innovation, studiokurbos GmbH, Fraunhofer Institute for Industrial Engineering IAO, Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB, Allround Team GmbH, University of Media Stuttgart, Institute of Human Factors and Technology Management IAT at the University of Stuttgart.
Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB 
