KARLI - AI for interaction in the vehicle of the future

Artificial Intelligence (AI) for adaptive, responsive and level-compliant interaction in the vehicle of the future

The data will be collected and used in KARLI in such a way that the project results are scalable to Big Data from production vehicles available in the future.
© freepik
The data will be collected and used in KARLI in such a way that the project results are scalable to Big Data from production vehicles available in the future.
In KARLI, IOSB is developing image-based AI algorithms for level-compliant behavior detection and motion sickness determination. Recognized activities and intentions are optimized for interaction with vehicle occupants, among other things.
© Fraunhofer IOSB / M. Zentsch [M]
In KARLI, IOSB is developing image-based AI algorithms for level-compliant behavior detection and motion sickness determination. Recognized activities and intentions are optimized for interaction with vehicle occupants, among other things.

Competence of Fraunhofer IOSB

In KARLI, Fraunhofer IOSB is developing the Advanced Occupant Monitoring System for the adaptive design of interactions in the vehicle interior. For this purpose, new data from road trips with changing automation levels are collected and refined. Data collection is performed in a Wizard-of-Oz vehicle in cooperation with Fraunhofer IAO. In this process, instructed and freely observed secondary activities on long car journeys are detected by Fraunhofer IOSB AI methods. The Fraunhofer IOSB Drag&Sense framework is optimized in the project in a simple plug-and-play hardware for flexible use in real vehicles. For the first time, we are working with new cameras and lighting adapted to the future automotive drag&senseve series standard, making our solutions compatible with our customers' requirements.

Short description of the project

The goal of “KARLI” (acronym of the German project title "Künstliche Intelligenz für Adaptive, Responsive und Levelkonforme Interaktion im Fahrzeug der Zukunft") is to develop an adaptive, responsive and level-compliant interaction between humans and vehicles of the future and to use AI methods to develop new safety-relevant assistance systems for automated vehicles. The AI know-how is driven by a consortium of strong partners from research and industry. The consortium management is shared between Continental (responsible for administrative management) and Fraunhofer IOSB (responsible for scientific and technical management).

Project goals

For the development of an adaptive, responsive and level-compliant interaction in the vehicles of the future, customer-relevant AI functions are being developed in KARLI that capture driver states and shape interactions for different automation levels. These AI functions are developed in KARLI from empirical and synthetically generated data. The data will be collected and used in KARLI in such a way that the project results are scalable to Big Data from production vehicles available in the future. KARLI provides the specifications and requirements for the vehicle architecture and sensor technology of the future so that upcoming interior sensors collect optimal Big Data for driver-vehicle models and AI interactions.

Project results

The target applications of KARLI are aligned with the overall goal of “level-compliant driver behavior”. The current driving situation, including the automation level, results in concrete requirements for the driver’s state, his behavior and the predictable ability to act. The recording of the driving situation and the driver’s state thus allows an actual-goal comparison and a targeted approach in the dialog between man and machine. Thus, on the one hand, mode and situation awareness on the part of the driver is created and the readiness for transition is promoted, and on the other hand, the trajectory planning and the automation level are adapted to the driver’s state. KARLI promotes driving with the highest possible level of automation, which has a positive effect on efficiency and safety. At the same time, motion sickness, misuse and abuse of the automation function are detected and avoided.

Project partners

Fraunhofer IOSB with its research group Perceptual User Interfaces together with

  • Allround Team GmbH - Data collection in the real vehicle for motion sickness issues
  • AUDI AG - Functional implementation of adaptive and responsive interactions
  • branmatt II legal - legal consulting for the overall project
  • Continental Automotive GmbH - administrative project coordinator, function implementation for AI-based recognition of level-compliant driver behavior
  • Ford-Werke GmbH - Functional implementation of a Visual Activity Manager to avoid motion sickness
  • Fraunhofer IAO - data collection and data refinement from journeys with the Wizard-of-Oz real vehicle
  • Fraunhofer IOSB - scientific - technical coordinator, camera-based occupant monitoring and data collection in real vehicles
  • Hochschule der Medien - User-centered development methods and user experience evaluation in the development process
  • INVENSITY GmbH - data management, GaiaX and Cloud, AI architecture consulting
  • paragon semvox GmbH - proactive voice interaction, adaptive and responsive assistance
  • studiokurbos GmbH - interaction design for adaptive and responsive interaction in vehicles
  • TWT GmbH Science & Innovation - outdoor context interpretation and synthesis with occupant state
  • University of Stuttgart IAT - Data collection for proactive voice interactions and for the recording of driver states via HMI
 

Department HAI

Would you like to find out more about our projects in the automotive sector? Read about our further activities within the Human-AI Interaction department here!

 

Project details

KARLI – Artificial Intelligence for Adaptive, Responsive and Level-Compliant Interaction in the vehicle of the future

Project duration: 2021-2024

The project is funded by the German Federal Ministry for Economic Affairs and Energy.