Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB 
Brief description of the project
In the future, automated vehicles will not only relieve the driver, but will also take over complete vehicle control for a certain time. This creates the need to transfer vehicle control between the driver and automation. For a safe handover, the automated vehicle must know the driver's current capabilities and adapt itself to them. Only efficient and comprehensive driver observation allows safe and comfortable transfer of control.
Project goals
The problem: Today's driver observation ignores secondary activities.
The current state of the art in driver observation is mainly limited to fatigue detection and gaze measurement for attention detection. A prognosis about the driver's distraction, e.g. due to the secondary activity he is engaged in, is sometimes not possible. With increasing automation, it is to be expected that secondary activities already performed by the driver today will occur more frequently. And with the reduction of human driving responsibility, a liberalization of the distraction bans in the passenger compartment is obvious, provided that it can be ensured that the driver can take back control of the vehicle in time.
Secondary activities can be detected by camera. Such systems show their strengths in the above-mentioned application because the possibilities in the vehicle interior are limited and a predefined number of activities to be detected can be narrowed down.
Project result - Driver observation for recording secondary activities.
At IOSB, camera-based driver observation is being expanded to include the body skeleton in addition to the direction of vision and head rotation. In this way, the interior detection system gains knowledge about where the driver's hands, head, and finally the driver's attention are located. By capturing the entire interior space of all objects and occupants, physical activities can be put into context and secondary activities can be classified in a robust way.
Based on the detected secondary activities, the project will investigate takeover periods in which the driver is basically ready to resume driving responsibility. Adaptive human-machine interfaces to alert the driver to the need for a take-over will be adapted accordingly.