In his master thesis at the Fraunhofer Institute for Optronics, Systems Engineering and Image Exploitation IOSB, Daniel Reichert designed, implemented and evaluated a novel vehicle assistance system for maneuvering maneuvers of trucks in cooperation with Rio, a subsidiary of the MAN Group. Subsequently, Daniel Secker developed and tested a real-time system for passenger cars as part of his master thesis. With the help of virtual reality glasses and stereo cameras, it makes the vehicle transparent to a certain extent, so that the driver can see all obstacles directly and three-dimensionally at their real position. Strictly speaking, the system creates an enhanced reality representation, an overlapping of the normal driver's perspective with a virtual view generated by a computer based on the camera shots.

"The goal of this method is to minimize the frequent accidents that occur when maneuvering trucks and cars. Until now, drivers often overlook obstacles or rely too much on existing assistance systems," explains Daniel Secker. Especially for trucks with trailers, it is only possible to capture the relevant environment by using additional mirrors. Even objects directly in front of the vehicle can only be seen indirectly. This represents a potential source of error and increases the reaction time.

Intuitive visualization through camera shots and VR

The transparent cockpit, on the other hand, should allow intuitive visualization. An algorithm recognizes the position of objects in the room on the basis of camera shots and colors them first orange and then red when they come dangerously close.

In a user study, Secker asked 15 test persons to park a car backwards - with and without VR glasses. Cardboard boxes served as obstacles. The result: Drivers can actually judge the distance better through their glasses, but only by highlighting the obstacles in color. The VR glasses alone did not yet produce the desired effect.

However, the virtual reality glasses used are too clumsy for a really practical system, as Secker himself points out. "For this we would need VR or even augmented reality glasses that are suitable for practical use. The driver could wear the latter permanently like normal glasses; only during maneuvers would the virtual display layers be faded in. "Unfortunately, the models available today still have too narrow a field of vision for this," says Secker.

How a transparent cockpit can look is also shown in this video, which was created as part of Daniel Reichert's master's thesis: https://www.facebook.com/riothelogisticsflow/videos/609405132770907