CoBot - Collaborative Robot

CoBot – the smart AI assembly workstation

With the CoBot - Collaborative Robot - Fraunhofer IOSB in cooperation with PWC strategy& demonstrates an innovative AI-based assembly assistance system in which robots and humans work together collaboratively in manual assembly.

The assistance system combines state-of-the-art AI machine learning methods to demonstrate the possibilities of an intuitive, worker-centric and efficient assembly assistance system that can intelligently respond and adapt to the workers and their activities. By recording the activities of the worker, the assistance system can understand in which step which actions are being followed. It can also intervene to ensure quality and use the robotic arms to proactively and efficiently assist in the various phases of assembly.

AI-based people and work environment detection for intelligent assembly assistance

Cameras installed on the table can be used to identify people in front of the table and continuously detect and analyze their posture and movements. The position and orientation of arms and hands can be used to readily infer grips and manual work. In addition, the camera detects the position and orientation of all tools used, so that not only movements but also interactions and the handling of screwdrivers, for example, can be traced.

The work steps to be performed are modeled and stored as instructions in the assistance system, allowing the actual activity to be compared with the target activity and deviations to be detected. At the same time, completed work steps are tracked, documented as completed and the display is switched to the next work step.

Intuitive human-machine interaction for efficient processes

To make human-machine interaction fast, efficient and intuitive, CoBot can be controlled using gestures and speech. A user interface displayed on the table with projectors reacts (thanks to body pose and arm detection) to approaching fingertips and is operated intuitively without further peripherals. Voice commands are particularly advantageous in situations where workers do not have a hand free for operation. Both the camera evaluation and the voice recognition run locally on site and are not cloud-based.

Smart and safe human-robot collaboration

By continuously detecting the people and work environment, the assembly assistance system continuously calculates all areas where the robotic arms can operate without colliding with the workers. In addition, the robot arms can collaboratively assist the worker in a work step by, for example, accepting objects or being autonomously used for background activities when they are not needed, for example to pick material boxes.

Dynamic projection in the work environment

As an example, two robotic arms have been installed in CoBot, the left one being used for physical assistance, the right one for dynamic projection and camera acquisition. CoBot is characterized by the fact that it is also possible to deal with occlusions in the work environment, e.g. if work participants bend over in front of the installed cameras during a screwing operation and obstruct the view, by realigning a camera attached to the right robot arm and moving it over the shoulder. At the same time, a projector attached to the arm is able to supplement the projections permanently displayed on the table surface with visualizations that can be freely aligned. Any surfaces and points in the working environment can thus be illuminated and enriched with information. In the course of a component assembly, for example, the respective screw positions can be illuminated in order to clarify the correct sequence.

Intuitive teach-in by demonstration

The CoBot assembly assistance system is equipped with a teach-in and editing mode that allows new work steps to be added and existing ones to be modified. These can be provided with text or recorded as a picture or video and are displayed in the appropriate form during execution. Positions to be bolted can also be specified by simple demonstration thanks to integrated tool tracking. The individualizations and requirements of the respective person can be stored in the worker profile.