Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB Development platform for image processing components
TULIPP project of EU consortium with Fraunhofer IOSB
Sei es fürs Auto oder die Drohne: Bildverarbeitungssoftware für eingebettete Systeme zu entwickeln nimmt viel Zeit in Anspruch und ist daher teuer. Die Plattform TULIPP erlaubt es nun, energieeffiziente Embedded-Bildverarbeitungssysteme schnell und kostengünstig zu entwickeln – und damit die Zeit bis zur MarSei es fürs Auto oder die Drohne: Bildverarbeitungssoftware für eingebettete Systeme zu entwickeln nimmt viel Zeit in Anspruch und ist daher teuer. Die Plattform TULIPP erlaubt es nun, energieeffiziente Embedded-Bildverarbeitungssysteme schnell und kostengünstig zu entwickeln – und damit die Zeit bis zur Markteinführung drastisch zu verkürzen. Ein EU-Konsortium, dem das Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB angehört, hat den Prozess vereinfacht.teinführung drastisch zu verkürzen. Ein EU-Konsortium, dem das Fraunhofer-Institut für Optronik, Systemtechnik und Bildauswertung IOSB angehört, hat den Prozess vereinfacht.
At first glance, drones, driver assistance systems and mobile medical diagnostic devices do not have much in common. However, they are increasingly using image processing components, for example for obstacle detection or pedestrian detection. In the case of mobile X-ray equipment, image processing can be used to ensure sufficient image quality even with reduced radiation, thus considerably reducing radiation exposure.
In contrast to a workstation computer, where dimensions and energy consumption are not really critical, such applications require small, lightweight, energy-efficient and yet real-time capable image processing components. Common computer architectures with processors are not a suitable hardware platform for these requirements. Therefore embedded systems based on so-called Field Programmable Gate Arrays (FPGAs) are often used. These are logic devices whose circuit structure can be freely configured using a special type of programming.
For this purpose, the hardware-oriented language VHDL is usually used. The problem is that most image processing applications are available in higher programming languages like C/C++, and their transfer to the embedded systems is very cumbersome. Not only is VHDL very different from other programming languages - in addition it has to be adapted to the respective hardware. So even already existing VHDL programs are not transferable to other hardware. So software developers have to start almost from scratch with every new system.
Starter kit for fast implementation of particularly energy-efficient embedded systems
A consortium of eight partners from six countries - including the Fraunhofer Institute for Optronics, Systems Engineering and Image Exploitation IOSB in Karlsruhe - has now considerably simplified this procedure within the Tulipp project. "The result was a development platform consisting of design guidelines, a configurable hardware platform, a real-time operating system that supports multicore processors, and a programming tool chain," summarizes Dr.-Ing. Igor Tchouchenkov, group leader at the Fraunhofer IOSB. "In addition, a starter kit launched by one of our partners as part of Tulipp will also help. With it, such applications can be developed much faster and easier. Porting C++ programs to FPGA, on which a developer has often worked for several months, can be accomplished within a few weeks with the Tulipp Starter Kit".
For the developer, this means: Based on the software programmed in C++, he first considers which code components are to be distributed to which hardware components and which program steps can be optimized or parallelized. The design guidelines that have been developed help him in this process. Then the starter kit comes into play. It contains the configurable hardware to which the required sensors and output devices can be connected, the multiprocessor-capable real-time operating system and the so-called STHEM toolchain. The applications contained therein allow the C++ program to be optimized in such a way that it can be ported to the FPGA as easily and quickly as possible. "A special focus of the toolchain is on energy optimization: After all, the image processing systems should be able to get by with a small battery if possible," says Tchouchenkov. "In the toolchain, the energy consumption can therefore be displayed and optimized individually for each code function".
A hundred times faster than the high-end PC
In order to develop and test the Tulipp platform, the consortium has worked on three specific use cases: The research team at the Fraunhofer IOSB was dedicated to the procedures for obstacle detection of a drone with stereo cameras, while other project partners dealt with pedestrian detection in the vicinity of a car and radiation reduction during X-ray examinations during an operation. Within the scope of Tulipp they have ported corresponding image processing methods from C++ to FPGA.
The result is impressive: The procedures, which originally took several seconds to analyze a single image on a high-end PC, now run in real time on the drone, i.e. approximately 30 images per second are now analyzed. The speed of pedestrian detection has also been increased by a factor of one hundred: Instead of one image per seven seconds, the system now produces 14 images per second. And radiation during X-ray examinations has been reduced to a quarter thanks to upstream filters and improved image evaluation. At the same time, the energy consumption of all three applications has been reduced many times over.
More informationen:
Last modified: