Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB About m3motion
The system m3motion is able to detect and track moving objects with cameras on a moving platform (MTI – Moving Target Indicator). Especially for security and surveillance tasks is it useful to track the motions of persons and objects or to generate mosaic images to allow human observers a better interpretation of the situation.
By estimating the camera motion continouslywe can stabilize or simply supress the background. We are able to exploit video streams or IR-sensors. The system runs in real-time (25Hz) on a standard PC hardware (>1.7GHz)
Optimum image recognition when the camera and target object(s) move
Application areas for image recognition systems
EO sensors are used in many moving sensor carriers in the fields of "Security" and "Surveillance" to detect dynamically changing scenes:
- Helicopters perform police duties and monitor borders or capture dynamically changing scenes in general
- For reconnaissance and surveillance purposes, the Bundeswehr uses unmanned aerial vehicles (drones) with imaging sensors in its worldwide operational areas
- Imaging sensors can also be found in many all-terrain land vehicles
Optimum image recognition in practice: the problems
Even stationary cameras are not always resistant to vibrations. Fluctuations of masts, self-movements of the sensor (e.g. via pan/tilt device) or of the sensor carrier, but also changes in the optics (zoom) make optimal image recognition more difficult.
In such cases, the human expert cannot always capture the dynamics of the photographed scene with sufficient precision and reliability. Moving target objects are overlooked or already captured objects are lost sight of again. Movements of the sensor itself (e.g. via the pan/tilt mechanism), the sensor carrier or changes in the optics (zoom) make optimal image recognition more difficult.
Support from existing image recognition systems
It is desirable to support the human expert with appropriate functionality. Previous systems are however very limited in their performance. Either they require highly precise information about the flight and recording parameters as well as assumptions to be made about the scene (background) or similar, or they are only able to make rough estimates of relative movements.
Approach and application areas
Robust methods estimate projective images between successive images of an image sequence. Based on this, differential images are calculated, whereby the changes in the images caused by camera movement (translation and/or rotation) or changes in optical properties (zoom) are compensated for. Subsequently, an adaptive thresholding method can be used to detect such scene objects that move relative to the background. The adaptive thresholding method also operates very robustly and is almost independent of contrast.
Even in non-cooperative scenarios, in which e.g. target persons want to remain undetected and try to hide themselves by clothing and movement patterns, the system acts like an MTI method according to the motto "whoever moves is detected".
The human expert can freely adjust the sensitivity of the system. Thus the operator decides whether he is interested in "fast", "moderate" or "slow" movements of potential targets. Apart from this parameter, however, the procedure is parameter-free. For example, the detection of a moving person or a slow moving vehicle is possible without individual parameterisation. To ensure that a target object, once detected, is not lost due to occlusion and/or interruption of movement, a real-time multi-target tracker is coupled. The number of simultaneously detectable target objects depends only on the performance of the computer.
This technology can handle three tasks at once:
Object tracking, Moving Target Indicator
The following film sequence shows an example of the use of the m³motion® system for vehicle and person tracking.
The right-hand area represents the scene recorded, with additional displacement vectors from image to image shown in green and the trajectory of a target object moving relative to the background shown in yellow. The left area shows the current detection of the moving target (red circle).
Automatic image carpet generation
(Image mosaic, panorama images) The system can also be configured so that the object background is not suppressed while the camera is moving, but the image is stabilized.
The images continuously generated from different parts of the scene are combined into a carpet of images which is updated in steps. The larger context thus displayed allows for improved interpretation of the scene, both by the human observer and by automatic procedures.
Image stabilization
By estimating the sensor's own movement, it is also possible to eliminate blurring and stabilise the image in real time and with high precision. In addition, the resolution of a particular snapshot can be subsequently improved by stacking several individual images from a film sequence. One thinks here of recordings from surveillance cameras at ATMs, which operate at very low frame rates.
Product maturity
The m³motion® system will soon be integrated into an image processor of a police helicopter and is about to be introduced to surveillance robots for private security services. Here, m³motion® is typically an automatic component in an interactive system for image-based surveillance.