3D reconstruction

Initial situation

Correct decisions are the result of accurate, precise knowledge of the situation and correct assessment of the options for action. This is all the more important the more urgent the situation to be responded to and the less likely it is that there will be a second chance. Whether in rescue operations after catastrophes, in the protection of critical infrastructure or in the context of a military operation under enemy influence, situational awareness and orientation also means, above all, the assignment of information in a spatial reference and the unconditional orientation in the immediate, relevant environment.

Above all, visual information, such as that provided by our eyes or reconnaissance with imaging sensors, finds particularly easy access to the mental process of decision-making. Particularly for operations in spatially confined areas, as often occurs in object protection, rescue missions or military operations in urban areas, the mini-drone has recently established itself as an agile and versatile tool for situational awareness. Cost-effective, easy to transport and operate, and undemanding to maintain, today's mini-drone systems represent ideal platforms for quickly and flexibly filling information gaps in the immediate vicinity.

Despite all the advantages of these reconnaissance systems, however, the disadvantages must not be ignored: the miniaturised navigation systems are imprecise and, in the case of GPS, can easily be disrupted or fail altogether (GPS signal failure in the event of shadowing or multipath propagation), the light mini drones fluctuate in the wind, which has a direct effect on the quality of the image data. Although the simple operation allows for use by less specialised personnel, the system is still easy to operate. However, they can easily be overwhelmed by the task of spatial orientation in the image data.

 

Task

To enable the user to concentrate fully on the actual task of decision making, the data should be pre-processed largely automatically. Two tasks are regarded as crucial for this: Firstly, the creation of the spatial reference of the sensor images should not be left to the operator, but should be solved automatically. The determination of the position by the operator should then take place directly in the image data thus prepared. Secondly, for reconnaissance tasks where two-dimensional information is not sufficient, the third dimension is to be opened up by 3D reconstruction from image sequences. The results of this can also be used for the localisation and navigation of the mini drone itself. For the investigation, experience and methods in image processing and 3D object analysis were used. The main problem to be solved is the application to the special situation of reconnaissance with mini drones, which is characterised by the use of miniaturised sensor technology, the existence of geometrically unfavourable flight paths and the evaluation of image sequences of basically unlimited length.
 

Result

For the 3D reconstruction from mini drone videos, special emphasis was placed on sensor self-calibration in order to reduce the demands on the knowledge of the internal sensor geometry. Furthermore, sensor self-localisation was made possible by means of image sequences of any length, so that conceptually real-time solutions are available. The environment is first reconstructed in the form of dense 3D point clouds. These intermediate results, superimposed with the colour information from the image data, can already provide a first impression of the 3D environment. In contrast to conventional approaches, the modelling of the 3D surfaces that follows in post-processing dispenses with explicit model assumptions such as house or roof shapes and instead carries out a triangular meshing of the point cloud and subsequent surface approximation using L1 splines. This procedure allows the description of almost any surface shape and is therefore, in contrast to the classical methods, also suitable for the reconstruction of damaged infrastructure, which is to be expected, for example, after an earthquake or other disaster.