Artificial intelligence in remote sensing

Keeping an overview with AI

AI applications are already widely used in many areas of everyday life. Appropriately trained networks write texts, design images and recognize structures in multidimensional spaces that often exceed human imagination.
Classic tasks in the evaluation of large (volumes of) aerial and satellite image data include searching, finding, classifying and counting various objects. Here, too, the human eye reaches its limits. If the evaluator looks at the entire image, smaller objects are quickly overlooked. If he examines enlarged image sections, it is difficult to maintain an overview of the entire image. This is why automated methods of digital image processing are increasingly being used for such tasks. Whether ships in ports, airplanes at airports or cars in parking lots: modern object detectors support the evaluation of remote sensing data in real time with the help of increasingly powerful hardware components. For this purpose, the large-format image is divided into tiles with overlapping areas and evaluated. Until now, however, this has often led to duplicate object detections in the overlapping areas.

For this reason, Fraunhofer IOSB developed an AI based on the ARM-NMS approach (Area Rescoring Mask - Non Maximum Suppression). Two-stage object detectors first create suggestions for detections and place so-called bounding boxes around the objects. The suggestions are then sorted out using the non-maximum suppression method. The problem with rectangular bounding boxes is that their shape usually does not match the outline of the objects to be detected. The Area Rescoring Mask method is therefore used to adapt the shapes of the bounding boxes to the shape of the object to be detected. The AI has also been trained to reliably evaluate scenes with widely varying object densities, which posed problems for previous approaches.

Save data volume with AI

Conventional RGB sensors (e.g. cell phone cameras) record three channels per pixel in the visible wavelength range. In contrast, hyperspectral imaging sensors measure up to several hundred channels per pixel. This allows material-specific differences in surfaces to be distinguished. However, the data volume of hyperspectral images increases enormously due to the additional image information (several 100 MB!). Due to limited transfer rates and storage media volumes, there are therefore restrictions on data transfer and archiving.

However, near-real-time applications in particular are becoming increasingly relevant in today's everyday life, which is why the fastest possible data transfer of large volumes of data is also becoming more important. However, as long as only a limited bandwidth (3G, 4G, 5G, ...) is available for this, the alternative is to compress the data. With lossless compression methods, the data is reconstructed identically to the original, but the performance or compression rate is severely limited. The aim is therefore to develop lossy approaches that are as powerful as possible and still cause only a small loss of information.

For this purpose, Fraunhofer IOSB developed a 1D-Convolutional Autoencoder architecture that enables compression rates between 4-100. The encoder compresses the hyperspectral data on the flight platform by training it with data from various hyperspectral sensors (satellite, airborne and UAV-borne). The decoder is installed in the receiving station, for example, and decompresses or reconstructs the original data precisely. The advantage of this method is that the procedure works without additional training for various sensors and their specifics due to the versatile training data set and is robust against different scene contents (land surface, land use, ...).