RESEARCH - Exploratory work in 3D reconstruction
Separation Bar

Online as well as global mosaicing techniques as discussed so far use plane-to-plane transformations (homographies) in order to describe the camera motion (and image to image transformations). This approach assumes that the ocean floor is flat, thus, reducing the complexity of the scene model and making it suitable for large area mapping.

Indeed, various parts of the sea floor showing high relief and topographical variations do not fulfil this so called planar scene assumption and, hence, cannot be accurately mapped applying these mosaicing approaches. However, undoubtedly these regions are also of considerable interest for biological and geological purposes. One possible work-around to tackle this problem is the use of ortho-mosaicing techniques. In this case, first the 3D model of the region is obtained either by stereoscopic or structure from motion techniques and then the 3D model is projected onto a plane. The result is a map similar to a 2D mosaic. While able to accurately model complex 3D scenes locally, however, these techniques have poor performance over large areas as they are in the focus of this project, which is mainly due to the high complexity of the 3D models.

To overcome the formerly mentioned problems an interesting and challenging objective of the proposal will be to explore the possibility of combining 2D and 3D ortho-mosaicing techniques. The aim is to survey the perspectives resulting from this linkage towards obtaining a system that combines adequate handling of reliefed 3D regions while at the same time maintaining the robustness of 2D mosaicing over large areas. Doing so various problems will have to be faced. One of the most important ones is related to the 2D image registration techniques that are very prone to errors in the case of complex 3D scenes, that is particularly acute in underwater scenes. However, this problem can be tackled by using an additional sensor in combination with the trinocular camera system. We propose the use of an echoscope. This sensor will provide the system with non- visual depth cues that will be combined with 3D information extracted from temporal and spatial redundancies given in the trinocular camera data. On the one hand the mosaicing system can use this information to improve the registration and in particular bound the estimation errors which is essential for robust 3D reconstruction and 2D motion estimation. Furthermore even local super-resolution techniques maybe applied to the data.

One of the most interesting aspects of this exploratory part is the possibility to analyze not only photometric aspects of the area but also topographic details. Once the global 2D mosaic has been built, the scientists can select specific regions of interest (ROI) that they would like to analyze in detail. The system may then automatically generate detailed and textured 3D local maps of that region using information from the trinocular camera system in conjunction with the echoscope. This could yield an attractive system feature for scientific purposes, since it can provide valuable high-resolution information regarding timely changes of those ROIs: (e.g. erosions of coral reefs, movements of sediment bodies, changes in biological communities at the seafloor, growth of hydrothermal vents, etc.).