Line-projection in omnidirectional vision: modelling, extraction and calibration in central and non-central cameras

Computer vision has an increasing interest in most fields of emerging technologies. A challenging topic in this field is to study how to enlarge the field of view of the camera systems to obtain more information of the environment in a single view. In particular omnidirectional vision can be useful in many applications such as estimating location in robotics, autonomous driving and unmanned aerial vehicles. The wide field of view (FOV) of omnidirectional cameras allows taking advantage of describing 3D scenarios using line features. On the one hand, line features represent natural landmarks in manmade environments, they are easy to understand, coincident with edges of constructive elements and often still present when having texture-less scenarios. On the other hand, long segments are especially useful for drift reduction because they are usually completely visible on the omnidirectional projection. However, in omnidirectional cameras line projections are distorted by the projection mapping becoming complex curves. This thesis is focused on the geometry of line projections (line-images) in omnidirectional systems. Main addressed topic of this work is line-image extraction on different kinds of central and non-central omnidirectional images. However, due to nature of projection in omnidirectional cameras, other addressed topics are camera calibration and, in the case on non-central cameras, 3D reconstruction from single images. In particular, we analyzed the following problems involving line projections in central and non-central omnidirectional cameras: Line-image extraction in central omnidirectional cameras. The presented research begins with line-image extraction from hypercatadioptric images which is used for estimating dominant directions in Manhattan scenarios. This extraction approach has been generalized to a framework for central imaging systems obtaining the equations of line-images curves for a set of different catadioptric and dioptric systems. Using this framework we have developed the plumb-line constraint for each of these systems and obtained analytical solutions that allow simultaneously recovering camera calibration and line-images. We have integrated these theoretical solutions in an automatic method for line-image extraction in central systems when the calibration is unknown. This proposal exploits location of image points and gradient direction on these points reducing the complexity of the extraction scheme. Line-image extraction and 3D line fitting in non-central omnidirectional cameras. The characteristics of the projection in non-central systems allow recovering the complete 3D information of a line from a single projection. The non-central systems addressed in this thesis are the conical and spherical catadioptric systems and the non-central circular panorama. Generalizing the description used for central systems we have developed the analytical line-image equations for conical and spherical catadioptric systems. The particularization for conical catadioptric systems allows us to recover the geometry of the mirror and the geometry of the 3D line from five points of the line-image. Main problem of recovering 3D from single projections in non-central systems is that results are very noise sensitive because four degrees of freedom are involved making line-image extraction a challenging unsolved task. To address this issue we propose different solutions and approaches. To select a set of rays in a robust framework and evaluate the accuracy we introduce the concept of effective baseline among rays which strongly depends on the class of non-central system. Regarding accuracy in line 3D fitting, the non-central panorama is the system obtaining the best results although its constructive difficulties usually prevents from a practical using. To solve the problem of automatic line-image extraction in non-central systems we propose a robust approach using a pre-evaluation step, similar to the one used in PROSAC, and using different kind of distances. In particular, we have analytically solved the Euclidean distance from a point to line-image for conical and spherical catadioptric systems and we propose a distance based on the projection of the closest points in the 3D space for the non-central circular panorama. Minimal solutions for line-image fitting in non-central cameras by imposing geometric constraints. Another way for increasing the accuracy in 3D line reconstruction is exploiting geometric constraints and using prior information. In this thesis, we propose a set of new minimal solutions for imposing additional constraints between pairs of lines in non-central systems. In particular, we have developed the cases of intersecting orthogonal lines and pairs of parallel lines. Finally we present the minimal solution for computing a line which is parallel to a given plane which can be exploited using the prior information of the vertical direction in a Manhattan scenario.