Method for the automated analysis, control and correction of stereoscopic distortions and parameters for 3D-TV applications: new image processing algorithms to improve the efficiency of stereo- and multi-camera 3D-TV productions

The background and motivation for the research performed within this thesis is the introduction of the Digital Cinema which allows for new workflows based on image processing algorithms. Thereby, the development of algorithms for stereoscopic 3D and multi-camera productions within the era of the Digital Cinema is of special interest. Several 3D productions have been released in the cinemas in the past years while the basic principle of 3D reproduction is still based on Wheatstone’s [Wheatstone38] and Brewster’s stereoscopic approach [Brewster56] where two views corresponding to two different viewing positions are presented to the viewer’s left and right eye. However, if the reproduced 3D content imposes unnatural viewing conditions when watched, e.g. due to an excessive amount of inherent parallax, an impaired 3D sensation can result which can even lead to visual fatigue and head-ache [IJsselsteijn00]. Consequently, specific 3D production rules as described in [Mendiburu08] and [Knorr12] have to be obeyed when high quality 3D content shall be produced. It includes a precise calibration of the two cameras with consistent electronic and optical parameters. Moreover, the stereo baseline and convergence distance have to be chosen according to the depth structure of the scene content. When performed without specific assistance systems, the calibration process and the choice of proper stereoscopic parameters as described by Lipton in [Lipton82] can be tedious tasks which require trained personnel and increase the overall production costs [Buchs11]. With the advent of digital cameras, it became possible to analyze and possibly correct the 3D signal electronically using dedicated stereoscopic image processors [Zilly10b, Sony] which facilitates the above mentioned tasks and allows for new 3D production workflows, possibly lowering the costs and improving the resulting quality. Against this background, within this thesis, a new and robust technique for camera pose estimation and rectification of uncalibrated stereo cameras based on a new method to estimate the fundamental matrix is proposed. The approach is subsequently enhanced towards trifocal setups involving a new estimation method for the trifocal tensor. To rectify the images acquired by uncalibrated cameras, a suitable feature detector is required. In this context, a new feature descriptor (SKB) is proposed and compared to existing descriptors such as SIFT, SURF or BRIEF. The different algorithms are combined, extended by new functions to calculate important stereoscopic parameters, and made accessible through an intuitive graphical user-interface which allows non-expert camera personnel to make use of it using an application which is called stereoscopic analyzer (STAN). Finally a new multicamera disparity estimation workflow is proposed and applied to a multi-camera setup suitable for the generation of display agnostic 3D content. 1 An extensive list of 3D movie releases which is maintained by Andrew Woods, Co-Chair of the annual Stereoscopic Displays and Applications (SD&A) conference, can be found on the following website [Woods14]: http://www.3dmovielist.com/