Dictionary learning in stereo imaging

This paper presents a new method for learning overcomplete d ic ionaries adapted to efficient joint representation of st ereo images. We first formulate a sparse stereo image model where t multi-view correlation is described by local geometric t ansforms of dictionary atoms in two stereo views. A maximum-likeliho od method for learning stereo dictionaries is then proposed , which includes a multi-view geometry constraint in the probabili stic modeling in order to obtain dictionaries optimized for the joint representation of stereo images. The dictionaries are lear ned by optimizing the maximum-likelihood objective functi on using the expectation-maximization algorithm. We illustrate the le arning algorithm in the case of omnidirectional images, whe re we learn scales of atoms in a parametric dictionary. The resulting di ctionaries provide both better performance in the joint rep resentation of stereo omnidirectional images and improved multi-view f eature matching. We finally discuss and demonstrate the bene fits of dictionary learning for distributed scene representation and camera pose estimation. Index Terms Sparse approximations, dictionary learning, multi-view i maging, omnidirectional cameras. I. I NTRODUCTION Multiple images of a 3D scene taken from different viewpoint s contain information about both 3D structure and texture of the objects in the scene. Therefore, these images give a ri cher description of the environment compared to a single vie w. Multi-view images are usually captured by a network of camer as distributed in a 3D scene. Such visual sensor networks can find usage in applications like 3D television, surveillance , robotics or exploration. However, dealing with the high di mensional visual information still poses many challenges, such as mul ti-view compression, 3D geometry estimation and scene anal ysis. Extraction of 3D information from multiple views relies on t he theory of the multiple view geometry [1], which relates image features that represent the same 3D objects in differe nt vi ws. Pixel-based image representation is used in most o f the image-based 3D geometry estimation methods that build dens e depth maps by computing pixel correspondences. However, pixel-based representations are highly inefficient for ima ge coding and compression. On the other hand, image represen tations with orthogonal bases are efficient for compression, but gen erally fail to efficiently capture the geometry of objects in a scene and the correlation between views. Therefore, multi-view i maging requires new image representation methods that give good performance in both compression and scene geometry estimat ion. This paper addresses the problem of learning dictionaries a dapted to the representation of multi-view images. We consi der sparse image approximations with overcomplete dictionari es of geometrical atoms. As the correlation between multi-v iew images arises from the geometric constraints on the objects in the scene, it can be simply described by local transforms o f geometric atoms [2]. We propose to learn dictionaries that e ffici ntly describe the content of natural images and simult aneously permit to capture the geometric correlation between multiview images. Dictionary learning for sparse signal represe ntations has become an extremely active area of research in the last fe w years, when it was realized that adapting the dictionary to a specific task or imposing a certain structure to the diction ary can yield significant improvements of performance in tar get applications. Researchers have addressed the problem of le arning dictionaries for image [3]–[5] and video representa tion [6]– [8]. To the best of our knowledge there has been however no wor k on learning dictionaries for multi-view representation. We concentrate on the problem of two views and develop a maximum likelihood (ML) method for learning dictionaries that lead to improved image approximation under the sparsity prior, a nd t the same time give better multi-view geometry estimati on from sparse low-level visual features. Our method builds up on the ML method for learning overcomplete dictionaries fro m natural monocular images, introduced by Olshausen and Fiel [3]. Additionally, the proposed probabilistic approach t o learning includes the epipolar geometry in the modeling, and hence ma tches corresponding atoms within the learning process itse lf. The optimization problem is cast as an energy minimization p roblem, that we finally solve with an Expectation-Maximizat on (EM) algorithm. The experimental results show the significa nt benefits of stereo dictionary learning for applications s uch as distributed scene representation and camera pose recovery . The organization of this paper is as follows. We first overvie w the related work on dictionary learning in Section II. The stereo image model is introduced in Section III. Section IV p resents the optimization problem for learning dictionarie s adapted to stereo images, while its energy minimization solution in g ven in Section V-B. Experimental results in omnidirectio nal imaging are presented in Section VI.