Robust image, depth, and occlusion generation from uncalibrated stereo

Philips is developing a product line of multi-view auto-stereoscopic 3D displays.1 For interfacing, the image-plus-depth format is used.2, 3 Being independent of specific display properties, such as number of views, view mapping on pixel grid, etc., this interface format allows optimal multi-view visualisation of content from many different sources, while maintaining interoperability between display types. A vastly growing number of productions from the entertainment industry are aiming at 3D movie theatres. These productions use a two view format, primarily intended for eye-wear assisted viewing. It has been shown4 how to convert these sequences into the image-plus-depth format. This results in a single layer depth profile, lacking information about areas that are occluded and can be revealed by the stereoscopic parallax. Recently, it has been shown how to compute for intermediate views for a stereo pair.4, 5 Unfortunately, these approaches are not compatible to the image-plus-depth format, which might hamper the applicability for broadcast 3D television.3

[1]  Hans Driessen,et al.  Philips 3D Solutions: From Content Creation to Visualization , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[2]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[3]  D. Nistér,et al.  Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation, and Occlusion Handling , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Robert-Paul Berretty,et al.  Real-time rendering for multiview autostereoscopic displays , 2006, Electronic Imaging.

[5]  A. K. Riemens,et al.  Real-time embedded system for stereo video processing for multiview displays , 2007, Electronic Imaging.

[6]  Richard Szeliski,et al.  Layered depth images , 1998, SIGGRAPH.

[7]  Slawomir J. Nasuto,et al.  NAPSAC: High Noise, High Dimensional Robust Estimation - it's in the Bag , 2002, BMVC.

[8]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[9]  Carlos Vázquez,et al.  Spline-based intermediate view reconstruction , 2007, Electronic Imaging.

[10]  Christoph Fehn,et al.  Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV , 2004, IS&T/SPIE Electronic Imaging.

[11]  John A. Clarke,et al.  Characterization and optimization of 3D-LCD module design , 1997, Electronic Imaging.

[12]  Ralph Braspenning,et al.  Efficient view synthesis from uncalibrated stereo , 2006, Electronic Imaging.

[13]  Bart Barenbrug.  3 D Throughout the video chain , 2006 .

[14]  Robert-Paul Berretty,et al.  High quality images from 2.5D video , 2003, Eurographics.

[15]  Shree K. Nayar,et al.  Rectifying transformations that minimize resampling effects , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.