A hardware architecture for real-time stereoscopic image generation from depth map

When you look at the object, the displacement of object is different in left and right eye. This displacement is called parallax. The farther the object is, the smaller the parallax become. Due to the parallax, people will have the feeling of stereoscopic vision. We develop a depth-image-based rendering (DIBR) algorithm to shift (or move) the position of the object in the image to generate the stereoscopic image with parallax according to its depth information. When the object is shifted (or moved) away from its original position, a hole will occur in the original position. Therefore we develop an image inpainting algorithm to fill the hole. In this paper, a hardware architecture is designed for real-time stereoscopic image generation from a 2D image and its depth map. In order to achieve the real-time application, we design a hardware architecture to accomplish the DIBR and image inpainting algorithm. There are totally 2,849 logic registers in this design. By simulation in the experiment, we can achieve the real-time computing in 150MHz clock (320 × 240 @30 fps).

[1]  Yue Qi,et al.  An image inpainting method , 2005, Ninth International Conference on Computer Aided Design and Computer Graphics (CAD-CG'05).

[2]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[3]  Guillermo Sapiro,et al.  Simultaneous structure and texture image inpainting , 2003, IEEE Trans. Image Process..

[4]  Liang-Gee Chen,et al.  Real-Time Depth Image based Rendering Hardware Accelerator for Advanced Three Dimensional Television System , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[5]  Guillermo Sapiro,et al.  Structure and texture filling-in of missing image blocks in wireless transmission and compression applications , 2003, IEEE Trans. Image Process..

[6]  G. Iddan,et al.  3D IMAGING IN THE STUDIO (AND ELSEWHERE...) , 2001 .

[7]  Liang-Gee Chen,et al.  Efficient Depth Image Based Rendering with Edge Dependent Depth Filter and Interpolation , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[8]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[9]  C. Fehn.  A 3 DTV Approach Using Depth-Image-Based Rendering ( DIBR ) , 2003 .

[10]  Christoph Fehn,et al.  Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV , 2004, IS&T/SPIE Electronic Imaging.

[11]  Guillermo Sapiro,et al.  Navier-stokes, fluid dynamics, and image and video inpainting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[12]  Yo-Sung Ho,et al.  Generation of ROI Enhanced Depth Maps Using Stereoscopic Cameras and a Depth Camera , 2008, IEEE Transactions on Broadcasting.

[13]  Sebastiano Battiato,et al.  3D stereoscopic image pairs by depth-map generation , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[14]  Yo-Sung Ho,et al.  Dynamic 3D human actor generation method using a time-of-flight depth camera , 2008, IEEE Transactions on Consumer Electronics.

[15]  Michael Lehmann,et al.  An all-solid-state optical range camera for 3D real-time imaging with sub-centimeter depth resolution (SwissRanger) , 2004, SPIE Optical Systems Design.

[16]  Zhi-Gang Zheng,et al.  A region based stereo matching algorithm using cooperative optimization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Ze-Nian Li,et al.  Review and Preview: Disocclusion by Inpainting for Image-Based Rendering , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).