Iterative mask generation method for handling occlusion in optical flow assisted view interpolation

Having a key role in 3D and free viewpoint TV applications, view interpolation techniques have attracted many researchers in recent years. In this paper we present a patch-based reconstruction scheme for view interpolation using optical flow disparity estimation. In the first step of the algorithm, reconstructed versions of the left and right views are obtained using the proposed patch-based scheme. Then mismatch masks are generated to find the best patches for final reconstruction. Finally the intermediate view is obtained by fusing the selected patches and the remaining holes are filled with a simple hole-filling algorithm. The performance of the proposed algorithm is compared with recent methods in terms of objective and subjective quality. The results show that the proposed method achieves an improvement of 3.3 dB on average.

[1]  Toshiaki Fujii,et al.  Free-Viewpoint TV , 2011, IEEE Signal Processing Magazine.

[2]  Dimitris Anastassiou,et al.  A theoretical study on an accurate reconstruction of multiview images based on the Viterbi algorithm , 1995, Proceedings., International Conference on Image Processing.

[3]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[4]  Derek Pang,et al.  Rectification-Based View Interpolation and Extrapolation for Multiview Video Coding , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[6]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[7]  Joseph J. LaViola.  Bringing VR and Spatial 3D Interaction to the Masses through Video Games , 2008, IEEE Computer Graphics and Applications.

[8]  I. Sexton,et al.  Stereoscopic and autostereoscopic display systems , 1999, IEEE Signal Process. Mag..

[9]  Ce Liu,et al.  Exploring new representations and applications for motion analysis , 2009 .

[10]  Ramin Samadani,et al.  Stereo Matching and View Interpolation Based on Image Domain Triangulation , 2013, IEEE Transactions on Image Processing.

[11]  Ronggang Wang,et al.  Intermediate view synthesis based on edge detecting , 2013, 2013 IEEE International Conference on Image Processing.

[12]  Heung-Yeung Shum,et al.  Image-Based Rendering and Synthesis , 2007, IEEE Signal Processing Magazine.

[13]  H. Park,et al.  A mesh-based disparity representation method for view interpolation and stereo image compression , 2006, IEEE Transactions on Image Processing.