Multiview image sequence enhancement

Realistic visualization is crucial for more intuitive representation of complex data, medical imaging, simulation and entertainment systems. Multiview autostereoscopic displays are great step towards achieving complete immersive user experience. However, providing high quality content for this type of displays is still a great challenge. Due to the different characteristics/settings of the cameras in the multivew setup and varying photometric characteristics of the objects in the scene, the same object may have different appearance in the sequences acquired by the different cameras. Images representing views recorded using different cameras in practice have different local noise, color and sharpness characteristics. View synthesis algorithms introduce artefacts due to errors in disparity estimation/bad occlusion handling or due to erroneous warping function estimation. If the input multivew images are not of sufficient quality and have mismatching color and sharpness characteristics, these artifacts may become even more disturbing. The main goal of our method is to simultaneously perform multiview image sequence denoising, color correction and the improvement of sharpness in slightly blurred regions. Results show that the proposed method significantly reduces the amount of the artefacts in multiview video sequences resulting in a better visual experience.

[1]  Qionghai Dai,et al.  Multi-view image denoising based on graphical model of surface patch , 2010, 2010 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video.

[2]  Gauthier Lafruit,et al.  Real-time stereo-based view synthesis algorithms: A unified framework and evaluation on commodity GPUs , 2009, Signal Process. Image Commun..

[3]  Attila Barsi,et al.  Towards mixed reality applications on light-field displays , 2014, 2014 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[4]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[5]  Ce Liu,et al.  Exploring new representations and applications for motion analysis , 2009 .

[6]  Vincent Nozick,et al.  Colorimetric Correction for Stereoscopic Camera Arrays , 2012, ACCV Workshops.

[7]  N. Paragios,et al.  A high-quality video denoising algorithm based on reliable motion estimation , 2010 .

[8]  Chao Xu,et al.  Color Correction and Compression for Multi-view Video Using H.264 Features , 2009, ACCV.

[9]  Kun Li,et al.  Collaborative color calibration for multi-camera systems , 2011, Signal Process. Image Commun..

[10]  Raanan Fattal,et al.  Edge-avoiding wavelets and their applications , 2009, ACM Trans. Graph..

[11]  Jong-Il Park,et al.  Computer Vision - ACCV 2012 Workshops , 2012, Lecture Notes in Computer Science.

[12]  Colin Doutre,et al.  Color Correction Preprocessing for Multiview Video Coding , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Sing Bing Kang,et al.  Stereo for Image-Based Rendering using Image Over-Segmentation , 2007, International Journal of Computer Vision.

[14]  B. Funt,et al.  Diagonal versus affine transformations for color correction. , 2000, Journal of the Optical Society of America. A, Optics, image science, and vision.

[15]  Harry Shum,et al.  A real-time image-based rendering and compression system with Kinect depth camera , 2014, 2014 19th International Conference on Digital Signal Processing.

[16]  C. Garcia,et al.  3D translational motion estimation from 2D displacements , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[17]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[18]  Michal Joachimiak,et al.  Multiview 3D video denoising in sliding 3D DCT domain , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[19]  Nicolas D. Georganas,et al.  Fast color correction using principal regions mapping in different color spaces , 2004, Real Time Imaging.

[20]  Truong Q. Nguyen,et al.  Adaptive non-local means for multiview image denoising: Searching for the right patches via a statistical approach , 2013, 2013 IEEE International Conference on Image Processing.

[21]  Greg Welch,et al.  Ensuring color consistency across multiple cameras , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[22]  Shree K. Nayar,et al.  Multiple view image denoising , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Mel W. Siegel,et al.  Just enough reality: comfortable 3-D viewing via microstereopsis , 2000, IEEE Trans. Circuits Syst. Video Technol..

[24]  Mei Yu,et al.  Fast color correction for multi-view video by modeling spatio-temporal variation , 2010, J. Vis. Commun. Image Represent..

[25]  M. Levoy,et al.  Automatic Color Calibration for Large Camera Arrays , 2005 .

[26]  P. Nasiopoulos,et al.  Correcting Sharpness Variations in Stereo Image Pairs , 2009, 2009 Conference for Visual Media Production.

[27]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..