See-through Image Enhancement through Sensor Fusion

Many hardware designs have been developed to allow a camera to be placed optically directly behind the screen. The purpose of such setups is to enable two-way video teleconferencing that maintains eye-contact. However, the image from the see-through camera usually exhibits a number of imaging artifacts such as low signal to noise ratio, incorrect color balance, and lost of details. We develop a novel image enhancement framework that utilizes an auxiliary color+depth camera that is mounted on the side of the screen. By fusing the information from both cameras, we are able to significantly improve the quality of the see-through image. Experimental results have demonstrated that our fusion method compares favorably against traditional image enhancement/warping methods that uses only a single image.

[1]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[2]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[3]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  R. Park,et al.  Introduction to the Science of Sociology , 1921 .

[5]  Dani Lischinski,et al.  Gradient Domain High Dynamic Range Compression , 2023 .

[6]  Michael F. Cohen,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[7]  Ming Zhang,et al.  Multiresolution Bilateral Filtering for Image Denoising , 2008, IEEE Transactions on Image Processing.

[8]  Martin Vetterli,et al.  Spatially adaptive wavelet thresholding with context modeling for image denoising , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[9]  Tat-Jen Cham,et al.  Analogous view transfer for gaze correction in video sequences , 2002, 7th International Conference on Control, Automation, Robotics and Vision, 2002. ICARCV 2002..

[10]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[11]  Jean-Michel Morel,et al.  Nonlocal Image and Movie Denoising , 2008, International Journal of Computer Vision.

[12]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[13]  Andrew D. Wilson.  TouchLight: an imaging touch screen and display for gesture-based interaction , 2004, ICMI '04.

[14]  Ingemar J. Cox,et al.  Teleconferencing eye contract using a virtual camera , 1993, INTERCHI Adjunct Proceedings.

[15]  Ingvar Claesson,et al.  Face Detection using Local SMQT Features and Split up Snow Classifier , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[16]  Ruigang Yang,et al.  Eye Gaze Correction with Stereovision for Video-Teleconferencing , 2002, ECCV.

[17]  Vladimir Vezhnevets,et al.  A Survey on Pixel-Based Skin Color Detection Techniques , 2003 .

[18]  Ramin Samadani,et al.  ConnectBoard: A remote collaboration system that supports gaze-aware interaction and sharing , 2009, 2009 IEEE International Workshop on Multimedia Signal Processing.

[19]  Ivan W. Selesnick,et al.  Video denoising using 2D and 3D dual-tree complex wavelet transforms , 2003, SPIE Optics + Photonics.

[20]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[21]  Ruigang Yang,et al.  A novel see-through screen based on weave fabrics , 2011, 2011 IEEE International Conference on Multimedia and Expo.