Video Deconfounding: Hearing-Aid Inspired Video Enhancement

We introduce a set of techniques for selective amplification of video subjects that are largely hidden from view by confounding light paths, such as daylight reflections from tinted automobile windows, reflections from windows and screens when imaging outdoor scenes at night from indoors, and reflected light from fluid surfaces. In these situations, the subject of interest is represented by only a small fraction of the light being captured at each pixel. We show that enhancement approaches commonly employed for selective amplification of speech in audio applications can serve as the basis for selective amplification of hidden objects in video streams.

[1]  Frédo Durand,et al.  Reflection removal using ghosting cues , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  William T. Freeman,et al.  A computational approach for obstruction-free photography , 2015, ACM Trans. Graph..

[3]  Deepjoy Das,et al.  Implementation And Performance Evaluation Of Background Subtraction Algorithms , 2014, ArXiv.

[4]  Frédo Durand,et al.  Video magnification in presence of large motions , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[6]  B. Shoushtarian.  A PRACTICAL APPROACH TO REAL-TIME DYNAMIC BACKGROUND GENERATION BASED ON A TEMPORAL MEDIAN FILTER , 2003 .

[7]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[8]  Michael Goesele,et al.  Image-based rendering for scenes with reflections , 2012, ACM Trans. Graph..

[9]  Richard Szeliski,et al.  Layer extraction from multiple images containing reflections and transparency , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[10]  Xiaochun Cao,et al.  Robust Separation of Reflection from Multiple Images , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.