Diminished Reality for Privacy Protection by Hiding Pedestrians in Motion Image Sequences Using Structure from Motion

We present a method for generating images in which people are hidden from image sequences taken with a hand-held camera. Our method is basically used for privacy protection of people whose images are unintentionally captured in image sequences. We hide people from images by reconstructing a 3D model of background and projecting it to 2D images. By detecting the area in which people are present beforehand, we can reconstruct a 3D model of the background without people. In the experiment, We compare our method with some conventional approaches for diminished reality.

[1]  Tomas Pajdla,et al.  Exploiting Visibility Information in Surface Reconstruction to Preserve Weakly Supported Surfaces , 2014, International scholarly research notices.

[2]  Jan Kautz,et al.  Background Inpainting for Videos with Dynamic Objects and a Free-Moving Camera , 2012, ECCV.

[3]  Jean-Philippe Pons,et al.  High Accuracy and Visibility-Consistent Dense Multiview Stereo , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[5]  Richard Szeliski,et al.  Towards Internet-scale multi-view stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[7]  Hideo Saito,et al.  Diminished Reality for Hiding a Pedestrian using Hand-Held Camera , 2015, 2015 IEEE International Symposium on Mixed and Augmented Reality Workshops.

[8]  Serge J. Belongie,et al.  Removing pedestrians from Google street view images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[9]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[10]  Naokazu Yokoya,et al.  Background Estimation for a Single Omnidirectional Image Sequence Captured with a Moving Camera , 2014, IPSJ Trans. Comput. Vis. Appl..

[11]  Marc Pollefeys,et al.  Modeling Dynamic Scenes Recorded with Freely Moving Cameras , 2010, ACCV.

[12]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[13]  Naokazu Yokoya,et al.  Diminished Reality Based on Image Inpainting Considering Background Geometry , 2016, IEEE Transactions on Visualization and Computer Graphics.

[14]  Jean-Yves Guillemaut,et al.  General Dynamic Scene Reconstruction from Multiple View Video , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Changchang Wu,et al.  Towards Linear-Time Incremental Structure from Motion , 2013, 2013 International Conference on 3D Vision.

[16]  Steven M. Seitz,et al.  Multicore bundle adjustment , 2011, CVPR 2011.

[17]  H. Saito,et al.  Diminished Reality using Multiple Handheld Cameras , 2007 .

[18]  Alexandru Telea,et al.  An Image Inpainting Technique Based on the Fast Marching Method , 2004, J. Graphics, GPU, & Game Tools.

[19]  Zhuwen Li,et al.  Diminished reality using appearance and 3D geometry of internet photo collections , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[20]  Wolfgang Broll,et al.  PixMix: A real-time approach to high-quality Diminished Reality , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[21]  Mark Fiala,et al.  ARTag, a fiducial marker system using digital techniques , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).