Realizing Half-Diminished Reality from video stream of manipulating objects

When we watch a video, in which human hands manipulate objects, these hands may obscure some parts of those objects. We are willing to make clear how the objects are manipulated by making the image of hands semi-transparent, and showing the complete images of the hands and the object. By carefully choosing a Half-Diminished Reality method, this paper proposes a method that can process the video in real time and verifies that the proposed method works well.

[1]  Zhengyou Zhang,et al.  Real-Time Whiteboard Capture and Processing Using a Video Camera for Remote Collaboration , 2007, IEEE Transactions on Multimedia.

[2]  Tao Xiang,et al.  Background Subtraction with DirichletProcess Mixture Models , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Wolfgang Broll,et al.  PixMix: A real-time approach to high-quality Diminished Reality , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[4]  Atsushi Yamashita,et al.  Half-diminished reality image using three RGB-D sensors for remote control robots , 2014, 2014 IEEE International Symposium on Safety, Security, and Rescue Robotics (2014).

[5]  Wen Gao,et al.  Nonparametric Background Generation , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[6]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[7]  Hideo Saito,et al.  Generation of see-through baseball movie from multi-camera views , 2010, 2010 IEEE International Workshop on Multimedia Signal Processing.

[8]  Z. Zivkovic.  Improved adaptive Gaussian mixture model for background subtraction , 2004, ICPR 2004.

[9]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interaction , 1999, ICVS.

[10]  Massimo Piccardi,et al.  Background subtraction techniques: a review , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[11]  H. Saito,et al.  Diminished Reality using Multiple Handheld Cameras , 2007 .

[12]  Mark Billinghurst,et al.  Interaction With Partially Transparent Hands And Objects , 2005, AUIC.

[13]  Vincent Lepetit,et al.  A semi-automatic method for resolving occlusion in augmented reality , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[14]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[15]  Bohyung Han,et al.  SEQUENTIAL KERNEL DENSITY APPROXIMATION THROUGH MODE PROPAGATION: APPLICATIONS TO BACKGROUND MODELING , 2004 .

[16]  Nassir Navab,et al.  Multiview paraperspective projection model for diminished reality , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[17]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[18]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  T. Xiang.  Background Subtraction with Dirichlet Process Mixture Models , 2013 .