ipProjector: A Vision-Based Framework for Unsupervised Ubiquitous Object Augmentation Using a Portable Projector-Camera System

AbstractipProjector stands for interactive portable projector which is a project trying to make a smart projection system while retaining absolute portability of the devices and setups. In this paper, an ipProjector framework is proposed for unsupervised real-time object augmentation applications. Scene-independent projector-camera geometric calibration is introduced so that the projected augmentation always appears in the desired geometry despite an unconstrained surface. Using nonintrusive projection, the system can ensure correct vision-based analysis because external appearances of the environment and object (as seen by the camera) are hardly changed by the overlaid projected content. Finally, an unsupervised visual detector is used to guarantee robustness of the detection regardless of lighting variations. Several image processing algorithms are applied in the framework, including image warping using perspective transformation, color conversion using the nearest-neighbor search algorithm, and multiscale visual detection. Based on a small-scale laboratory setting, experiments were conducted to evaluate the proposed approaches, and one example was built to prove the proposed concept in object augmentation.

[1]  Markus H. Gross,et al.  Embedding imperceptible patterns into projected images for simultaneous acquisition and display , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[2]  Darius Burschka,et al.  Illumination-invariant image-based novelty detection in a cognitive mobile robot's environment , 2010, 2010 IEEE International Conference on Robotics and Automation.

[3]  Narendra Ahuja,et al.  Real-time O(1) bilateral filtering , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Greg Welch,et al.  The office of the future: a unified approach to image-based modeling and spatially immersive displays , 1998, SIGGRAPH.

[5]  Bernt Schiele,et al.  Cooperative Augmentation of Smart Objects with Projector-Camera Systems , 2007, UbiComp.

[6]  Mark Fiala,et al.  Automatic Projector Calibration Using Self-Identifying Patterns , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[7]  Atsushi Shimada,et al.  Object Detection under Varying Illumination Based on Adaptive Background Modeling Considering Spatial Locality , 2009, PSIVT.

[8]  Srinivasa G. Narasimhan,et al.  Illustrating motion through DLP photography , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[9]  Shree K. Nayar,et al.  A Projector-Camera System with Real-Time Photometric Adaptation for Dynamic Environments , 2005, CVPR.

[10]  James L. Crowley,et al.  Projecting Rectified Images in an Augmented Environment , 2003 .

[11]  Ramesh Raskar,et al.  Seamless Multi-Projector Display on Curved Screens , 2003, IPT/EGVE.

[12]  Jingyi Yu,et al.  Catadioptric projectors , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Paul A. Beardsley,et al.  A Handheld Projector Supported by Computer Vision , 2006, ACCV.

[14]  Zhenjiang Miao,et al.  Extraction of High-Resolution Face Image from Low-Resolution and Variant Illumination  Video Sequences , 2008, 2008 Congress on Image and Signal Processing.

[15]  Johannes Schöning,et al.  Map torchlight: a mobile augmented reality camera projector unit , 2009, CHI Extended Abstracts.

[16]  Horst Bischof,et al.  Classifier grids for robust adaptive object detection , 2009, CVPR.

[17]  Henry Fuchs,et al.  Real-Time Projector Tracking on Complex Geometry Using Ordinary Imagery , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Paul A. Beardsley,et al.  A self-correcting projector , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[19]  Scott E. Hudson,et al.  Moveable interactive projected displays using projector based tracking , 2005, UIST.

[20]  Pattie Maes,et al.  SixthSense: a wearable gestural interface , 2009, SIGGRAPH ASIA Art Gallery & Emerging Technologies.

[21]  Ramesh Raskar,et al.  A Low-Cost Projector Mosaic with Fast Registration , 2002 .

[22]  David G. Lowe,et al.  Shape indexing using approximate nearest-neighbour search in high-dimensional spaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Markus H. Gross,et al.  Scalable 3D video of dynamic scenes , 2005, The Visual Computer.

[24]  Markus H. Gross,et al.  Interactive environment-aware display bubbles , 2006, UIST.

[25]  Gudrun Klinker,et al.  Using laser projectors for augmented reality , 2008, VRST '08.

[26]  Xiang Cao,et al.  Interacting with dynamically defined information spaces using a handheld projector and a pen , 2006, UIST.

[27]  Shuntaro Yamazaki,et al.  Temporal Dithering of Illumination for Fast Active Vision , 2008, ECCV.

[28]  Oliver Bimber,et al.  Dynamic Adaptation of Projected Imperceptible Codes , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[29]  Jong-Il Park,et al.  Content Adaptive Embedding of Complementary Patterns for Nonintrusive Direct-Projected Augmented Reality , 2007, HCI.

[30]  Rahul Sukthankar,et al.  Smarter Presentations: Exploiting Homography in Camera-Projector Systems , 2001, ICCV.

[31]  Antonio Krüger,et al.  Shelftorchlight: Augmenting a shelf using a camera projector unit , 2010 .

[32]  Tieniu Tan,et al.  An Illumination Invariant Change Detection Algorithm , 2002 .

[33]  Weibin Sun,et al.  Robust checkerboard recognition for efficient nonplanar geometry registration in projector-camera systems , 2008, PROCAMS '08.

[34]  Nicolai Petkov,et al.  Comparison of texture features based on Gabor filters , 2002, IEEE Trans. Image Process..