Issues in Control of a Robotic Spatial Augmented Reality System

A robotic spatial augmented reality (RSAR) system combines a robotics technology with a spatial augmented reality system (SAR) where cameras are used to recognize real objects and projectors augment information and user interface directly on the surface of the recognized objects, rather than relying on handheld display devices. Moreover, a robotic module is actively used to discover and utilize the context of users and environments. The control of a RSAR system involves several issues from different technical fields such as classical inverse kinematics of motors where projector-camera pairs are mounted, inverse projection problems to find appropriate internal/external parameters of projectors and cameras, and image warping in graphics pipeline to compensate the kinematic constraints. In this paper, we investigate various control issues related to a RSAR system and propose basic approaches to handle them, specially focused on the prototype RSAR system developed in ETRI.

[1]  Pattie Maes,et al.  SixthSense: a wearable gestural interface , 2009, SIGGRAPH ASIA Art Gallery & Emerging Technologies.

[2]  Eun-Sun Cho,et al.  Future robotic computer: A new type of computing device with robotic functions , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[3]  Ramesh Raskar,et al.  Projectors for graphics , 2008, SIGGRAPH '08.

[4]  Ronald Azuma,et al.  Recent Advances in Augmented Reality , 2001, IEEE Computer Graphics and Applications.

[5]  Brian P. Bailey,et al.  Build your world and play in it: Interacting with surface particles on complex objects , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[6]  Dieter Fox,et al.  A large-scale hierarchical multi-view RGB-D object dataset , 2011, 2011 IEEE International Conference on Robotics and Automation.

[7]  Pattie Maes,et al.  LuminAR: portable robotic augmented reality interface design and prototype , 2010, UIST '10.

[8]  K. Sato,et al.  Free-form Shape Design System using Stereoscopic Projector - HYPERREAL 2.0 , 2006, 2006 SICE-ICASE International Joint Conference.

[9]  Hrvoje Benko,et al.  Combining multiple depth cameras and projectors for interactions on, above and between surfaces , 2010, UIST.

[10]  Mark Fiala,et al.  Automatic Projector Calibration Using Self-Identifying Patterns , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops.

[11]  Gwi-Tae Park,et al.  Anamorphosis Projection by Ubiquitous Display in Intelligent Space , 2009, HCI.

[12]  Tobias Höllerer,et al.  The City of Sights: Design, construction, and measurement of an Augmented Reality stage set , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[13]  Ruigang Yang,et al.  PixelFlex: a reconfigurable multi-projector display system , 2001, Proceedings Visualization, 2001. VIS '01..

[14]  Hrvoje Benko,et al.  SketchSpace: designing interactive behaviors with passive materials , 2011, CHI EA '11.

[15]  Greg Welch,et al.  Table-top spatially-augmented realty: bringing physical models to life with projected imagery , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[16]  Soon-Yong Park,et al.  Active Calibration of Camera-Projector Systems Based on Planar Homography , 2010, 2010 20th International Conference on Pattern Recognition.

[17]  T. Yoshikawa,et al.  Task-Priority Based Redundancy Control of Robot Manipulators , 1987 .

[18]  Rolf R. Hainich.  The End of Hardware, 3rd Edition: Augmented Reality and Beyond , 2009 .