AR replay in a small workspace

We propose “AR replay”, a framework to record the working scene including a tutor's action in a small workspace, and then replay the tutor's action in front of a learner's view in an AR fashion (Figure.1). This framework uses one RGB-D camera for recording and replaying. On learning a task in a small workspace, when a tutor cannot be in the workspace, it is useful for a learner to check the action of the tutor by a video which was taken in advance in the same workspace. If the video can be replayed in an AR fashion, it will be more useful. We propose a new “AR replay” method by using one RGB-D camera. In our “AR replay”, the action of tutor is aligned to the right place and the learner can check the action from various viewpoints. The action is shown as 3D dynamic shape with color and it is aligned to the workspace by the static geometric clues in the workspace. Since we expect the RGB-D camera is maneuvered to frame the interaction between the tutor and the static workspace environment, we assume the demand of changing viewpoint from the original recorded camera viewpoint is limited to some extent on checking the “AR replay”. Our preliminary experimental system can acquire the 3D shapes about tutor's action and the workspace environment. Moreover, this system can produce the “AR replay” on a video see-through display, with which a learner can shift the viewpoint from the original path of the RGB-D camera in order to have the better view of the interaction between the tutor and the static workspace environment.

[1]  H. Fuchs,et al.  A First Look at a Telepresence System with Room-Sized Real-Time 3 D Capture and Life-Sized Tracked Display Wall , 2011 .

[2]  Steven K. Feiner,et al.  Augmented reality in the psychomotor phase of a procedural task , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[3]  Xiaojun Wu,et al.  Real-time 3D shape reconstruction, dynamic 3D mesh deformation, and high fidelity visualization for 3D video , 2004, Comput. Vis. Image Underst..

[4]  Henry Fuchs,et al.  Real-time volumetric 3D capture of room-sized scenes for telepresence , 2012, 2012 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[5]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[6]  Hideo Saito,et al.  Task support system by displaying instructional video onto AR workspace , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[7]  Horst Bischof,et al.  Interactive 4D overview and detail visualization in augmented reality , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[8]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Richard Szeliski,et al.  Reconstructing building interiors from images , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[10]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[11]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[12]  I. Kitahara,et al.  Live mixed-reality 3D video in soccer stadium , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[13]  Henry Fuchs,et al.  Encumbrance-free telepresence system with real-time 3D capture and display using commodity depth cameras , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[14]  Tomoya Ishikawa,et al.  In-Situ 3D Indoor Modeler with a Camera and Self-contained Sensors , 2009, HCI.

[15]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[16]  Diego Borro,et al.  Providing guidance for maintenance operations using automatic markerless augmented reality system , 2011, ISMAR.

[17]  Marc Levoy,et al.  Efficient variants of the ICP algorithm , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.