Performance Characteristics of a Camera-Based Tangible Input Device for Manipulation of 3D Information

This paper describes a prototype tangible six degree of freedom (6 DoF) input device that is inexpensive and intuitive to use: a cube with colored corners of specific shapes, tracked by a single camera, with pose estimated in real time. A tracking and automatic color adjustment system are designed so that the device can work robustly with noisy surroundings and is invariant to changes in lighting and background noise. A system evaluation shows good performance for both refresh (above 60 FPS on average) and accuracy of pose estimation (average angular error of about 1). A user study of 3D rotation tasks shows that the device outperforms other 6 DoF input devices used in a similar desktop environment. The device has the potential to facilitate interactive applications such as games as well as viewing 3D information.

[1]  Jovan Popović,et al.  Real-time hand-tracking with a color glove , 2009, SIGGRAPH 2009.

[2]  Jurriaan D. Mulder,et al.  Spatial input device structure and bimanual object manipulation in virtual environments , 2006, VRST '06.

[3]  Jianliang Tang,et al.  Complete Solution Classification for the Perspective-Three-Point Problem , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Michael Rohs,et al.  Hoverflow: exploring around-device interaction with IR distance sensors , 2009, Mobile HCI.

[5]  Albrecht Schmidt,et al.  A Display Cube as a Tangible User Interface , 2005 .

[6]  Colin Ware,et al.  Frames of reference in virtual object rotation , 2004, APGV '04.

[7]  Jeff Rose,et al.  Rotating virtual objects with real handles , 1999, TCHI.

[8]  Jeremy R. Cooperstock,et al.  Evaluation of Docking Task Performance Using Mid-air Interaction Techniques , 2015, SUI.

[9]  Shumin Zhai,et al.  User performance in relation to 3D input device design , 1998, COMG.

[10]  Hiroshi Ishii,et al.  Beyond: collapsible input device for direct 3D manipulation beyond the screen , 2010, UIST '10.

[11]  Michael Goesele,et al.  Back to the Future: Learning Shape Models from 3D CAD Data , 2010, BMVC.

[12]  Sidney S. Fels,et al.  pCubee: a perspective-corrected handheld cubic display , 2010, CHI.

[13]  Shumin Zhai,et al.  Human Performance in Six Degree of Freedom Input Control , 2002 .

[14]  Alexandre R. J. François,et al.  Adaptive Color Background Modeling for Real-Time Segmentation of Video Streams* , 1999 .

[15]  Nguyen-Thong Dang,et al.  Influence of Orientation Offset between Control and Display Space on User Performance during the Rotation of 3D Objects , 2009, EGVE/ICAT/EuroVR.

[16]  Benjamin Salem,et al.  InterCUBE: A Study into Merging Action and Interaction Spaces , 2007, INTERACT.

[17]  Paul Mason,et al.  MagicMouse: an inexpensive 6-degree-of-freedom mouse , 2003, GRAPHITE '03.

[18]  Vincent Lepetit,et al.  Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes , 2012, ACCV.

[19]  Jean-Baptiste de la Rivière,et al.  CubTile: a multi-touch cubic interface , 2008, VRST '08.

[20]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[21]  Cordelia Schmid,et al.  Multi-view object class detection with a 3D geometric model , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[23]  Joseph J. LaViola,et al.  An Exploration of Non-Isomorphic 3D Rotation in Surround Screen Virtual Environments , 2007, 2007 IEEE Symposium on 3D User Interfaces.

[24]  Bernd Fröhlich,et al.  The cubic mouse: a new device for three-dimensional input , 2000, CHI.

[25]  Jae Yeol Lee,et al.  CAPTIVE: a cube with augmented physical tools , 2014, CHI Extended Abstracts.

[26]  Richard Szeliski,et al.  The VideoMouse: a camera-based multi-degree-of-freedom input device , 1999, UIST '99.

[27]  Joe Tullio,et al.  Usability analysis of 3D rotation techniques , 1997, UIST '97.

[28]  Ivan Poupyrev,et al.  Non-isomorphic 3D rotational techniques , 2000, CHI.

[29]  Abigail Sellen,et al.  A study in interactive 3-D rotation using 2-D control devices , 1988, SIGGRAPH.

[30]  Shahram Izadi,et al.  SideSight: multi-"touch" interaction around small devices , 2008, UIST '08.

[31]  Joze Guna,et al.  An Analysis of the Precision and Reliability of the Leap Motion Sensor and Its Suitability for Static and Dynamic Tracking , 2014, Sensors.

[32]  A. Raftery.  Bayesian Model Selection in Social Research , 1995 .