DodecaPen: Accurate 6DoF Tracking of a Passive Stylus

We propose a system for real-time six degrees of freedom (6DoF) tracking of a passive stylus that achieves sub-millimeter accuracy, which is suitable for writing or drawing in mixed reality applications. Our system is particularly easy to implement, requiring only a monocular camera, a 3D printed dodecahedron, and hand-glued binary square markers. The accuracy and performance we achieve are due to model-based tracking using a calibrated model and a combination of sparse pose estimation and dense alignment. We demonstrate the system performance in terms of speed and accuracy on a number of synthetic and real datasets, showing that it can be competitive with state-of-the-art multi-camera motion capture systems. We also demonstrate several applications of the technology ranging from 2D and 3D drawing in VR to general object manipulation and board games.

[1]  John F. Hughes,et al.  Sculpting: an interactive volumetric modeling technique , 1991, SIGGRAPH.

[2]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[3]  Geehyuk Lee,et al.  IrCube tracker: an optical 6-DOF tracker based on LED directivity , 2011, UIST '11.

[4]  Vlastimil Havran,et al.  Light Chisel: 6DOF Pen Tracking , 2015, Comput. Graph. Forum.

[5]  Francisco José Madrid-Cuevas,et al.  Generation of fiducial marker dictionaries using Mixed Integer Linear Programming , 2016, Pattern Recognit..

[6]  Marc Alexa,et al.  Joint 5D Pen Input for Light Field Displays , 2015, UIST.

[7]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[8]  Vincent Lepetit,et al.  Dense Methods for Image Alignment with an Application to 3D Tracking , 2014 .

[9]  Thomas Petersen.  A Comparison of 2D-3D Pose Estimation Methods , 2008 .

[10]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[11]  Woontack Woo,et al.  An empirical evaluation of virtual hand techniques for 3D object manipulation in a tangible augmented reality environment , 2010, 2010 IEEE Symposium on 3D User Interfaces (3DUI).

[12]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[13]  Francisco José Madrid-Cuevas,et al.  Automatic generation and detection of highly reliable fiducial markers under occlusion , 2014, Pattern Recognit..

[14]  Sriram Subramanian,et al.  Multi-layer interaction for digital tables , 2006, UIST.

[15]  Ivan Poupyrev,et al.  Lumitrack: low cost, high precision, high speed tracking with projected m-sequences , 2013, UIST.

[16]  Ming-Hsuan Yang,et al.  Direct 3D pose estimation of a planar target , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[17]  Maneesh Agrawala,et al.  3D puppetry: a kinect-based interface for 3D animation , 2012, UIST.

[18]  Geehyuk Lee,et al.  The IrPen: A 6-DOF Pen for Interaction with Tablet Computers , 2014, IEEE Computer Graphics and Applications.

[19]  Javier Díaz,et al.  Real-Time Model-Based Rigid Object Pose Estimation and Tracking Combining Dense and Sparse Visual Cues , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Hiroshi Ishii,et al.  Tangible bits: towards seamless interfaces between people, bits and atoms , 1997, CHI.

[22]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[23]  Pierre Alliez,et al.  Anisotropic polygonal remeshing , 2003, ACM Trans. Graph..

[24]  Patrick Baudisch,et al.  Hover widgets: using the tracking state to extend the capabilities of pen-operated devices , 2006, CHI.

[25]  Mark Fiala,et al.  ARTag, a fiducial marker system using digital techniques , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[26]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[27]  D. Alspach.  A gaussian sum approach to the multi-target identification-tracking problem , 1975, Autom..