Markerless tracking using Polar Correlation of camera optical flow

We present a novel, real-time, markerless vision-based tracking system, employing a rigid orthogonal configuration of two pairs of opposing cameras. Our system uses optical flow over sparse features to overcome the limitation of vision-based systems that require markers or a pre-loaded model of the physical environment. We show how opposing cameras enable cancellation of common components of optical flow leading to an efficient tracking algorithm. Experiments comparing our device with an electromagnetic tracker show that its average tracking accuracy is 80% over 185 frames, and it is able to track large range motions even in outdoor settings.

[1]  Hongdong Li,et al.  Motion estimation for multi-camera systems using global optimization , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  R. Chellappa,et al.  Markerless Motion Capture using Multiple Cameras , 2005, Computer Vision for Interactive and Intelligent Environment (CVIIE'05).

[3]  Greg Welch,et al.  High-Performance Wide-Area Optical Tracking: The HiBall Tracking System , 2001, Presence: Teleoperators & Virtual Environments.

[4]  Nick Barnes,et al.  Estimation of the Epipole using Optical Flow at Antipodal Points , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[5]  James W. Davis,et al.  A perceptual user interface for recognizing head gesture acknowledgements , 2001, PUI '01.

[6]  Steven S. Beauchemin,et al.  The computation of optical flow , 1995, CSUR.

[7]  Hongdong Li,et al.  Motion Estimation for Nonoverlapping Multicamera Rigs: Linear Algebraic and L∞ Geometric Solutions , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Éric Marchand,et al.  Real-time markerless tracking for augmented reality: the virtual visual servoing framework , 2006, IEEE Transactions on Visualization and Computer Graphics.

[9]  Robert Pless.  Camera cluster in motion: motion estimation for generalized camera designs , 2004, IEEE Robotics & Automation Magazine.

[10]  Edward Cutrell,et al.  FlowMouse: A Computer Vision-Based Pointing and Gesture Input Device , 2005, INTERACT.

[11]  Daphna Weinshall,et al.  A computer vision system for on-screen item selection by finger pointing , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[12]  P. Fergus,et al.  Perceptual User Interface as Games Controller , 2008, 2008 5th IEEE Consumer Communications and Networking Conference.

[13]  Perttu Hämäläinen,et al.  A Computer Vision and Hearing Based User Interface for a Computer Game for Children , 2002, User Interfaces for All.

[14]  Hongdong Li,et al.  Five-Point Motion Estimation Made Easy , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[15]  Allan D. Jepson,et al.  Linear subspace methods for recovering translational direction , 1994 .

[16]  Yi-Ping Hung,et al.  Ego-motion estimation using optical flow fields observed from multiple cameras , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Jan-Michael Frahm,et al.  Robust 6DOF Motion Estimation for Non-Overlapping, Multi-Camera Systems , 2008, 2008 IEEE Workshop on Applications of Computer Vision.

[18]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[19]  H. C. Longuet-Higgins,et al.  The interpretation of a moving retinal image , 1980, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[20]  Nick Barnes,et al.  Directions of egomotion from antipodal points , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[22]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[23]  Frank Dellaert,et al.  A multi-camera 6-DOF pose tracker , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[24]  Perttu Hämäläinen,et al.  Children's intuitive gestures in vision-based action games , 2005, CACM.

[25]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Jakub Segen,et al.  Gesture VR: vision-based 3D hand interace for spatial interaction , 1998, MULTIMEDIA '98.

[27]  Eero P. Simoncelli,et al.  Linear Structure From Motion , 1994 .

[28]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[29]  Matthew Turk,et al.  Moving from GUIs to PUIs , 1998 .

[30]  H. C. Longuet-Higgins,et al.  A computer algorithm for reconstructing a scene from two projections , 1981, Nature.

[31]  Olivier Stasse,et al.  MonoSLAM: Real-Time Single Camera SLAM , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Paul A. Beardsley,et al.  Computer Vision for Interactive Computer Graphics , 1998, IEEE Computer Graphics and Applications.

[33]  Lei Chen,et al.  Robust and fast similarity search for moving object trajectories , 2005, SIGMOD '05.

[34]  Hongdong Li,et al.  A linear approach to motion estimation using generalized camera models , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Geanbry Demming.  Sony EyeToy TM: Developing Mental Models for 3-D Interaction in a 2-D Gaming Environment , 2004, APCHI.

[36]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[37]  Frank Dellaert,et al.  Visual SLAM with a Multi-Camera Rig , 2006 .

[38]  Thomas B. Moeslund,et al.  Vision-Based User Interface for Interacting with a Virtual Environment , 2000 .

[39]  Mubarak Shah,et al.  A virtual 3D blackboard: 3D finger tracking using a single camera , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[40]  Berthold K. P. Horn,et al.  Passive navigation , 1982, Comput. Vis. Graph. Image Process..

[41]  Gary Bradski,et al.  Computer Vision Face Tracking For Use in a Perceptual User Interface , 1998 .

[42]  Greg Welch,et al.  Motion Tracking: No Silver Bullet, but a Respectable Arsenal , 2002, IEEE Computer Graphics and Applications.

[43]  J.-Y. Bouguet,et al.  Pyramidal implementation of the lucas kanade feature tracker , 1999 .

[44]  Takeo Kanade,et al.  Motion estimation using multiple non-overlapping cameras for small unmanned aerial vehicles , 2008, 2008 IEEE International Conference on Robotics and Automation.

[45]  R. Koch,et al.  Markerless Image-based 3 D Tracking for Real-time Augmented Reality Applications , 2005 .