A LINE-MOD-based markerless tracking approachfor AR applications

Markerless tracking is still a very challenging problem in augmented reality applications, especially the real elements are textureless. In this paper, we proposed a model-based method to tackle the markerless tracking problem. Motivated by LINE-MOD algorithm, one of the state-of-the-art object detection methods, and multiview-based 3D model retrieval approach, we built a camera tracking system utilizing image retrieval. In the off-line training stage, 3D models were used to generate templates automatically. To estimate the camera pose accurately in the online matching stage, LINE-MOD was adapted into a scale-invariant descriptor using depth information obtained from Softkinetic, and an interpolation method combined with other mathematical calculations was used for camera pose refinement. The experimental result shows that the proposed method is fast and robust for markerless tracking in augmented reality environment; the tracking accuracy is much closer to that of ARToolKit markers.

[1]  Naokazu Yokoya,et al.  Geometric and photometric registration for real-time augmented reality , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[2]  Dieter Schmalstieg,et al.  Real-Time Detection and Tracking for Augmented Reality on Mobile Phones , 2010, IEEE Transactions on Visualization and Computer Graphics.

[3]  Henrik I. Christensen,et al.  Real-time 3D model-based tracking using edge and keypoint features for robotic manipulation , 2010, 2010 IEEE International Conference on Robotics and Automation.

[4]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[5]  Vincent Lepetit,et al.  Texture-less object tracking with online training using an RGB-D camera , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[6]  Silvio Savarese,et al.  A coarse-to-fine model for 3D pose estimation and sub-category recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Eyal Ofek,et al.  SnapToReality: Aligning Augmented Reality to the Real World , 2016, CHI.

[8]  Jiri Matas,et al.  P-N learning: Bootstrapping binary classifiers by structural constraints , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Ralf Dörner,et al.  Accuracy in optical tracking with fiducial markers: an accuracy function for ARToolKit , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[10]  Holger Regenbrecht,et al.  Interactive multi-marker calibration for augmented reality applications , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[11]  Somkiat Wangsiripitak,et al.  Reducing mismatching under time-pressure by reasoning about visibility and occlusion , 2010, BMVC.

[12]  Jae Yeol Lee,et al.  Context-aware 3D visualization and collaboration services for ubiquitous cars using augmented reality , 2008 .

[13]  Horst Bischof,et al.  Learning Features for Tracking , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Jae Yeol Lee,et al.  Hand gesture-based tangible interactions for manipulating virtual objects in a mixed reality environment , 2010 .

[15]  Jun Hong,et al.  Automated positioning of 3D virtual scene in AR-based assembly and disassembly guiding system , 2015 .

[16]  Vincent Lepetit,et al.  Dominant orientation templates for real-time detection of texture-less objects , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Roberto Cipolla,et al.  Real-Time Visual Tracking of Complex Structures , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Daniel Cremers,et al.  LSD-SLAM: Large-Scale Direct Monocular SLAM , 2014, ECCV.

[19]  Vincent Lepetit,et al.  Learning descriptors for object recognition and 3D pose estimation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[21]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[22]  Vincent Lepetit,et al.  Gradient Response Maps for Real-Time Detection of Textureless Objects , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Andrew Y. C. Nee,et al.  An authorable context-aware augmented reality system to assist the maintenance technicians , 2012 .

[24]  Jun Rekimoto,et al.  Matrix: a realtime object identification and registration method for augmented reality , 1998, Proceedings. 3rd Asia Pacific Computer Human Interaction (Cat. No.98EX110).

[25]  Vincent Lepetit,et al.  Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes , 2012, ACCV.

[26]  Wolfgang Friedrich,et al.  ARVIKA-augmented reality for development, production and service , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[27]  Jae Yeol Lee,et al.  AR/RP-based tangible interactions for collaborative design evaluation of digital products , 2009 .

[28]  Sinisa Todorovic,et al.  From contours to 3D object detection and pose estimation , 2011, 2011 International Conference on Computer Vision.

[29]  Dieter Fox,et al.  DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Soh-Khim Ong,et al.  Augmented reality aided interactive manual assembly design , 2013, The International Journal of Advanced Manufacturing Technology.

[31]  Dieter Schmalstieg,et al.  Multiple target detection and tracking with guaranteed framerates on mobile phones , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[32]  Ming Ouhyoung,et al.  On Visual Similarity Based 3D Model Retrieval , 2003, Comput. Graph. Forum.

[33]  Jae Wook Jeon,et al.  Determination of 3D object pose in point cloud with CAD model , 2015, 2015 21st Korea-Japan Joint Workshop on Frontiers of Computer Vision (FCV).

[34]  Andrew Y. C. Nee,et al.  Orientation planning of robot end-effector using augmented reality , 2013 .

[35]  Andrew Y. C. Nee,et al.  Augmented reality applications in design and manufacturing , 2012 .

[36]  Guido Bugmann,et al.  Vision-based urban navigation procedures for verbally instructed robots , 2005, Robotics Auton. Syst..

[37]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.