3D Ground-Truth Systems for Object/Human Recognition and Tracking

We have been researching three dimensional (3D) ground-truth systems for performance evaluation of vision and perception systems in the fields of smart manufacturing and robot safety. In this paper we first present an overview of different systems that have been used to provide ground-truth (GT) measurements and then we discuss the advantages of physically-sensed ground-truth systems for our applications. Then we discuss in detail the three ground- truth systems that we have used in our experiments: ultra wide-band, indoor GPS, and a camera-based motion capture system. Finally, we discuss three different perception-evaluation experiments where we have used these GT systems.

[1]  Robert Schmitt,et al.  Performance evaluation of iGPS for industrial applications , 2010, 2010 International Conference on Indoor Positioning and Indoor Navigation.

[2]  Roger D. Eastman,et al.  Ground Truth Systems for Object Recognition and Tracking , 2013 .

[3]  Ignas Niemegeers,et al.  A survey of indoor positioning systems for wireless personal networks , 2009, IEEE Communications Surveys & Tutorials.

[4]  Michael J. Black,et al.  A Quantitative Evaluation of Video-based 3D Person Tracking , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[5]  Remco C. Veltkamp,et al.  UMPM benchmark: A multi-person dataset with synchronized video and motion capture data for evaluation of articulated human motion and interaction , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[6]  Fiorenzo Franceschini,et al.  Experimental comparison of dynamic tracking performance of iGPS and laser tracker , 2011 .

[7]  Tom Chau,et al.  A Review of Indoor Localization Technologies: towards Navigational Assistance for Topographical Disorientation , 2010 .

[8]  Sandor S. Szabo,et al.  A Testbed for Evaluation of Speed and Separation Monitoring in a Human Robot Collaborative Environment , 2012 .

[9]  David Mihalcik,et al.  The Design and Implementation of ViPER , 2005 .

[10]  Roger V. Bostelman,et al.  Safe Control of Manufacturing Vehicles Research Towards Standard Test Methods | NIST , 2012 .

[11]  Jing Liu,et al.  Survey of Wireless Indoor Positioning Techniques and Systems , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[12]  Adam Jacoff,et al.  Performance measurements for evaluating static and dynamic multiple human detection and tracking systems in unstructured environments , 2009, PerMIS.

[13]  Michael J. Black,et al.  HumanEva: Synchronized Video and Motion Capture Dataset for Evaluation of Articulated Human Motion , 2006 .

[14]  Raj Madhavan,et al.  Proceedings of the 8th Workshop on Performance Metrics for Intelligent Systems, PerMIS 2008, Gaithersburg, Maryland, USA, August 19-21, 2008 , 2008, PerMIS.

[15]  Emilia Villani,et al.  Analysis of the indoor GPS system as feedback for the robotic alignment of fuselages using laser radar measurements as comparison , 2012 .

[16]  Kamel S. Saidi,et al.  Model-Based 3D Tracking in Multiple Camera Views , 2010 .

[17]  Michael J. Black,et al.  HumanEva: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion , 2010, International Journal of Computer Vision.

[18]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[19]  Sebastian Tilch,et al.  Survey of optical indoor positioning systems , 2011, 2011 International Conference on Indoor Positioning and Indoor Navigation.

[20]  Roger Bostelman,et al.  Standard test procedures and metrics development for automated guided vehicle safety standards , 2012, PerMIS.

[21]  Klaithem Al Nuaimi,et al.  A survey of indoor positioning systems and algorithms , 2011, 2011 International Conference on Innovations in Information Technology.

[22]  Claudia Depenthal,et al.  Path tracking with IGPS , 2010, 2010 International Conference on Indoor Positioning and Indoor Navigation.