A Probabilistic Framework for Learning Kinematic Models of Articulated Objects

Robots operating in domestic environments generally need to interact with articulated objects, such as doors, cabinets, dishwashers or fridges. In this work, we present a novel, probabilistic framework for modeling articulated objects as kinematic graphs. Vertices in this graph correspond to object parts, while edges between them model their kinematic relationship. In particular, we present a set of parametric and non-parametric edge models and how they can robustly be estimated from noisy pose observations. We furthermore describe how to estimate the kinematic structure and how to use the learned kinematic models for pose prediction and for robotic manipulation tasks. We finally present how the learned models can be generalized to new and previously unseen objects. In various experiments using real robots with different camera systems as well as in simulation, we show that our approach is valid, accurate and efficient. Further, we demonstrate that our approach has a broad set of applications, in particular for the emerging fields of mobile manipulation and service robotics.

[1]  Kurt Konolige,et al.  Projected texture stereo , 2010, 2010 IEEE International Conference on Robotics and Automation.

[2]  Jana Kosecka,et al.  Visual door detection integrating appearance and shape cues , 2008, Robotics Auton. Syst..

[3]  Daniel Cremers,et al.  Efficient Dense Scene Flow from Sparse or Dense Stereo Data , 2008, ECCV.

[4]  Oliver Brock,et al.  Manipulating articulated objects with interactive perception , 2008, 2008 IEEE International Conference on Robotics and Automation.

[5]  John K. Tsotsos,et al.  Active Vision for Door Localization and Door Opening using Playbot: A Computer Controlled Wheelchair for People with Mobility Impairments , 2008, 2008 Canadian Conference on Computer and Robot Vision.

[6]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[7]  Advait Jain,et al.  Behavior-Based Door Opening with Equilibrium Point Control , 2009 .

[8]  Vincent Lepetit,et al.  Monocular Model-Based 3D Tracking of Rigid Objects: A Survey , 2005, Found. Trends Comput. Graph. Vis..

[9]  Craig Gotsman,et al.  Articulated Object Reconstruction and Markerless Motion Capture from Depth Video , 2008, Comput. Graph. Forum.

[10]  Evangelos E. Milios,et al.  Globally Consistent Range Scan Alignment for Environment Mapping , 1997, Auton. Robots.

[11]  D. Fox,et al.  Manipulator and Object Tracking for In Hand Model Acquisition , 2010 .

[12]  Lars Petersson,et al.  Visually guided manipulation tasks , 2002, Robotics Auton. Syst..

[13]  Vasant Honavar,et al.  Efficient Markov Network Structure Discovery using Independence Tests , 2006, SDM.

[14]  Clifford Stein,et al.  Introduction to Algorithms, 2nd edition. , 2001 .

[15]  Nico Blodow,et al.  Towards 3D Point cloud based object maps for household environments , 2008, Robotics Auton. Syst..

[16]  Wolfram Burgard,et al.  Vision-based detection for learning articulation models of cabinet doors and drawers in household environments , 2010, 2010 IEEE International Conference on Robotics and Automation.

[17]  Kurt Konolige,et al.  Calibrating a Multi-arm Multi-sensor Robot: A Bundle Adjustment Approach , 2010, ISER.

[18]  Kurt Konolige,et al.  Small Vision Systems: Hardware and Implementation , 1998 .

[19]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[20]  Rüdiger Dillmann,et al.  Combining force and visual feedback for physical interaction tasks in humanoid robots , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[21]  Neil D. Lawrence,et al.  Probabilistic Non-linear Principal Component Analysis with Gaussian Process Latent Variable Models , 2005, J. Mach. Learn. Res..

[22]  Michael Beetz,et al.  Laser-based perception for door and handle identification , 2009, 2009 International Conference on Advanced Robotics.

[23]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[24]  Advait Jain,et al.  Pulling open doors and drawers: Coordinating an omni-directional base and a compliant arm with Equilibrium Point control , 2010, 2010 IEEE International Conference on Robotics and Automation.

[25]  Wolfram Burgard,et al.  Nonlinear Constraint Network Optimization for Efficient Map Learning , 2009, IEEE Transactions on Intelligent Transportation Systems.

[26]  Jean-Jacques E. Slotine,et al.  A simple strategy for opening an unknown door , 1997, Proceedings of International Conference on Robotics and Automation.

[27]  Qiang Shen,et al.  Learning Bayesian Network Equivalence Classes with Ant Colony Optimization , 2009, J. Artif. Intell. Res..

[28]  Maxim Likhachev,et al.  Planning for autonomous door opening with a mobile manipulator , 2010, 2010 IEEE International Conference on Robotics and Automation.

[29]  David A. Forsyth,et al.  Skeletal parameter estimation from optical motion capture data , 2004, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[30]  Wisama Khalil,et al.  Model Identification , 2019, Springer Handbook of Robotics, 2nd Ed..

[31]  Shuzi Yang,et al.  Kinematic-Parameter Identification for Serial-Robot Calibration Based on POE Formula , 2010, IEEE Transactions on Robotics.

[32]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[33]  David G. Lowe,et al.  Fitting Parameterized Three-Dimensional Models to Images , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Geoffrey A. Hollinger,et al.  HERB: a home exploring robotic butler , 2010, Auton. Robots.

[35]  FuaPascal,et al.  Monocular model-based 3D tracking of rigid objects , 2005 .

[36]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[37]  Marc Pollefeys,et al.  Automatic Kinematic Chain Building from Feature Trajectories of Articulated Objects , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[38]  Kurt Konolige,et al.  Autonomous door opening and plugging in with a personal robot , 2010, 2010 IEEE International Conference on Robotics and Automation.

[39]  Ying Zheng,et al.  Contact feature extraction on a balancing manipulation platform , 2010, 2010 IEEE International Conference on Robotics and Automation.

[40]  Mark Fiala,et al.  ARTag, a fiducial marker system using digital techniques , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[41]  Oussama Khatib,et al.  Compliant Control of Multicontact and Center-of-Mass Behaviors in Humanoid Robots , 2010, IEEE Transactions on Robotics.

[42]  Siddhartha S. Srinivasa,et al.  Manipulation planning with caging grasps , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[43]  Sebastian Thrun,et al.  Bayesian Network Induction via Local Neighborhoods , 1999, NIPS.

[44]  Jörg Stückler,et al.  Improving indoor navigation of autonomous robots by an explicit representation of doors , 2010, 2010 IEEE International Conference on Robotics and Automation.

[45]  K. Dautenhahn,et al.  Care-O-bot 3 — Rationale for human-robot interaction design , 2008 .

[46]  Wolfram Burgard,et al.  Operating articulated objects based on experience , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[47]  David Maxwell Chickering,et al.  Learning Equivalence Classes of Bayesian Network Structures , 1996, UAI.

[48]  Advait Jain,et al.  Pulling open novel doors and drawers with equilibrium point control , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[49]  David Maxwell Chickering,et al.  Learning Bayesian Networks is NP-Complete , 2016, AISTATS.

[50]  Quoc V. Le,et al.  High-accuracy 3D sensing for mobile manipulation: Improving object detection and door opening , 2009, 2009 IEEE International Conference on Robotics and Automation.

[51]  Wolfram Burgard,et al.  Learning Kinematic Models for Articulated Objects , 2009, IJCAI.

[52]  Udo Frese Treemap: An O(log n) algorithm for indoor simultaneous localization and mapping , 2006, Auton. Robots.

[53]  Andrew Y. Ng,et al.  Probabilistic Mobile Manipulation in Dynamic Environments, with Application to Opening Doors , 2007, IJCAI.

[54]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[55]  Richard S. Zemel,et al.  Unsupervised Learning of Skeletons from Motion , 2008, ECCV.

[56]  David Nistér,et al.  Preemptive RANSAC for live structure and motion estimation , 2005, Machine Vision and Applications.

[57]  Bodo Rosenhahn,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence Combined Region-and Motion-based 3d Tracking of Rigid and Articulated Objects , 2022 .

[58]  Trevor Darrell,et al.  Recovering Articulated Model Topology from Observed Rigid Motion , 2002, NIPS.

[59]  Alex Zelinsky,et al.  Learning OpenCV---Computer Vision with the OpenCV Library (Bradski, G.R. et al.; 2008)[On the Shelf] , 2009, IEEE Robotics & Automation Magazine.

[60]  Sebastian Thrun,et al.  Detecting and modeling doors with mobile robots , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[61]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[62]  Ashutosh Saxena,et al.  Learning to open new doors , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[63]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[64]  Cyrill Stachniss,et al.  Hierarchical optimization on manifolds for online 2D and 3D mapping , 2010, 2010 IEEE International Conference on Robotics and Automation.

[65]  Frank Dellaert,et al.  Square Root SAM , 2005, Robotics: Science and Systems.