Distributional Depth-Based Estimation of Object Articulation Models

We propose a method that efficiently learns distributions over articulation model parameters directly from depth images without the need to know articulation model categories a priori. By contrast, existing methods that learn articulation models from raw observations typically only predict point estimates of the model parameters, which are insufficient to guarantee the safe manipulation of articulated objects. Our core contributions include a novel representation for distributions over rigid body transformations and articulation model parameters based on screw theory, von Mises-Fisher distributions, and Stiefel manifolds. Combining these concepts allows for an efficient, mathematically sound representation that implicitly satisfies the constraints that rigid body transformations and articulations must adhere to. Leveraging this representation, we introduce a novel deep learning based approach, DUST-net, that performs category-independent articulation model estimation while also providing model uncertainties. We evaluate our approach on several benchmarking datasets and real-world objects and compare its performance with two current state-of-the-art methods. Our results demonstrate that DUST-net can successfully learn distributions over articulation models for novel objects across articulation model categories, which generate point estimates with better accuracy than state-of-the-art methods and effectively capture the uncertainty over predicted model parameters due to noisy inputs.

[1]  Scott Niekum,et al.  Efficient Hierarchical Robot Motion Planning Under Uncertainty and Hybrid Dynamics , 2018, CoRL.

[2]  Oliver Brock,et al.  Interactive Perception: Leveraging Action in Perception and Perception in Action , 2016, IEEE Transactions on Robotics.

[3]  Matthew R. Walter,et al.  A Multiview Approach to Learning Articulated Motion Models , 2017, ISRR.

[4]  Scott Niekum,et al.  Learning Hybrid Object Kinematics for Efficient Hierarchical Planning Under Uncertainty , 2019, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Thomas Brox,et al.  Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Prediction , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Michael Gleicher,et al.  Inferring geometric constraints in human demonstrations , 2018, CoRL.

[7]  Howie Choset,et al.  Probabilistic pose estimation using a Bingham distribution-based linear filter , 2018, Int. J. Robotics Res..

[8]  Yuval Tassa,et al.  MuJoCo: A physics engine for model-based control , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Gerhard Kurz,et al.  Unscented Orientation Estimation Based on the Bingham Distribution , 2013, IEEE Transactions on Automatic Control.

[10]  Yan-Bin Jia Plücker Coordinates for Lines in the Space , 2020 .

[11]  Y. Chikuse Statistics on special manifolds , 2003 .

[12]  Scott Niekum,et al.  ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theory , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[13]  K. Mardia,et al.  The von Mises–Fisher Matrix Distribution in Orientation Statistics , 1977 .

[14]  Gregory D. Hager,et al.  Nothing But Geometric Constraints: A Model-Free Method for Articulated Object Pose Estimation , 2020, ArXiv.

[15]  Howie Choset,et al.  Estimating SE(3) elements using a dual quaternion based linear Kalman filter , 2016, Robotics: Science and Systems.

[16]  John J. Leonard,et al.  SE-Sync: A certifiably correct algorithm for synchronization over the special Euclidean group , 2016, Int. J. Robotics Res..

[17]  A. James Distributions of Matrix Variates and Latent Roots Derived from Normal Samples , 1964 .

[18]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Leonidas J. Guibas,et al.  SAPIEN: A SimulAted Part-Based Interactive ENvironment , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Matthew R. Walter,et al.  Learning Articulated Motions From Visual Demonstration , 2014, Robotics: Science and Systems.

[21]  Oliver Brock,et al.  An integrated approach to visual perception of articulated objects , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Christopher G. Atkeson,et al.  Online Bayesian changepoint detection for articulated motion models , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Oliver Brock,et al.  Manipulating articulated objects with interactive perception , 2008, 2008 IEEE International Conference on Robotics and Automation.

[24]  A. Lynn Abbott,et al.  Category-Level Articulated Object Pose Estimation , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[26]  Julie A. Shah,et al.  C-LEARN: Learning geometric constraints from demonstrations for multi-step manipulation in shared autonomy , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[27]  J. Andrew Bagnell,et al.  Interactive segmentation, tracking, and kinematic modeling of unknown 3D articulated objects , 2013, 2013 IEEE International Conference on Robotics and Automation.

[28]  Yisong Yue,et al.  Learning for Safety-Critical Control with Control Barrier Functions , 2019, L4DC.

[29]  Oussama Khatib,et al.  Springer Handbook of Robotics , 2007, Springer Handbooks.

[30]  Stefanie Tellex,et al.  Learning to Generalize Kinematic Models to Novel Objects , 2019, CoRL.

[31]  Adrien Bartoli,et al.  The 3D Line Motion Matrix and Alignment of Line Reconstructions , 2004, International Journal of Computer Vision.

[32]  Leonidas J. Guibas,et al.  ShapeNet: An Information-Rich 3D Model Repository , 2015, ArXiv.

[33]  Calculation and Properties of Zonal Polynomials , 2020, Math. Comput. Sci..

[34]  Hui Huang,et al.  RPM-Net , 2019, ACM Trans. Graph..

[35]  Xiaogang Wang,et al.  Shape2Motion: Joint Analysis of Motion Parts and Attributes From 3D Shapes , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Oliver Brock,et al.  Coupled recursive estimation for online interactive perception of articulated objects , 2019, Int. J. Robotics Res..

[37]  Yuchen Cui,et al.  Active Reward Learning from Critiques , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[38]  Jingxuan Li,et al.  Learning Articulated Constraints From a One-Shot Demonstration for Robot Manipulation Planning , 2019, IEEE Access.

[39]  Oliver Brock,et al.  Online interactive perception of articulated objects with multi-level recursive estimation based on task-specific priors , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[40]  Leonidas J. Guibas,et al.  PartNet: A Large-Scale Benchmark for Fine-Grained and Hierarchical Part-Level 3D Object Understanding , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Wolfram Burgard,et al.  A Probabilistic Framework for Learning Kinematic Models of Articulated Objects , 2011, J. Artif. Intell. Res..