Coupled recursive estimation for online interactive perception of articulated objects

We present online multi-modal perception systems for extracting kinematic and dynamic models of articulated objects from physical interactions with the environment. The systems rely on a RGB-D stre...

[1]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[2]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[3]  Yaakov Bar-Shalom,et al.  Multi-target tracking using joint probabilistic data association , 1980, 1980 19th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[4]  Raj Bhatnagar,et al.  Discovery of Temporal Dependencies between Frequent Patterns in Multivariate Time Series , 2007, 2007 IEEE Symposium on Computational Intelligence and Data Mining.

[5]  Dominiek Reynaerts,et al.  Constraint-Based Interaction Control of Robots Featuring Large Compliance and Deformation , 2015, IEEE Transactions on Robotics.

[6]  D. Kersten,et al.  Opposite Modulation of High- and Low-Level Visual Aftereffects by Perceptual Grouping , 2012, Current Biology.

[7]  Ian D. Reid,et al.  Articulated structure from motion by factorization , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  Ian D. Walker,et al.  Occlusion-aware reconstruction and manipulation of 3D articulated objects , 2012, 2012 IEEE International Conference on Robotics and Automation.

[9]  W. A. Phillips,et al.  The function of dynamic grouping in vision , 2000, Trends in Cognitive Sciences.

[10]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[11]  Richard S. Zemel,et al.  Unsupervised Learning of Skeletons from Motion , 2008, ECCV.

[12]  J. Andrew Bagnell,et al.  Interactive segmentation, tracking, and kinematic modeling of unknown 3D articulated objects , 2013, 2013 IEEE International Conference on Robotics and Automation.

[13]  Oliver Brock,et al.  Differentiable Particle Filters: End-to-End Learning with Algorithmic Priors , 2018, Robotics: Science and Systems.

[14]  Marc Pollefeys,et al.  Automatic Kinematic Chain Building from Feature Trajectories of Articulated Objects , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[15]  Lea Fleischer,et al.  The Senses Considered As Perceptual Systems , 2016 .

[16]  Oliver Brock,et al.  Interactive Perception: Leveraging Action in Perception and Perception in Action , 2016, IEEE Transactions on Robotics.

[17]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Joris De Schutter,et al.  Specification of force-controlled actions in the "task frame formalism"-a synthesis , 1996, IEEE Trans. Robotics Autom..

[19]  J.M. Hollerbach,et al.  Identifying mass parameters for gravity compensation and automatic torque sensor calibration , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[20]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[21]  Davide Scaramuzza,et al.  SVO: Fast semi-direct monocular visual odometry , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[22]  V. Maljkovic,et al.  Implicit short-term memory and event frequency effects in visual search , 2005, Vision Research.

[23]  J. Gibson.  The Ecological Approach to Visual Perception , 1979 .

[24]  Bodo Rosenhahn,et al.  3D Object Recognition and Pose Estimation for Multiple Objects Using Multi-Prioritized RANSAC and Model Updating , 2012, DAGM/OAGM Symposium.

[25]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[26]  E. Spelke,et al.  Object perception, object-directed action, and physical knowledge in infancy , 1995 .

[27]  Oliver Brock,et al.  Physics-Based Selection of Informative Actions for Interactive Perception , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[28]  Edwin Olson,et al.  AprilTag 2: Efficient and robust fiducial detection , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[29]  Fang Fang,et al.  Perceptual grouping and inverse fMRI activity patterns in human visual cortex. , 2008, Journal of vision.

[30]  Pierre E. Dupont,et al.  Friction modeling in dynamic robot simulation , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[31]  Wolfram Burgard,et al.  Operating articulated objects based on experience , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Wolfram Burgard,et al.  Vision-based detection for learning articulation models of cabinet doors and drawers in household environments , 2010, 2010 IEEE International Conference on Robotics and Automation.

[33]  D. Hubel,et al.  Segregation of form, color, movement, and depth: anatomy, physiology, and perception. , 1988, Science.

[34]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[35]  Cheng Qiu,et al.  Responses in early visual areas to contour integration are context dependent , 2016, Journal of vision.

[36]  Paul Timothy Furgale,et al.  Associating Uncertainty With Three-Dimensional Poses for Use in Estimation Problems , 2014, IEEE Transactions on Robotics.

[37]  P. J. Foley.  The foreperiod and simple reaction time. , 1959, Canadian journal of psychology.

[38]  Jeannette Bohg,et al.  Three-dimensional object reconstruction of symmetric objects by fusing visual and tactile sensing , 2014, Int. J. Robotics Res..

[39]  Stefan Schaal,et al.  Probabilistic Articulated Real-Time Tracking for Robot Manipulation , 2016, IEEE Robotics and Automation Letters.

[40]  A. Borst.  Seeing smells: imaging olfactory learning in bees , 1999, Nature Neuroscience.

[41]  Wolfram Burgard,et al.  Learning Kinematic Models for Articulated Objects , 2009, IJCAI.

[42]  James R. Bergen,et al.  Visual odometry , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[43]  Eric Horvitz,et al.  Dynamic Network Models for Forecasting , 1992, UAI.

[44]  Wolfram Burgard,et al.  Learning the dynamics of doors for robotic manipulation , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[45]  Wolfram Burgard,et al.  A Probabilistic Framework for Learning Kinematic Models of Articulated Objects , 2011, J. Artif. Intell. Res..

[46]  S. Hochstein,et al.  The reverse hierarchy theory of visual perceptual learning , 2004, Trends in Cognitive Sciences.

[47]  Oliver Brock,et al.  Cross-modal interpretation of multi-modal sensor streams in interactive perception based on coupled recursion , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[48]  Wei Sun,et al.  Autoscanning for coupled scene reconstruction and proactive object analysis , 2015, ACM Trans. Graph..

[49]  A. Yuille,et al.  Inferential Models of the Visual Cortical Hierarchy* , 2013 .

[50]  Connor Schenck,et al.  Interactive object recognition using proprioceptive and auditory feedback , 2011, Int. J. Robotics Res..

[51]  Rajesh P. N. Rao,et al.  Dynamic Model of Visual Recognition Predicts Neural Response Properties in the Visual Cortex , 1997, Neural Computation.

[52]  Petter Ögren,et al.  An Adaptive Control Approach for Opening Doors and Drawers Under Uncertainties , 2016, IEEE Transactions on Robotics.

[53]  Thiagalingam Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation , 2001 .

[54]  Gaurav S. Sukhatme,et al.  Active articulation model estimation through interactive perception , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[55]  Gaurav S. Sukhatme,et al.  Multi-step planning for robotic manipulation , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[56]  J. Jonas,et al.  Human optic nerve fiber count and optic disc size. , 1992, Investigative ophthalmology & visual science.

[57]  Dieter Fox,et al.  Map-Based Multiple Model Tracking of a Moving Object , 2004, RoboCup.

[58]  S. Hochstein,et al.  View from the Top Hierarchies and Reverse Hierarchies in the Visual System , 2002, Neuron.

[59]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[61]  K. Nakayama,et al.  Priming of pop-out: I. Role of features , 1994, Memory & cognition.

[62]  Christopher G. Atkeson,et al.  Estimation of Inertial Parameters of Manipulator Loads and Links , 1986 .

[63]  David Mumford,et al.  On the computational architecture of the neocortex , 2004, Biological Cybernetics.

[64]  Oliver Brock,et al.  A novel type of compliant and underactuated robotic hand for dexterous grasping , 2016, Int. J. Robotics Res..

[65]  R. Held,et al.  MOVEMENT-PRODUCED STIMULATION IN THE DEVELOPMENT OF VISUALLY GUIDED BEHAVIOR. , 1963, Journal of comparative and physiological psychology.

[66]  Ales Ude,et al.  Physical interaction for segmentation of unknown textured and non-textured rigid objects , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[67]  Mike Stilman,et al.  Task constrained motion planning in robot joint space , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[68]  James J. Clark,et al.  Data Fusion for Sensory Information Processing Systems , 1990 .

[69]  Susan J. Hespos,et al.  Physics for infants: characterizing the origins of knowledge about objects, substances, and number. , 2012, Wiley interdisciplinary reviews. Cognitive science.

[70]  Sergey Levine,et al.  Backprop KF: Learning Discriminative Deterministic State Estimators , 2016, NIPS.

[71]  Michael H Herzog.  Perceptual grouping , 2018, Current Biology.

[72]  Weiwei Huang,et al.  Decoupled state estimation for humanoids using full-body dynamics , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[73]  Oliver Kroemer,et al.  Maximally informative interaction learning for scene exploration , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[74]  Oliver Brock,et al.  Interactive Perception of Articulated Objects , 2010, ISER.

[75]  Joel W. Burdick,et al.  Combined shape, appearance and silhouette for simultaneous manipulator and object tracking , 2012, 2012 IEEE International Conference on Robotics and Automation.

[76]  Henry A. Kautz,et al.  Learning and inferring transportation routines , 2004, Artif. Intell..

[77]  Takeo Kanade,et al.  A Multibody Factorization Method for Independently Moving Objects , 1998, International Journal of Computer Vision.

[78]  Tim Gollisch,et al.  Eye Smarter than Scientists Believed: Neural Computations in Circuits of the Retina , 2010, Neuron.

[79]  Árni Kristjánsson,et al.  Temporal Consistency Is Currency in Shifts of Transient Visual Attention , 2010, PloS one.

[80]  Advait Jain,et al.  The complex structure of simple devices: A survey of trajectories and forces that open doors and drawers , 2010, 2010 3rd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics.

[81]  Oliver Brock,et al.  Online interactive perception of articulated objects with multi-level recursive estimation based on task-specific priors , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[82]  Raúl Rojas,et al.  Neural Networks - A Systematic Introduction , 1996 .