Perception Methods for Continuous Humanoid Locomotion Over Uneven Terrain

Mobile robots can safely operate in environments that pose risks to human health, such as disaster zones and planetary exploration. Robots in these environments may encounter uneven terrain like debris fields, staircases, and rock ledges that are impossible to traverse with wheels or tracks but are surmountable by legged humanoid platforms through careful selection of footholds. The DARPA Robotics Challenge demonstrated that today's field robots are capable of uneven terrain traversal but they moved slowly and only for short durations. The stretches of walking are separated by longer stationary periods consumed by LIDAR data acquisition and human operator decision making. With the goal of improving autonomy, speed, and reliability, this thesis research investigates new algorithms for continuous locomotion over uneven terrain through online terrain perception and continuous footstep re-planning. A new algorithm for planar segmentation of terrain features is presented, along with a novel approach that integrates stereo depth fusion for terrain perception and online footstep re-planning using mixed-integer quadratic optimization. The approach is implemented within a novel software framework called Director, and results are validated on hardware using the Atlas humanoid robot with autonomous laboratory experiments and semi-autonomous field experiments at the DARPA Robotics Challenge Finals. Thesis Supervisor: Russ Tedrake Title: Professor of Computer Science

[1]  Kyungdon Joo,et al.  Vision system and depth processing for DRC-HUBO+ , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Bertrand Douillard,et al.  On the segmentation of 3D LIDAR point clouds , 2011, 2011 IEEE International Conference on Robotics and Automation.

[3]  Robin Deits,et al.  Computing Large Convex Regions of Obstacle-Free Space Through Semidefinite Programming , 2014, WAFR.

[4]  Andrew Y. Ng,et al.  A control architecture for quadruped locomotion over rough terrain , 2008, 2008 IEEE International Conference on Robotics and Automation.

[5]  David G. Lowe,et al.  Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration , 2009, VISAPP.

[6]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[7]  Roland Kwitt,et al.  PCL and ParaView — Connecting the dots , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[8]  William E. Lorensen,et al.  The visualization toolkit (2nd ed.): an object-oriented approach to 3D graphics , 1998 .

[9]  Robin Deits,et al.  Continuous humanoid locomotion over uneven terrain using stereo fusion , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[10]  Olivier Stasse,et al.  Real-time replanning using 3D environment for humanoid robot , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[11]  Daniel Maier,et al.  Real-time navigation in 3D environments based on depth camera data , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[12]  Wolfram Burgard,et al.  OctoMap: an efficient probabilistic 3D mapping framework based on octrees , 2013, Autonomous Robots.

[13]  Nancy M. Amato,et al.  Approximate convex decomposition of polygons , 2004, SCG '04.

[14]  Andrew Y. Ng,et al.  Stereo vision and terrain modeling for quadruped robots , 2009, 2009 IEEE International Conference on Robotics and Automation.

[15]  Olivier Stasse,et al.  Real-time footstep planning for humanoid robots among 3D obstacles using a hybrid bounding box , 2012, 2012 IEEE International Conference on Robotics and Automation.

[16]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[17]  Kazuhito Yokoi,et al.  Biped walking pattern generation by using preview control of zero-moment point , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[18]  Vibhav Vineet,et al.  Conditional Random Fields as Recurrent Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[19]  Seth J. Teller,et al.  Drift-free humanoid state estimation fusing kinematic, inertial and LIDAR sensing , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[20]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[21]  Olivier Stasse,et al.  Fast Humanoid Robot Collision-Free Footstep Planning Using Swept Volume Approximations , 2012, IEEE Transactions on Robotics.

[22]  Satoshi Kagami,et al.  Planning and Control of a Humanoid Robot for Navigation on Uneven Multi-scale Terrain , 2010, ISER.

[23]  Olivier Stasse,et al.  Integrating Walking and Vision to Increase Humanoid Autonomy , 2008, Int. J. Humanoid Robotics.

[24]  Michael Beetz,et al.  Leaving Flatland: Efficient real‐time three‐dimensional perception and motion planning , 2009, J. Field Robotics.

[25]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[26]  Takeo Kanade,et al.  Vision-guided humanoid footstep planning for dynamic environments , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[27]  Olivier Stasse,et al.  Weakly collision-free paths for continuous humanoid footstep planning , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[28]  Timothy Bretl,et al.  Motion Planning for a Six-Legged Lunar Robot , 2006, WAFR.

[29]  John J. Leonard,et al.  Kintinuous: Spatially Extended KinectFusion , 2012, AAAI 2012.

[30]  Pieter Abbeel,et al.  BigBIRD: A large-scale 3D database of object instances , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[31]  Christoph Stiller,et al.  Segmentation of 3D lidar data in non-flat urban environments using a local convexity criterion , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[32]  John J. Leonard,et al.  Real-time large-scale dense RGB-D SLAM with volumetric fusion , 2014, Int. J. Robotics Res..

[33]  Maren Bennewitz,et al.  Autonomous climbing of spiral staircases with humanoids , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[34]  Robin Deits,et al.  Footstep planning on uneven terrain with mixed-integer convex optimization , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[35]  Masahiro Fujita,et al.  A modular architecture for humanoid robot navigation , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[36]  Michael Beetz,et al.  Real-time perception-guided motion planning for a personal robot , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[37]  Robin Deits.  convex segmentation and mixed-integer footstep planning for a walking robot , 2014 .

[38]  David G. Kirkpatrick,et al.  On the shape of a set of points in the plane , 1983, IEEE Trans. Inf. Theory.

[39]  Ian R. Manchester,et al.  Bounding on rough terrain with the LittleDog robot , 2011, Int. J. Robotics Res..

[40]  Olivier Stasse,et al.  A biped walking pattern generator based on “half-steps” for dimensionality reduction , 2011, 2011 IEEE International Conference on Robotics and Automation.

[41]  J. How,et al.  Mixed-integer programming for control , 2005, Proceedings of the 2005, American Control Conference, 2005..

[42]  Satoshi Kagami,et al.  Autonomous navigation of a humanoid robot over unknown rough terrain using a laser range sensor , 2012, Int. J. Robotics Res..

[43]  Alexandre Boulch,et al.  Fast and Robust Normal Estimation for Point Clouds with Sharp Features , 2012, Comput. Graph. Forum.

[44]  Masahiro Fujita,et al.  3D Perception and Environment Map Generation for Humanoid Robot Navigation , 2008, Int. J. Robotics Res..

[45]  Johannes Garimort,et al.  Humanoid navigation with dynamic footstep plans , 2011, 2011 IEEE International Conference on Robotics and Automation.

[46]  Nicholas Roy,et al.  Reliable Dynamic Motions for a Stiff Quadruped , 2009, ISER.

[47]  Olivier Stasse,et al.  Toward Reactive Vision-Guided Walking on Rough Terrain: An Inverse-Dynamics Based Approach , 2014, Int. J. Humanoid Robotics.

[48]  Masayuki Inaba,et al.  Footstep planning among obstacles for biped robots , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).