Episodic Learning for Safe Bipedal Locomotion with Control Barrier Functions and Projection-to-State Safety

This paper combines episodic learning and control barrier functions in the setting of bipedal locomotion. The safety guarantees that control barrier functions provide are only valid with perfect model knowledge; however, this assumption cannot be met on hardware platforms. To address this, we utilize the notion of projection-to-state safety paired with a machine learning framework in an attempt to learn the model uncertainty as it affects the barrier functions. The proposed approach is demonstrated both in simulation and on hardware for the AMBER-3M bipedal robot in the context of the stepping-stone problem, which requires precise foot placement while walking dynamically.

[1]  Lorenz Wellhausen,et al.  Learning quadrupedal locomotion over challenging terrain , 2020, Science Robotics.

[2]  Yisong Yue,et al.  Learning for Safety-Critical Control with Control Barrier Functions , 2019, L4DC.

[3]  Aaron D. Ames,et al.  Toward benchmarking locomotion economy across design configurations on the modular robot: AMBER-3M , 2017, 2017 IEEE Conference on Control Technology and Applications (CCTA).

[4]  Paulo Tabuada,et al.  Control barrier function based quadratic programs with application to adaptive cruise control , 2014, 53rd IEEE Conference on Decision and Control.

[5]  Koushil Sreenath,et al.  Rapidly Exponentially Stabilizing Control Lyapunov Functions and Hybrid Zero Dynamics , 2014, IEEE Transactions on Automatic Control.

[6]  Aaron D. Ames,et al.  Adaptive Safety with Control Barrier Functions , 2019, 2020 American Control Conference (ACC).

[7]  Jaime F. Fisac,et al.  A General Safety Framework for Learning-Based Control in Uncertain Robotic Systems , 2017, IEEE Transactions on Automatic Control.

[8]  Daniel E. Koditschek,et al.  Hybrid zero dynamics of planar biped walkers , 2003, IEEE Trans. Autom. Control..

[9]  Koushil Sreenath,et al.  3D dynamic walking on stepping stones with control barrier functions , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[10]  Aaron D. Ames,et al.  Multi-Layered Safety for Legged Robots via Control Barrier Functions and Model Predictive Control , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Franco Blanchini,et al.  Set invariance in control , 1999, Autom..

[12]  Koushil Sreenath,et al.  Dynamic Walking on Stepping Stones with Gait Library and Control Barrier Functions , 2016, WAFR.

[13]  Aaron D. Ames,et al.  Guaranteeing Safety of Learned Perception Modules via Measurement-Robust Control Barrier Functions , 2020, CoRL.

[14]  Koushil Sreenath,et al.  Exponential Control Barrier Functions for enforcing high relative-degree safety-critical constraints , 2016, 2016 American Control Conference (ACC).

[15]  Marco Hutter,et al.  Per-Contact Iteration Method for Solving Contact Dynamics , 2018, IEEE Robotics and Automation Letters.

[16]  Paulo Tabuada,et al.  Control Barrier Function Based Quadratic Programs for Safety Critical Systems , 2016, IEEE Transactions on Automatic Control.

[17]  Aaron D. Ames,et al.  Dynamic Humanoid Locomotion: A Scalable Formulation for HZD Gait Optimization , 2018, IEEE Transactions on Robotics.

[18]  L. Perko.  Differential Equations and Dynamical Systems , 1991 .

[19]  David D. Fan,et al.  Bayesian Learning-Based Adaptive Control for Safety Critical Systems , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Aaron D. Ames,et al.  Towards a Framework for Realizable Safety Critical Control through Active Set Invariance , 2018, 2018 ACM/IEEE 9th International Conference on Cyber-Physical Systems (ICCPS).

[21]  S. Sastry.  Nonlinear Systems: Analysis, Stability, and Control , 1999 .

[22]  Aaron D. Ames,et al.  Input-to-State Safety With Control Barrier Functions , 2018, IEEE Control Systems Letters.

[23]  Claire J. Tomlin,et al.  Gaussian Process-based Min-norm Stabilizing Controller for Control-Affine Systems with Uncertain Input Effects and Dynamics , 2020, 2021 American Control Conference (ACC).

[24]  Robert J. Griffin,et al.  Footstep Planning for Autonomous Walking Over Rough Terrain , 2019, 2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids).

[25]  Andreas Krause,et al.  Safe learning of regions of attraction for uncertain, nonlinear systems with Gaussian processes , 2016, 2016 IEEE 55th Conference on Decision and Control (CDC).

[26]  Koushil Sreenath,et al.  Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functions , 2020, Robotics: Science and Systems.

[27]  Koushil Sreenath,et al.  Dynamic bipedal locomotion over stochastic discrete terrain , 2018, Int. J. Robotics Res..

[28]  Byron Boots,et al.  Accelerating Imitation Learning with Predictive Models , 2018, AISTATS.

[29]  Aaron D. Ames,et al.  Episodic Learning with Control Lyapunov Functions for Uncertain Robotic Systems* , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[30]  Peter Fankhauser,et al.  Robust Rough-Terrain Locomotion with a Quadrupedal Robot , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[31]  E. Westervelt,et al.  Feedback Control of Dynamic Bipedal Robot Locomotion , 2007 .

[32]  Franck Plestan,et al.  Asymptotically stable walking for biped robots: analysis via systems with impulse effects , 2001, IEEE Trans. Autom. Control..

[33]  Paulo Tabuada,et al.  Robustness of Control Barrier Functions for Safety Critical Control , 2016, ADHS.

[34]  Li Wang,et al.  Safe Learning of Quadrotor Dynamics Using Barrier Certificates , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[35]  Aaron D. Ames,et al.  Human-inspired walking via unified PD and impedance control , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[36]  Aaron D. Ames,et al.  A Control Barrier Perspective on Episodic Learning via Projection-to-State Safety , 2020, IEEE Control Systems Letters.