Planning and Control in Face of Uncertainty with Applications to Legged Robots

The present work aims at motion control and planning for autonomous robots, specifically legged robots. Motion control and planning have always been an important component for creating autonomous robots. There has been significant progress in this field for UAVs and wheeled robots since the early ages of robotics. However, motion control and planning for legged robots is still a challenging problem and some of the recent, notable robotic challenges have demonstrated the necessity of more robust and online approaches. To this end, our work in this dissertation revolves around three main challenges of motion control and planning in legged robots namely real-time planning, robustness of the control structure and adaptation to unknown or neglected dynamics. One of the essential requirements for robust planning in real world applications is the capability of finding solutions in a real-time fashion to adjust the plan with the current state measurements. Many of today’s online approaches have achieved this efficacy through task decomposition and model reduction approaches, since whole-body approaches are often significantly slower to be run online. To address this problem, we have proposed an efficient algorithm for whole-body planning of floating based robots with switching dynamics. Our proposed method is based on an optimal control approach which synthesizes an optimal feedback control policy for continuous inputs and optimizes the switching times in between two consecutive stance modes. Through this optimization approach, we have demonstrated that a wide spectrum of motions can be generated for a quadrupedal robot with predetermined gait sequences. Furthermore, we have employed this algorithm for planning highly dynamic gaits (e.g. trotting) in a nonlinear Model Predictive Control (MPC) fashion on hardware. This is one of the earliest works on whole-body MPC for periodic gait generation of legged systems.

[1]  Stefan Schaal,et al.  Learning force control policies for compliant manipulation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Cedric de Crousaz,et al.  Aggressive Optimal Control for Agile Flight with a Slung Load , 2014 .

[3]  Stefan Schaal,et al.  Optimal distribution of contact forces with inverse-dynamics control , 2013, Int. J. Robotics Res..

[4]  N. Roy,et al.  On Stochastic Optimal Control and Reinforcement Learning by Approximate Inference , 2013 .

[5]  Vijay Kumar,et al.  Trajectory generation and control of a quadrotor with a cable-suspended load - A differentially-flat hybrid system , 2013, 2013 IEEE International Conference on Robotics and Automation.

[6]  Rafael Fierro,et al.  Trajectory generation for swing-free maneuvers of a quadrotor with suspended payload: A dynamic programming approach , 2012, 2012 IEEE International Conference on Robotics and Automation.

[7]  C. C. Cheah,et al.  Stability of task-space feedback control for robots with uncertain actuator model: theory and experiments , 2003, IECON'03. 29th Annual Conference of the IEEE Industrial Electronics Society (IEEE Cat. No.03CH37468).

[8]  MORITZ DIEHL,et al.  A Real-Time Iteration Scheme for Nonlinear Optimization in Optimal Feedback Control , 2005, SIAM J. Control. Optim..

[9]  Daniel Mellinger,et al.  Control of Quadrotors for Robust Perching and Landing , 2010 .

[10]  S. Yakowitz The stagewise Kuhn-Tucker condition and differential dynamic programming , 1986 .

[11]  Ralph L. Hollis,et al.  Differentially flat trajectory generation for a dynamically stable mobile robot , 2013, 2013 IEEE International Conference on Robotics and Automation.

[12]  Martin A. Riedmiller,et al.  Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Images , 2015, NIPS.

[13]  Umashankar Nagarajan,et al.  Shape space planner for shape-accelerated balancing mobile robots , 2013, Int. J. Robotics Res..

[14]  Aaron Hertzmann,et al.  Robust physics-based locomotion using low-dimensional planning , 2010, SIGGRAPH 2010.

[15]  Masayuki Inaba,et al.  Dynamically-Stable Motion Planning for Humanoid Robots , 2002, Auton. Robots.

[16]  Todd D. Murphey,et al.  Trajectory generation for underactuated control of a suspended mass , 2012, 2012 IEEE International Conference on Robotics and Automation.

[17]  Thomas B. Schön,et al.  From Pixels to Torques: Policy Learning with Deep Dynamical Models , 2015, ICML 2015.

[18]  C. Iung,et al.  Linear quadratic optimization for hybrid systems , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).

[19]  Zoran Popovic,et al.  Discovery of complex behaviors through contact-invariant optimization , 2012, ACM Trans. Graph..

[20]  Yunpeng Pan,et al.  Probabilistic Differential Dynamic Programming , 2014, NIPS.

[21]  Jonas Buchli,et al.  An efficient optimal planning and control framework for quadrupedal locomotion , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Masayoshi Tomizuka,et al.  Robust impedance control with applications to a series-elastic actuated system , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  Olivier Stasse,et al.  Fast Humanoid Robot Collision-Free Footstep Planning Using Swept Volume Approximations , 2012, IEEE Transactions on Robotics.

[24]  Olivier Stasse,et al.  Optimal control for whole-body motion generation using center-of-mass dynamics for predefined multi-contact configurations , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[25]  Vijay Kumar,et al.  Mixed Integer Quadratic Program trajectory generation for a quadrotor with a cable-suspended payload , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[26]  E. Todorov,et al.  A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems , 2005, Proceedings of the 2005, American Control Conference, 2005..

[27]  Stefan Schaal,et al.  Inverse dynamics control of floating base systems using orthogonal decomposition , 2010, 2010 IEEE International Conference on Robotics and Automation.

[28]  Jonas Buchli,et al.  Sequential Linear Quadratic Optimal Control for Nonlinear Switched Systems , 2016, ArXiv.

[29]  Athanasios Sideris,et al.  A Riccati approach to equality constrained Linear Quadratic Optimal control , 2010, Proceedings of the 2010 American Control Conference.

[30]  Yuval Tassa,et al.  An integrated system for real-time model predictive control of humanoid robots , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[31]  Nicolas Mansard,et al.  Robustness to Joint-Torque-Tracking Errors in Task-Space Inverse Dynamics , 2016, IEEE Transactions on Robotics.

[32]  Jürgen Hesselbach,et al.  Robust task-space control of hydraulic robots , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[33]  Carl E. Rasmussen,et al.  PILCO: A Model-Based and Data-Efficient Approach to Policy Search , 2011, ICML.

[34]  Roland Siegwart,et al.  Hybrid Operational Space Control for Compliant Legged Systems , 2012, RSS 2012.

[35]  Athanasios Sideris,et al.  An active set method for constrained linear quadratic optimal control , 2010, Proceedings of the 2010 American Control Conference.

[36]  K. Mombaur,et al.  Modeling and Optimal Control of Human-Like Running , 2010, IEEE/ASME Transactions on Mechatronics.

[37]  Siddhartha S. Srinivasa,et al.  CHOMP: Covariant Hamiltonian optimization for motion planning , 2013, Int. J. Robotics Res..

[38]  Koushil Sreenath,et al.  Optimal Robust Control for Bipedal Robots through Control Lyapunov Function based Quadratic Programs , 2015, Robotics: Science and Systems.

[39]  Peter E. Caines,et al.  On the Hybrid Optimal Control Problem: Theory and Algorithms , 2007, IEEE Transactions on Automatic Control.

[40]  Yuval Tassa,et al.  Synthesis and stabilization of complex behaviors through online trajectory optimization , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[41]  H. Kappen Linear theory for control of nonlinear stochastic systems. , 2004, Physical review letters.

[42]  Alberto Bemporad,et al.  Dynamic programming for constrained optimal control of discrete-time linear hybrid systems , 2005, Autom..

[43]  Maryam Kamgarpour,et al.  Multiphase mixed-integer optimal control framework for aircraft conflict avoidance , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[44]  Wittmann Robert,et al.  Model-based predictive bipedal walking stabilization , 2016 .

[45]  Nicolas Mansard,et al.  Prioritized optimal control: A hierarchical differential dynamic programming approach , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[46]  L. Armijo Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[47]  Sangbae Kim,et al.  Online Planning for Autonomous Running Jumps Over Obstacles in High-Speed Quadrupeds , 2015, Robotics: Science and Systems.

[48]  Lydia Tapia,et al.  A reinforcement learning approach towards autonomous suspended load manipulation using aerial robots , 2013, 2013 IEEE International Conference on Robotics and Automation.

[49]  Carl E. Rasmussen,et al.  Gaussian process dynamic programming , 2009, Neurocomputing.

[50]  J. Chestnutt,et al.  Planning Biped Navigation Strategies in Complex Environments , 2003 .

[51]  Nicholas Roy,et al.  State Estimation for Legged Robots: Consistent Fusion of Leg Kinematics and IMU , 2013 .

[52]  Oskar von Stryk,et al.  Direct and indirect methods for trajectory optimization , 1992, Ann. Oper. Res..

[53]  Bruce A. Conway,et al.  Discrete approximations to optimal trajectories using direct transcription and nonlinear programming , 1992 .

[54]  J. Buchli,et al.  Path Integral Stochastic Optimal Control for Reinforcement Learning , 2014 .

[55]  Jan Peters,et al.  Policy Search for Motor Primitives in Robotics , 2008, NIPS 2008.

[56]  Jay H. Lee,et al.  Model predictive control: past, present and future , 1999 .

[57]  Eiichi Yoshida,et al.  Generation of whole-body optimal dynamic multi-contact motions , 2013, Int. J. Robotics Res..

[58]  Gerhard Neumann,et al.  Variational Inference for Policy Search in changing situations , 2011, ICML.

[59]  Yuval Tassa,et al.  Control-limited differential dynamic programming , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[60]  Jonas Buchli,et al.  Projection based whole body motion planning for legged robots , 2015, ArXiv.

[61]  A. Giua,et al.  Optimal control of switched autonomous linear systems , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[62]  Alberto Bemporad,et al.  Control of systems integrating logic, dynamics, and constraints , 1999, Autom..

[63]  Darwin G. Caldwell,et al.  A reactive controller framework for quadrupedal locomotion on challenging terrain , 2013, 2013 IEEE International Conference on Robotics and Automation.

[64]  J. Pantoja,et al.  Differential dynamic programming and Newton's method , 1988 .

[65]  D. Bertsekas,et al.  Efficient dynamic programming implementations of Newton's method for unconstrained optimal control problems , 1989 .

[66]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[67]  Jun Nakanishi,et al.  Inverse Dynamics Control with Floating Base and Constraints , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[68]  Vicenç Gómez,et al.  Optimal control as a graphical model inference problem , 2009, Machine Learning.

[69]  Y. Wardi,et al.  Optimal control of switching times in switched dynamical systems , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[70]  Stefan Schaal,et al.  Reinforcement learning of full-body humanoid motor skills , 2010, 2010 10th IEEE-RAS International Conference on Humanoid Robots.

[71]  Eiichi Yoshida,et al.  Vertical ladder climbing by the HRP-2 humanoid robot , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[72]  Olivier Stasse,et al.  A Reactive Walking Pattern Generator Based on Nonlinear Model Predictive Control , 2017, IEEE Robotics and Automation Letters.

[73]  Romeo Ortega,et al.  On adaptive impedance control of robot manipulators , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[74]  Aaron D. Ames,et al.  Towards the Unification of Locomotion and Manipulation through Control Lyapunov Functions and Quadratic Programs , 2013, CPSW@CISS.

[75]  Atsuo Takanishi,et al.  Realization of dynamic biped walking stabilized by trunk motion on a sagittally uneven surface , 1990, EEE International Workshop on Intelligent Robots and Systems, Towards a New Frontier of Applications.

[76]  Jun Morimoto,et al.  Learning from demonstration and adaptation of biped locomotion , 2004, Robotics Auton. Syst..

[77]  Peter E. Caines,et al.  On the relation between the Minimum Principle and Dynamic Programming for Hybrid systems , 2014, 53rd IEEE Conference on Decision and Control.

[78]  Frédéric Kratz,et al.  An Optimal Control Approach for Hybrid Systems , 2003, Eur. J. Control.

[79]  Stefan Schaal,et al.  Risk sensitive nonlinear optimal control with measurement uncertainty , 2016, ArXiv.

[80]  Sandra Hirche,et al.  Uncertainty-dependent optimal control for robot control considering high-order cost statistics , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[81]  Jörn Malzahn,et al.  Comparison of open-loop and closed-loop disturbance observers for series elastic actuators , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[82]  Sarah Tang Aggressive Maneuvering of a Quadrotor with a Cable-Suspended Payload , 2017 .

[83]  Jonas Buchli,et al.  Efficient kinematic planning for mobile manipulators with non-holonomic constraints using optimal control , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[84]  Evanghelos Zafiriou,et al.  Robust process control , 1987 .

[85]  Rafael Fierro,et al.  Agile Load Transportation : Safe and Efficient Load Manipulation with Aerial Robots , 2012, IEEE Robotics & Automation Magazine.

[86]  Oussama Khatib,et al.  A unified approach for motion and force control of robot manipulators: The operational space formulation , 1987, IEEE J. Robotics Autom..

[87]  J. Betts Survey of Numerical Methods for Trajectory Optimization , 1998 .

[88]  Stefan Schaal,et al.  Learning variable impedance control , 2011, Int. J. Robotics Res..

[89]  Jaakko Lehtinen,et al.  Online motion synthesis using sequential Monte Carlo , 2014, ACM Trans. Graph..

[90]  Robert F. Stengel,et al.  Optimal Control and Estimation , 1994 .

[91]  Ronald Lumia,et al.  Rapid Transport of Suspended Payloads , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[92]  Ralph L. Hollis,et al.  A dynamically stable single-wheeled mobile robot with inverse mouse-ball drive , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[93]  Jun Nakanishi,et al.  Task space control with prioritization for balance and locomotion , 2007 .

[94]  J. Dunn A projected Newton method for minimization problems with nonlinear inequality constraints , 1988 .

[95]  Magnus Egerstedt,et al.  A controlled-precision algorithm for mode-switching optimization , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[96]  Christopher G. Atkeson,et al.  Dynamic Balance Force Control for compliant humanoid robots , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[97]  Russ Tedrake,et al.  Whole-body motion planning with centroidal dynamics and full kinematics , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[98]  Russ Tedrake,et al.  A direct method for trajectory optimization of rigid bodies through contact , 2014, Int. J. Robotics Res..

[99]  Stefan Schaal,et al.  Inverse dynamics control of floating-base robots with external constraints: A unified view , 2011, 2011 IEEE International Conference on Robotics and Automation.

[100]  Alexander Herzog,et al.  Momentum control with hierarchical inverse dynamics on a torque-controlled humanoid , 2014, Autonomous Robots.

[101]  V. Borkar,et al.  A unified framework for hybrid control: model and optimal control theory , 1998, IEEE Trans. Autom. Control..

[102]  A. Ijspeert,et al.  Dynamic hebbian learning in adaptive frequency oscillators , 2006 .

[103]  K. Ohno A new approach to differential dynamic programming for discrete time systems , 1978 .

[104]  Pierre-Brice Wieber,et al.  Trajectory Free Linear Model Predictive Control for Stable Walking in the Presence of Strong Perturbations , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[105]  J. Bobrow,et al.  An efficient sequential linear quadratic algorithm for solving nonlinear optimal control problems , 2005 .

[106]  Robin Deits,et al.  Footstep planning on uneven terrain with mixed-integer convex optimization , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[107]  Darwin G. Caldwell,et al.  Trajectory and foothold optimization using low-dimensional models for rough terrain locomotion , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[108]  Jonas Buchli,et al.  Efficient whole-body trajectory optimization using contact constraint relaxation , 2016, 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids).

[109]  George M. Siouris,et al.  Applied Optimal Control: Optimization, Estimation, and Control , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[110]  Daniel P. Ferris,et al.  Running in the real world: adjusting leg stiffness for different surfaces , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[111]  Andrew Y. Ng,et al.  A control architecture for quadruped locomotion over rough terrain , 2008, 2008 IEEE International Conference on Robotics and Automation.

[112]  Todd D. Murphey,et al.  Second-Order Switching Time Optimization for Nonlinear Time-Varying Dynamic Systems , 2011, IEEE Transactions on Automatic Control.

[113]  Ferdinando Cannella,et al.  Design of HyQ – a hydraulically and electrically actuated quadruped robot , 2011 .

[114]  Sami Haddadin,et al.  Soft robotics for the hydraulic atlas arms: Joint impedance control with collision detection and disturbance compensation , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[115]  Martin Jaggi,et al.  Revisiting Frank-Wolfe: Projection-Free Sparse Convex Optimization , 2013, ICML.

[116]  T. Flash,et al.  Task-Dependent Selection of Grasp Kinematics and Stiffness in Human Object Manipulation , 2007, Cortex.

[117]  Pierre-Brice Wieber,et al.  Fast Direct Multiple Shooting Algorithms for Optimal Robot Control , 2005 .

[118]  Darwin G. Caldwell,et al.  Planning and execution of dynamic whole-body locomotion for a hydraulic quadruped on challenging terrain , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[119]  D. Mayne A Second-order Gradient Method for Determining Optimal Trajectories of Non-linear Discrete-time Systems , 1966 .

[120]  Roland Siegwart,et al.  Fast nonlinear Model Predictive Control for unified trajectory optimization and tracking , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[121]  M. Kumagai,et al.  Development of a robot balancing on a ball , 2008, 2008 International Conference on Control, Automation and Systems.

[122]  H. Kappen Path integrals and symmetry breaking for optimal control theory , 2005, physics/0505066.

[123]  Emanuel Todorov,et al.  Linearly-solvable Markov decision problems , 2006, NIPS.

[124]  Lydia Tapia,et al.  Learning swing-free trajectories for UAVs with a suspended load , 2013, 2013 IEEE International Conference on Robotics and Automation.

[125]  Jun Nakanishi,et al.  Learning Attractor Landscapes for Learning Motor Primitives , 2002, NIPS.

[126]  Jonas Buchli,et al.  Trajectory Optimization Through Contacts and Automatic Gait Discovery for Quadrupeds , 2016, IEEE Robotics and Automation Letters.

[127]  Aude Billard,et al.  Learning Compliant Manipulation through Kinesthetic and Tactile Human-Robot Interaction , 2014, IEEE Transactions on Haptics.

[128]  Rhodes,et al.  Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games , 1973 .

[129]  Miomir Vukobratovic,et al.  Contribution to the study of anthropomorphic systems , 1972, Kybernetika.

[130]  Scott Kuindersma,et al.  Modeling and Control of Legged Robots , 2016, Springer Handbook of Robotics, 2nd Ed..

[131]  Spyros G. Tzafestas,et al.  Robust Sliding-mode Control of Nine-link Biped Robot Walking , 1997, J. Intell. Robotic Syst..

[132]  H. Sussmann,et al.  A maximum principle for hybrid optimal control problems , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).

[133]  Jonas Buchli,et al.  Risk Sensitive, Nonlinear Optimal Control: Iterative Linear Exponential-Quadratic Optimal Control with Gaussian Noise , 2015, ArXiv.

[134]  Athanasios Sideris,et al.  A Riccati approach for constrained linear quadratic optimal control , 2011, Int. J. Control.

[135]  G. Oriolo,et al.  Robotics: Modelling, Planning and Control , 2008 .

[136]  Daniel D. Lee,et al.  Search-based planning for a legged robot over rough terrain , 2009, 2009 IEEE International Conference on Robotics and Automation.

[137]  Vijay Kumar,et al.  Trajectory Generation and Control for Precise Aggressive Maneuvers with Quadrotors , 2010, ISER.

[138]  Martin A. Riedmiller,et al.  Approximate real-time optimal control based on sparse Gaussian process models , 2014, 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[139]  Vijay Kumar,et al.  Dynamics, Control and Planning for Cooperative Manipulation of Payloads Suspended by Cables from Multiple Quadrotor Robots , 2013, Robotics: Science and Systems.

[140]  Ronald Lumia,et al.  Rapid Swing-Free Transport of Nonlinear Payloads Using Dynamic Programming , 2008 .

[141]  Twan Koolen,et al.  Summary of Team IHMC's virtual robotics challenge entry , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[142]  D. Mayne,et al.  Sequential quadratic programming algorithm for discrete optimal control problems with control inequality constraints , 1991 .

[143]  Jonas Buchli,et al.  Learning of closed-loop motion control , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[144]  Roland Siegwart,et al.  Reinforcement learning of single legged locomotion , 2013, IROS 2013.

[145]  Péter Fankhauser,et al.  Modeling and Control of a Ballbot , 2010 .

[146]  Darwin G. Caldwell,et al.  Dynamic torque control of a hydraulic quadruped robot , 2012, 2012 IEEE International Conference on Robotics and Automation.

[147]  Olivier Stasse,et al.  Whole-body model-predictive control applied to the HRP-2 humanoid , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[148]  Panos J. Antsaklis,et al.  Optimal control of switched systems based on parameterization of the switching instants , 2004, IEEE Transactions on Automatic Control.

[149]  Pieter Abbeel,et al.  Value Iteration Networks , 2016, NIPS.

[150]  Edo Jelavic,et al.  Robust whole-body motion control of legged robots , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[151]  Stefan Schaal,et al.  Reinforcement Learning With Sequences of Motion Primitives for Robust Manipulation , 2012, IEEE Transactions on Robotics.

[152]  O. V. Stryk,et al.  Numerical Solution of Optimal Control Problems by Direct Collocation , 1993 .

[153]  Markus H. Gross,et al.  Hierarchical planning and control for complex motor tasks , 2015, Symposium on Computer Animation.

[154]  Jerry E. Pratt,et al.  A Controller for the LittleDog Quadruped Walking on Rough Terrain , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[155]  Alin Albu-Schäffer,et al.  Human-Like Adaptation of Force and Impedance in Stable and Unstable Interactions , 2011, IEEE Transactions on Robotics.

[156]  F. Allgöwer,et al.  A quasi-infinite horizon nonlinear model predictive control scheme with guaranteed stability , 1997 .

[157]  Kai Henning Koch,et al.  Optimization-Based Walking Generation for Humanoid Robot , 2012, SyRoCo.

[158]  S. K. MIX,et al.  SUCCESSIVE APPROXIMATION METHODS FOR THE SOLUTION OF OPTIMAL CONTROL PROBLEMS , 2002 .

[159]  H. Kappen,et al.  Path integral control and state-dependent feedback. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[160]  Roland Siegwart,et al.  Practice Makes Perfect: An Optimization-Based Approach to Controlling Agile Motions for a Quadruped Robot , 2016, IEEE Robotics & Automation Magazine.

[161]  Yuval Tassa Fast Model Predictive Control for Reactive Robotic Swimming , 2010 .

[162]  Dominik Belter,et al.  RRT-BASED MOTION PLANNER AND BALANCE CONTROLLER FOR A BIPED ROBOT , 2016 .

[163]  Gaurav S. Sukhatme,et al.  Combining Model-Based and Model-Free Updates for Deep Reinforcement Learning , 2017 .

[164]  Twan Koolen,et al.  Team IHMC's Lessons Learned from the DARPA Robotics Challenge Trials , 2015, J. Field Robotics.

[165]  Christopher G. Atkeson,et al.  Robust dynamic walking using online foot step optimization , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[166]  Olivier Sigaud,et al.  Policy Improvement Methods: Between Black-Box Optimization and Episodic Reinforcement Learning , 2012 .

[167]  Evangelos Theodorou,et al.  Tendon-driven control of biomechanical and robotic systems: A path integral reinforcement learning approach , 2012, 2012 IEEE International Conference on Robotics and Automation.

[168]  Friedrich Pfeiffer,et al.  A collocation method for real-time walking pattern generation , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[169]  Nicolas Mansard,et al.  A dedicated solver for fast operational-space inverse dynamics , 2012, 2012 IEEE International Conference on Robotics and Automation.

[170]  Stefan Schaal,et al.  A Generalized Path Integral Control Approach to Reinforcement Learning , 2010, J. Mach. Learn. Res..

[171]  Michael Neunert,et al.  Online Walking Motion and Contact Optimization for Quadruped Robots , 2017, ICRA 2017.

[172]  Masayuki Inaba,et al.  Online decision of foot placement using singular LQ preview regulation , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[173]  I. Postlethwaite,et al.  Accounting for uncertainty in anti-windup synthesis , 2004, Proceedings of the 2004 American Control Conference.

[174]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[175]  Vijay Kumar,et al.  Design, modeling, estimation and control for aerial grasping and manipulation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[176]  Farbod Fahimi,et al.  Robust control of underactuated bipeds using sliding modes , 2007, Robotica.

[177]  Andrei Herdt,et al.  Online Walking Motion Generation with Automatic Footstep Placement , 2010, Adv. Robotics.

[178]  Farhad Aghili,et al.  A unified approach for inverse and direct dynamics of constrained multibody systems based on linear projection operator: applications to control and simulation , 2005, IEEE Transactions on Robotics.

[179]  François Keith,et al.  Dynamic Whole-Body Motion Generation Under Rigid Contacts and Other Unilateral Constraints , 2013, IEEE Transactions on Robotics.

[180]  Nikolaos G. Tsagarakis,et al.  Robust and adaptive whole-body controller for humanoids with multiple tasks under uncertain disturbances , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[181]  Mohamed A. Zohdy,et al.  Robust control of biped robots , 2000, Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No.00CH36334).