Advanced Walking Skills for Bipedal Locomotion

Most control algorithms for current bipedal robots relied on the stability analysis of ZeroMoment Point (ZMP), which was rstly introduced by Vukobratović in 1968. However, control approaches based on ZMP have yet common shortcomings, such as, unnatural motions, high energy costs, high computational burden, absence of natural dynamics. Due to their disadvantages, many researchers studied the biological and biomechanical aspects of human locomotion, from which valuable ndings can be transferred to the design of control approaches for bipedal walking robots. Among them, a Bio-Inspired Behavior-Based Bipedal Locomotion Control (B4LC) was developed by Luksch based on the concepts found in human motion control with the hypothesis that a control system similar to human one will enable a robot to perform locomotion in a likewise fashion. Examining the achievement of the B4LC approach, it can be observed that several re nements and extensions are desirable. The current walking skills contain walking initiation, cyclic walking and standing with perturbations. Moreover, to extend the walking skills of a humanlike robot, certain advanced walking skills, such as, walking termination, curve walking, and upslope walking in walking are studied and control methods for them are designed based on biological and biomechanical literature. To enhance the robustness of bipedal walking in more challenging environments, a control structure is thus required. It features detection and rejection of unexpected disturbances during walking, especially the external pushes. Based on the processing of sensory information, di erent control strategies should be applied according to the state of the biped to come to a relative stable state. The external pushes can come from sagittal and frontal plane and happen during certain time instants of a gait cycle. Investigating the literature in push recovery of human walking leads to development of speci c control strategies for joints in lower limb. Besides the external disturbances, voluntary control from robot itself may also result in instability of the robot. Neuroscientists assume that ve electromyography (emg) components account for muscle activity during walking under di erent speeds. This nding brings us the implication for designing the corresponding motor patterns across di erent walking speeds. With approximating those ve emg components to Gaussian curves, it can deduce the problem as the search of parameters representing those curves, which are the mean values, the variances and the peak of amplitudes, respectively. By employing the Swarm Particle Optimization method, parameters for those motor patterns can be automatically found and optimized in terms of stability and energy cost. Analyzing all parameters within di erent walking speeds, construction of a function of parameters for motor patterns across walking speeds can be accomplished. Furthermore, the drawback of manually tuning process for parameters of the control units and of the biped model still persists. To nd optimal control parameters, the introduction of

[1]  H. Barbeau,et al.  Adaptation of the walking pattern to uphill walking in normal and spinal-cord injured subjects , 1999, Experimental Brain Research.

[2]  S. Stigler Gauss and the Invention of Least Squares , 1981 .

[3]  Karsten Berns,et al.  Adaptive motor patterns and reflexes for bipedal locomotion on rough terrain , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[4]  A. Takanishi,et al.  Development of a Humanoid Robot Capable of Leaning on a Walk-assist Machine , 2006, The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006..

[5]  Vinutha Kallem,et al.  Rate of change of angular momentum and balance maintenance of biped robots , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[6]  E. Ozcan,et al.  Particle swarm optimization: surfing the waves , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[7]  Jean Pailhous,et al.  Intentional on-line adaptation of stride length in human walking , 1999, Experimental Brain Research.

[8]  A B Schultz,et al.  Age and gender differences in single-step recovery from a forward fall. , 1999, The journals of gerontology. Series A, Biological sciences and medical sciences.

[9]  Frans van den Bergh,et al.  An analysis of particle swarm optimizers , 2002 .

[10]  Kenji KANEKO,et al.  Humanoid robot HRP-3 , 2004, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[11]  A. Thorstensson,et al.  Trunk movements in human locomotion. , 1984, Acta physiologica Scandinavica.

[12]  Kazuhito Yokoi,et al.  The 3D linear inverted pendulum mode: a simple modeling for a biped walking pattern generation , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[13]  K. Cheng,et al.  Effect of arm swing on single-step balance recovery. , 2014, Human movement science.

[14]  F. Lacquaniti,et al.  Five basic muscle activation patterns account for muscle activity during human locomotion , 2004, The Journal of physiology.

[15]  G. W. Lange,et al.  Electromyographic and kinematic analysis of graded treadmill walking and the implications for knee rehabilitation. , 1996, The Journal of orthopaedic and sports physical therapy.

[16]  Jan Peters,et al.  Policy Search for Motor Primitives in Robotics , 2008, NIPS 2008.

[17]  A. Hof,et al.  Speed dependence of averaged EMG profiles in walking. , 2002, Gait & posture.

[18]  Jun Morimoto,et al.  Learning CPG-based Biped Locomotion with a Policy Gradient Method: Application to a Humanoid Robot , 2005, 5th IEEE-RAS International Conference on Humanoid Robots, 2005..

[19]  A. Hof The 'extrapolated center of mass' concept suggests a simple control of balance in walking. , 2008, Human movement science.

[20]  Alin Albu-Schäffer,et al.  Bipedal walking control based on Capture Point dynamics , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Jie Zhao,et al.  Controlling Human-like Locomotion of a Biped by a Biologically Motivated Approach , 2011 .

[22]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[23]  M. Clerc,et al.  The swarm and the queen: towards a deterministic and adaptive particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[24]  Jerry E. Pratt,et al.  Learning Capture Points for humanoid push recovery , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[25]  Twan Koolen,et al.  Capturability-based analysis and control of legged locomotion, Part 2: Application to M2V2, a lower-body humanoid , 2012, Int. J. Robotics Res..

[26]  J. Kuhtz-Buschbeck,et al.  Stable patterns of upper limb muscle activation in different conditions of human walking , 2015 .

[27]  A. Arampatzis,et al.  Dynamic stability control in forward falls: postural corrections after muscle fatigue in young and older adults , 2008, European Journal of Applied Physiology.

[28]  Elena Torta,et al.  Evaluation of a Small Socially-Assistive Humanoid Robot in Intelligent Homes for the Care of the Elderly , 2014, J. Intell. Robotic Syst..

[29]  H. Sebastian Seung,et al.  Learning to Walk in 20 Minutes , 2005 .

[30]  Todd A Kuiken,et al.  Trip recovery strategies following perturbations of variable duration. , 2014, Journal of biomechanics.

[31]  Takashi Matsumoto,et al.  Real time motion generation and control for biped robot -1st report: Walking gait pattern generation- , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[32]  Richard S. Sutton,et al.  Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming , 1990, ML.

[33]  Shuuji Kajita,et al.  Creating facial motions of Cybernetic Human HRP-4C , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[34]  Christian Ott,et al.  Control applications of TORO — A Torque controlled humanoid robot , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[35]  Prachuab Vanitchatchavan Termination of human gait , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[36]  M Vukobratović,et al.  On the stability of biped locomotion. , 1970, IEEE transactions on bio-medical engineering.

[37]  K. Berns,et al.  A Learning Arcitecture based on Reinforcement Learning for Adaptive Control of the Walking Machine , 1995 .

[38]  Darwin G. Caldwell,et al.  Robot motor skill coordination with EM-based Reinforcement Learning , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[39]  C. Cooke,et al.  Kinematic and postural characteristics of sprint running on sloping surfaces , 2001, Journal of sports sciences.

[40]  Karsten Berns,et al.  Experimental verification of an approach for disturbance estimation and compensation on a simulated biped during perturbed stance , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[41]  Taku Komura,et al.  A Feedback Controller for Biped Humanoids that Can Counteract Large Perturbations During Gait , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[42]  Karsten Berns,et al.  Introducing FINROC: A Convenient Real-time Framework for Robotics based on a Systematic Design Approach , 2012 .

[43]  Christopher Rasmussen A hybrid vision + ladar rural road follower , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[44]  Karsten Berns,et al.  Unmanned Ground Vehicles for Urban Search and Rescue in the {EU-FP7} Project {ICARUS} , 2014 .

[45]  Kazuhito Yokoi,et al.  Balance control based on Capture Point error compensation for biped walking on uneven terrain , 2012, 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012).

[46]  A. E. Patla,et al.  Online steering: coordination and control of body center of mass, head and body reorientation , 1999, Experimental Brain Research.

[47]  Jie Zhao,et al.  Integrating Capture Point into Biologically Motivated Controlled Biped for Maintaining Stable State , 2012 .

[48]  Shuuji Kajita,et al.  Cybernetic human HRP-4C , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[49]  Stefan Schaal,et al.  Policy Gradient Methods for Robotics , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[50]  Karsten Berns,et al.  Construction site navigation for the autonomous excavator Thor , 2015, 2015 6th International Conference on Automation, Robotics and Applications (ICARA).

[51]  A B Schultz,et al.  Muscle activities used by young and old adults when stepping to regain balance during a forward fall. , 2000, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[52]  Albertus Hendrawan Adiwahono,et al.  Push Recovery through walking phase Modification for bipedal locomotion , 2013, Int. J. Humanoid Robotics.

[53]  Shiyuan Yang,et al.  Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm , 2007, Inf. Process. Lett..

[54]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[55]  Karsten Berns,et al.  Biologically motivated push recovery strategies for a 3D bipedal robot walking in complex environments , 2013, 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[56]  Karsten Berns,et al.  Learning a reactive posture control on the four-legged walking machine BISAM , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[57]  F. Horak Postural orientation and equilibrium: what do we need to know about neural control of balance to prevent falls? , 2006, Age and ageing.

[58]  Karsten Berns,et al.  Learning Control of a Six-Legged Walking Machine , 1994 .

[59]  Gwi-Tae Park,et al.  Study on the Mobility of Service Robots , 2012 .

[60]  George L. Nemhauser,et al.  Handbooks in operations research and management science , 1989 .

[61]  R. Blickhan,et al.  The tri-segmented limbs of therian mammals: kinematics, dynamics, and self-stabilization--a review. , 2006, Journal of experimental zoology. Part A, Comparative experimental biology.

[62]  D. T. Greenwood Principles of dynamics , 1965 .

[63]  Miomir Vukobratovic,et al.  Zero-Moment Point - Thirty Five Years of its Life , 2004, Int. J. Humanoid Robotics.

[64]  Olivier Stasse,et al.  Real-time replanning using 3D environment for humanoid robot , 2011, 2011 11th IEEE-RAS International Conference on Humanoid Robots.

[65]  Ji-Yong Lee,et al.  Autonomous task execution of a humanoid robot using a cognitive model , 2010, 2010 IEEE International Conference on Robotics and Biomimetics.

[66]  A. Schultz,et al.  Effects of age on rapid ankle torque development. , 1996, The journals of gerontology. Series A, Biological sciences and medical sciences.

[67]  Francesco Lacquaniti,et al.  Motor Control Programs and Walking , 2006, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[68]  S. Rossignol,et al.  Dynamic sensorimotor interactions in locomotion. , 2006, Physiological reviews.

[69]  A. McIntosh,et al.  Gait dynamics on an inclined walkway. , 2006, Journal of biomechanics.

[70]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[71]  Kikuo Fujimura,et al.  The intelligent ASIMO: system overview and integration , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[72]  Sergey V. Drakunov,et al.  Capture Point: A Step toward Humanoid Push Recovery , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[74]  Karsten Berns,et al.  An Activation-Based Behavior Control Architecture for Walking Machines , 2002, Int. J. Robotics Res..

[75]  Karsten Berns,et al.  Seamless Extension of a Robot Control Framework to Bare Metal Embedded Nodes , 2014, GI-Jahrestagung.

[76]  H. Barbeau,et al.  Postural adaptation to walking on inclined surfaces: I. Normal strategies. , 2002, Gait & posture.

[77]  Holly A. Yanco,et al.  Analysis of Human‐robot Interaction at the DARPA Robotics Challenge Trials , 2015, J. Field Robotics.

[78]  Tobias Luksch,et al.  Human-like Control of Dynamically Walking Bipedal Robots , 2010 .

[79]  E B Simonsen,et al.  Excitability of the soleus H reflex during graded walking in humans. , 1995, Acta physiologica Scandinavica.

[80]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[81]  Karsten Berns,et al.  Safe predictive mobile robot navigation in aware environments , 2015, 2015 12th International Conference on Informatics in Control, Automation and Robotics (ICINCO).

[82]  D. Jahnigen,et al.  Kinematics of recovery from a stumble. , 1993, Journal of gerontology.

[83]  R. B. Stein,et al.  Function of sural nerve reflexes during human walking , 1998, The Journal of physiology.

[84]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[85]  Todd A. Kuiken,et al.  The effect of perturbation onset timing and length on tripping recovery strategies , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[86]  A. Arampatzis,et al.  Age-related degeneration in leg-extensor muscle–tendon units decreases recovery performance after a forward fall: compensation with running experience , 2006, European Journal of Applied Physiology.

[87]  Jun Morimoto,et al.  A simple reinforcement learning algorithm for biped walking , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[88]  Karsten Berns,et al.  Vision-Based Person Detection for Safe Navigation of Commercial Vehicle , 2014, IAS.

[89]  Simeon M. Berman Mathematical statistics;: An introduction based on the normal distribution , 1971 .

[90]  F. Lacquaniti,et al.  Motor patterns in human walking and running. , 2006, Journal of neurophysiology.

[91]  Todd A. Kuiken,et al.  Recovery strategy identification throughout swing phase using kinematic data from the tripped leg , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[92]  Hun-ok Lim,et al.  Algorithm of pattern generation for mimicking disabled person’s gait , 2008, 2008 2nd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics.

[93]  Shirley Rietdyk,et al.  Context-dependent reflex control: some insights into the role of balance , 1998, Experimental Brain Research.

[94]  Karsten Berns,et al.  Trajectory Planning and Lateral Control for Agricultural Guidance Applications , 2013 .

[95]  A E Patla,et al.  Intralimb dynamics simplify reactive control strategies during locomotion. , 1997, Journal of biomechanics.

[96]  Herman Pontzer,et al.  Bipedal and quadrupedal locomotion in chimpanzees. , 2014, Journal of human evolution.

[97]  A B Schultz,et al.  Stepping over obstacles: dividing attention impairs performance of old more than young adults. , 1996, The journals of gerontology. Series A, Biological sciences and medical sciences.

[98]  H Forssberg,et al.  Phase-dependent modulations of anticipatory postural activity during human locomotion. , 1991, Journal of neurophysiology.

[99]  Karsten Berns,et al.  Principles in Framework Design Applied in Networked Robotics , 2013, TA.

[100]  Joseph Hamill,et al.  Biomechanical Basis of Human Movement , 1995 .

[101]  M. Tinetti,et al.  Fall risk index for elderly patients based on number of chronic disabilities. , 1986, The American journal of medicine.

[102]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[103]  Benjamin J. Stephens,et al.  Push Recovery Control for Force-Controlled Humanoid Robots , 2011 .

[104]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[105]  Karsten Berns,et al.  Mobile Robots in Smart Environments: The Current Situation , 2012, AMS.

[106]  Karsten Berns,et al.  Development of complex robotic systems using the behavior-based control architecture iB2C , 2010, Robotics Auton. Syst..

[107]  Kazuhito Yokoi,et al.  Whole-body motion of a humanoid robot for passing through a door - opening a door by impulsive force - , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[108]  Karsten Berns,et al.  Formal verification of safety behaviours of the outdoor robot ravon , 2007, ICINCO-RA.

[109]  Kazuhito Yokoi,et al.  Biped walking pattern generation by using preview control of zero-moment point , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[110]  Michael Günther,et al.  Intelligence by mechanics , 2007, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[111]  Albertus Hendrawan Adiwahono,et al.  Push recovery controller for bipedal robot walking , 2009, 2009 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.

[112]  Ji-Yong Lee,et al.  Providing services using network-based humanoids in a home environment , 2011, IEEE Transactions on Consumer Electronics.

[113]  Jan Peters,et al.  Learning motor primitives for robotics , 2009, 2009 IEEE International Conference on Robotics and Automation.

[114]  Andrea N. Lay,et al.  The effects of sloped surfaces on locomotion: a kinematic and kinetic analysis. , 2006, Journal of biomechanics.

[115]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[116]  Laura A. Wojcik,et al.  Age differences in using a rapid step to regain balance during a forward fall. , 1997, The journals of gerontology. Series A, Biological sciences and medical sciences.

[117]  Constantinos N Maganaris,et al.  Tripping without falling; lower limb strength, a limitation for balance recovery and a target for training in the elderly. , 2008, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[118]  D. Winter,et al.  Strategies for recovery from a trip in early and late swing during human walking , 2004, Experimental Brain Research.

[119]  Alin Albu-Schäffer,et al.  Overview of the torque-controlled humanoid robot TORO , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[120]  Stefan Schaal,et al.  Scalable Techniques from Nonparametric Statistics for Real Time Robot Learning , 2002, Applied Intelligence.

[121]  Martin Fodslette Meiller A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning , 1993 .

[122]  Stefan Schaal,et al.  Reinforcement learning by reward-weighted regression for operational space control , 2007, ICML '07.

[123]  Karsten Berns,et al.  A mechanism of Particle Swarm Optimization on motor patterns in the {B4LC} system , 2015 .

[124]  M C Do,et al.  A biomechanical study of balance recovery during the fall forward. , 1982, Journal of biomechanics.

[125]  M. Rodgers Dynamic biomechanics of the normal foot and ankle during walking and running. , 1988, Physical therapy.

[126]  R. Stein,et al.  Analysis of rapid stopping during human walking. , 1998, Journal of neurophysiology.

[127]  H. Sebastian Seung,et al.  Stochastic policy gradient reinforcement learning on a simple 3D biped , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[128]  Francesco Lacquaniti,et al.  Patterned control of human locomotion , 2012, The Journal of physiology.

[129]  A L Hof,et al.  The condition for dynamic stability. , 2005, Journal of biomechanics.

[130]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.