Generating walking behaviours in legged robots

Many legged robots have been built with a variety of different abilities, from running to hopping to climbing stairs. Despite this however, there has been no consistency of approach to the problem of getting them to walk. Approaches have included breaking down the walking step into discrete parts and then controlling them separately, using springs and linkages to achieve a passive walking cycle, and even working out the necessary movements in simulation and then imposing them on the real robot. All of these have limitations, although most were successful at the task for which they were designed. However, all of them fall into one of two categories: either they alter the dynamics of the robots physically so that the robot, whilst very good at walking, is not as general purpose as it once was (as with the passive robots), or they control the physical mechanism of the robot directly to achieve their goals, and this is a difficult task. In this thesis a design methodology is described for building controllers for 3D dynamically stable walking, inspired by the best walkers and runners around — ourselves — so the controllers produced are based on the vertebrate Central Nervous System. This means that there is a low-level controller which adapts itself to the robot so that, when switched on, it can be considered to simulate the springs and linkages of the passive robots to produce a walking robot, and this now active mechanism is then controlled by a relatively simple higher level controller. This is the best of both worlds — we have a robot which is inherently capable of walking, and thus is easy to control like the passive walkers, but also retains the general purpose abilities which makes it so potentially useful. This design methodology uses an evolutionary algorithm to generate low-level controllers for a selection of simulated legged robots. The thesis also looks in detail at previous walking robots and their controllers and shows that some approaches, including staged evolution and handcoding designs, may be unnecessary, and indeed inappropriate, at least for a general purpose controller. The specific algorithm used is evolutionary, using a simple genetic algorithm to allow adaptation to different robot configurations, and the controllers evolved are continuous time neural networks. These are chosen because of their ability to entrain to the movement of the robot, allowing the whole robot and network to be considered as a single dynamical system, which can then be controlled by a higher level system. An extensive program of experiments investigates the types of neural models and network structures which are best suited to this task, and it is shown that stateless and simple dynamic neural models are significantly outperformed as controllers by more complex, biologically plausible ones but that other ideas taken from biological systems, including network connectivities, are not generally as useful and reasons for this are examined. The thesis then shows that this system, although only developed on a single robot, is capable of automatically generating controllers for a wide selection of different test designs. Finally it shows that high level controllers, at least to control steering and speed, can be easily built on top of this now active walking mechanism.

[1]  Hiroaki Kitano Cell Differentiation and Neurogenesis in Evolutionary Large Scale Chaos , 1995, ECAL.

[2]  Jean-Arcady Meyer,et al.  Evolution and development of neural controllers for locomotion, gradient-following, and obstacle-avoidance in artificial insects , 1998, IEEE Trans. Neural Networks.

[3]  S.J.J. Smith,et al.  Empirical Methods for Artificial Intelligence , 1995 .

[4]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[5]  David R. Jefferson,et al.  An Artificial Neural Network Representation for Artificial Organisms , 1990, PPSN.

[6]  Jessica K. Hodgins,et al.  Three-dimensional human running , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[7]  Una-May O'Reilly,et al.  Genetic Programming II: Automatic Discovery of Reusable Programs. , 1994, Artificial Life.

[8]  Graham F. Spencer,et al.  Automatic Generation of Programs for Crawling and Walking , 1993, International Conference on Genetic Algorithms.

[9]  Tad McGeer,et al.  Passive Dynamic Walking , 1990, Int. J. Robotics Res..

[10]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[11]  D. Wolpert,et al.  Is the cerebellum a smith predictor? , 1993, Journal of motor behavior.

[12]  Ádám Rotaru-Varga Modularity in Evolved Artificial Neural Networks , 1999, ECAL.

[13]  Hugo de Garis,et al.  Genetic Programming , 1990, ML.

[14]  A. Berger FUNDAMENTALS OF BIOSTATISTICS , 1969 .

[15]  M. A. Lewis,et al.  Self-organization of locomotory controllers in robots and animals , 1996 .

[16]  H. Benjamin Brown,et al.  Experiments in Balance with a 3D One-Legged Hopping Machine , 1984 .

[17]  M.-Y. Cheng,et al.  Measurement of robustness for biped locomotion using a linearized Poincaré map , 1996, Robotica.

[18]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[19]  Stefano Nolfi,et al.  How to Evolve Autonomous Robots: Different Approaches in Evolutionary Robotics , 1994 .

[20]  A. Opstal Dynamic Patterns: The Self-Organization of Brain and Behavior , 1995 .

[21]  Randall D. Beer,et al.  On the Dynamics of Small Continuous-Time Recurrent Neural Networks , 1995, Adapt. Behav..

[22]  Karl Sims,et al.  Evolving 3D Morphology and Behavior by Competition , 1994, Artificial Life.

[23]  Rodney A. Brooks,et al.  A Robot that Walks; Emergent Behaviors from a Carefully Evolved Network , 1989, Neural Computation.

[24]  R. McN. Alexander,et al.  Three Uses for Springs in Legged Locomotion , 1990, Int. J. Robotics Res..

[25]  Jean-Arcady Meyer,et al.  Evolution and development of control architectures in animats , 1995, Robotics Auton. Syst..

[26]  R. McN. Alexander,et al.  The Gaits of Bipedal and Quadrupedal Animals , 1984 .

[27]  Simon Perkins,et al.  Incremental Aquisition of Complex Visual Behaviour using Genetic Programming and Robot Shaping , 1998 .

[28]  G. P. van Galen,et al.  Coordination of multi-joint movements: An introduction to emerging views , 1995 .

[29]  R. M. Alexander Energy-saving mechanisms in walking and running. , 1991, The Journal of experimental biology.

[30]  D. Parisi,et al.  Growing neural networks , 1991 .

[31]  Karl Sims,et al.  Evolving virtual creatures , 1994, SIGGRAPH.

[32]  Tad McGeer Powered flight, child's play, silly wheels and walking machines , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[33]  Bernard Espiau,et al.  Limit Cycles in a Passive Compass Gait Biped and Passivity-Mimicking Control Laws , 1997, Auton. Robots.

[34]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[35]  Brian Hopkins,et al.  Modelling the early development of bipedal locomotion: A multidisciplinary approach , 1995 .

[36]  S. Grillner,et al.  A computer-based model for realistic simulations of neural networks. II. The segmental network generating locomotor rhythmicity in the lamprey. , 1992 .

[37]  Brad Fullmer and Risto Miikkulainen Using Marker-Based Genetic Encoding Of Neural Networks To Evolve Finite-State Behaviour , 1991 .

[38]  Rodney A. Brooks,et al.  A robot that walks; emergent behaviors from a carefully evolved network , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[39]  F.E. Zajac,et al.  Restoring unassisted natural gait to paraplegics via functional neuromuscular stimulation: a computer simulation study , 1990, IEEE Transactions on Biomedical Engineering.

[40]  Marc H. Raibert,et al.  Legged Robots That Balance , 1986, IEEE Expert.

[41]  Tariq Samad,et al.  Towards the Genetic Synthesisof Neural Networks , 1989, ICGA.

[42]  Peter J. Angeline Evolutionary algorithms and emergent intelligence , 1993 .

[43]  Jessica K. Hodgins,et al.  Slipping and Tripping Reflexes for Bipedal Robots , 1997, Auton. Robots.

[44]  J. Szentágothai,et al.  Brain Research , 2009, Experimental Neurology.

[45]  Jan Torreele,et al.  Temporal Processing with Recurrent Networks: An Evolutionary Approach , 1991, ICGA.

[46]  David Eilam,et al.  Comparative Morphology of Locomotion in Vertebrates , 1995 .

[47]  Robert Playter,et al.  Passive Dynamics in the Control of Gymnastic Maneuvers , 1995 .

[48]  Randall D. Beer,et al.  A qualitative dynamical analysis of evolved locomotion controllers , 1993 .

[49]  Olivier Michel,et al.  From the Chromosome to the Neural Network , 1995, ICANNGA.

[50]  William Whittaker,et al.  Exploring Mount Erebus by walking robot , 1993, Robotics Auton. Syst..

[51]  Jessica K. Hodgins,et al.  Dynamically Stable Legged Locomotion , 1983 .

[52]  F. Delcomyn Neural basis of rhythmic behavior in animals. , 1980, Science.

[53]  Robert L. Schaefer,et al.  Introduction to Contemporary Statistical Methods , 1988 .

[54]  L. Darrell Whitley,et al.  Optimizing Neural Networks Using FasterMore Accurate Genetic Search , 1989, ICGA.

[55]  David H. Ackley,et al.  Interactions between learning and evolution , 1991 .

[56]  E. Marsolais,et al.  Functional walking in paralyzed patients by means of electrical stimulation. , 1983, Clinical orthopaedics and related research.

[57]  D. Willshaw,et al.  Artificial Lampreys: Comparing Naturally and Artificially Evolved Swimming Controllers , 1997 .

[58]  Vittorio Maniezzo,et al.  Genetic evolution of the topology and weight distribution of neural networks , 1994, IEEE Trans. Neural Networks.

[59]  Laci Jalics,et al.  Rocking, Tapping and Stepping: A Prelude to Dance , 1997, Auton. Robots.

[60]  Richard D. Braatz,et al.  On the "Identification and control of dynamical systems using neural networks" , 1997, IEEE Trans. Neural Networks.

[61]  D. Parisi,et al.  “Genotypes” for neural networks , 1998 .

[62]  Hiroaki Kitano,et al.  Designing Neural Networks Using Genetic Algorithms with Graph Generation System , 1990, Complex Syst..

[63]  I. Shimoyama,et al.  Dynamic Walk of a Biped , 1984 .

[64]  Michiel van de Panne,et al.  Parameterized gait synthesis , 1996, IEEE Computer Graphics and Applications.

[65]  Shigeo Hirose,et al.  Intermittent trot gait of a quadruped walking machine dynamic stability control of an omnidirectional walk , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[66]  Thomas Kindermann,et al.  Walking: A Complex Behavior Controlled by Simple Networks , 1995, Adapt. Behav..

[67]  Randall D. Beer,et al.  Evolving Dynamical Neural Networks for Adaptive Behavior , 1992, Adapt. Behav..

[68]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[69]  É. Marey,et al.  Animal mechanism : a treatise on terrestrial and aerial locomotion , 2022 .

[70]  I. Stewart,et al.  Coupled nonlinear oscillators and the symmetries of animal gaits , 1993 .

[71]  Randall D. Beer,et al.  Application of evolved locomotion controllers to a hexapod robot , 1996, Robotics Auton. Syst..

[72]  S. Grillner,et al.  Neuronal network generating locomotor behavior in lamprey: circuitry, transmitters, membrane properties, and simulation. , 1991, Annual review of neuroscience.

[73]  A. P. Wieland,et al.  Evolving neural network controllers for unstable systems , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[74]  V. S. Gurfinkel,et al.  Walking robot with supervisory control , 1981 .

[75]  Alexander Rm,et al.  Energy-saving mechanisms in walking and running. , 1991 .

[76]  Andrew H. Fagg,et al.  Genetic programming approach to the construction of a neural network for control of a walking robot , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[77]  Norman I. Badler,et al.  Animating human locomotion with inverse dynamics , 1996, IEEE Computer Graphics and Applications.

[78]  D. Parisi,et al.  Phylogenetic Recapitulation in the Ontogeny of Artificial Neural Networks , 1993 .

[79]  Roy Featherstone,et al.  Robot Dynamics Algorithms , 1987 .

[80]  A. Cangelosi,et al.  Cell division and migration in a 'genotype' for neural networks (Cell division and migration in neural networks) , 1993 .

[81]  Scott McMillan,et al.  Computational dynamics for robotic systems on land and under water , 1995 .

[82]  E. Mjolsness,et al.  Genetic Algorithms for Genetic Neural Nets. , 1991 .

[83]  S. Grillner Neurobiological bases of rhythmic motor acts in vertebrates. , 1985, Science.

[84]  Gavan Lintern,et al.  Dynamic patterns: The self-organization of brain and behavior , 1997, Complex.

[85]  D. Parisi,et al.  Phenotypic plasticity in evolving neural networks , 1994, Proceedings of PerAc '94. From Perception to Action.

[86]  Hugo de Garis,et al.  Genetic Programming: Artificial Nervous Systems, Artificial Embryos and Embryological Electronics , 1990, PPSN.