Machine Learning With AIBO Robots in the Four-Legged League of RoboCup

Robot learning is a growing area of research at the intersection of robotics and machine learning. The main contributions of this paper include a review of how machine learning has been used on Sony AIBO robots and at RoboCup, with a focus on the four-legged league during the years 1998-2004. The review shows that the application-oriented use of machine learning in the four-legged league was still conservative and restricted to a few well-known and easy-to-use methods such as standard decision trees, evolutionary hill climbing, and support vector machines. Method-oriented spin-off studies emerged more frequently and increasingly addressed new and advanced machine learning techniques. Further, the paper presents some details about the growing impact of machine learning in the software system developed by the authors' robot soccer team-the NUbots

[1]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[2]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[3]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[4]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[5]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[6]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[7]  Sridhar Mahadevan,et al.  Robot Learning , 1993 .

[8]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[9]  M. Dorigo.  Introduction to the Special Issue on Learning Autonomous Robots , 1996 .

[10]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[11]  Dave Cliff,et al.  Challenges in evolving controllers for physical robots , 1996, Robotics Auton. Syst..

[12]  Didier Dubois,et al.  Representing partial ignorance , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[13]  Zbigniew Michalewicz,et al.  Handbook of Evolutionary Computation , 1997 .

[14]  Masahiro Fujita,et al.  An open architecture for robot entertainment , 1997, AGENTS '97.

[15]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[16]  Jeffrey Horn,et al.  Handbook of evolutionary computation , 1997 .

[17]  Hiroaki Kitano,et al.  RoboCup: A Challenge Problem for AI , 1997, AI Mag..

[18]  Hiroaki Kitano,et al.  RoboCup: A Challenge Problem for AI and Robotics , 1997, RoboCup.

[19]  北野 宏明,et al.  RoboCup-97 : robot soccer World Cup I , 1998 .

[20]  Manuela M. Veloso,et al.  The CMTrio-98 Sony-Legged Robot Team , 1998, RoboCup.

[21]  Hiroaki Kitano,et al.  Sony legged robot for RoboCup challenge , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[22]  Hiroaki Kitano,et al.  Playing soccer with legged robots , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[23]  Hiroaki Kitano,et al.  Development of an Autonomous Quadruped Robot for Robot Entertainment , 1998, Auton. Robots.

[24]  Minoru Asada,et al.  BabyTigers-98: Osaka Legged Robot Team , 1998, RoboCup.

[25]  Gregory S. Hornby,et al.  Autonomous evolution of gaits with the Sony Quadruped Robot , 1999 .

[26]  Mike Lawther,et al.  UNSW United , 1999, RoboCup.

[27]  Mike Lawther,et al.  Playing Soccer With Quadruped Robots , 1999 .

[28]  Minoru Asada,et al.  BabyTigers-99: Osaka Legged Robot Team , 1999, RoboCup.

[29]  Francis K. H. Quek.  An algorithm for the rapid computation of boundaries of run-length encoded regions , 2000, Pattern Recognit..

[30]  Manuela M. Veloso,et al.  Multiagent Systems: A Survey from a Machine Learning Perspective , 2000, Auton. Robots.

[31]  Minoru Asada,et al.  Observation strategy for decision making based on information criterion , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[32]  Dongbing Gu,et al.  Landmark‐based navigation of industrial mobile robots , 2000 .

[33]  Dongbing Gu,et al.  A Multi-Agent System for Cooperative Quadruped Walking Robots , 2000 .

[34]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[35]  Masahiro Fujita,et al.  Evolving robust gaits with AIBO , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[36]  Andreas Birk,et al.  Interdisciplinary approaches to robot learning , 2000 .

[37]  Yuichi Kobayashi.  Team ARAIBO , 2000, RoboCup.

[38]  Guillaume Marceau.  The McGill's RedDogs Legged League System , 2000, RoboCup.

[39]  Hiroaki Kitano,et al.  RoboCup-99: Robot Soccer World Cup III , 2003, Lecture Notes in Computer Science.

[40]  Manuela M. Veloso,et al.  Fast and inexpensive color image segmentation for interactive robots , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[41]  Claude Sammut,et al.  The UNSW RoboCup 2000 Sony Legged League Team , 2000, RoboCup.

[42]  Hiroaki Kitano,et al.  RoboCup-98: Robot Soccer World Cup II , 2001, Lecture Notes in Computer Science.

[43]  Peter Stone,et al.  RoboCup 2000: Robot Soccer World Cup IV , 2001, RoboCup.

[44]  Minoru Asada,et al.  BabyTigers 2001: Osaka Legged Robot Team , 2001, RoboCup.

[45]  Dongbing Gu,et al.  Reactive behaviours and agent architecture for Sony legged robots to play football , 2001 .

[46]  Minoru Asada,et al.  Visual Attention Control by Sensor Space Segmentation for a Small Quadruped Robot Based on Information Criterion , 2001, RoboCup.

[47]  Dieter Fox,et al.  Team Description: UW Huskies-01 , 2001, RoboCup.

[48]  Dongbing Gu,et al.  Evolving Fuzzy Logic Controllers for Sony Legged Robots , 2001, RoboCup.

[49]  Claude Sammut,et al.  Stochastic Gradient Descent Localisation in Quadruped Robots , 2001, RoboCup.

[50]  Claude Sammut,et al.  The UNSW RoboCup 2001 Sony Legged Robot League Team , 2001, RoboCup.

[51]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[52]  Claude Sammut,et al.  Omnidirectional Locomotion for Quadruped Robots , 2001, RoboCup.

[53]  Okyay Kaynak,et al.  Cerberus 2001 Team Description , 2001, RoboCup.

[54]  Daniel D. Lee,et al.  Biologically inspired computation and learning in Sensorimotor Systems , 2001, SPIE Optics + Photonics.

[55]  Dongbing Gu,et al.  Essex Rovers 2001 Team Description , 2001, RoboCup.

[56]  Masahiro Fujita,et al.  AIBO: Toward the Era of Digital Creatures , 2001, Int. J. Robotics Res..

[57]  Sridhar Mahadevan,et al.  Machine Learning for Robots A Comparison of Di erent Paradigms , 2002 .

[58]  Dieter Fox,et al.  An experimental comparison of localization methods continued , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[59]  Bernhard Nebel,et al.  Towards a Life-Long Learning Soccer Agent , 2002, RoboCup.

[60]  Peter Stone.  Multiagent Competitions and Research: Lessons from RoboCup and TAC , 2002, RoboCup.

[61]  Bernhard Nebel,et al.  CS Freiburg: coordinating robots for successful soccer playing , 2002, IEEE Trans. Robotics Autom..

[62]  Tamio Arai,et al.  Real-Time Decision Making under Uncertainty of Self-localization Results , 2002, RoboCup.

[63]  Jens Ziegler,et al.  Adaptive Methods to Improve Self-localization in Robot Soccer , 2002, RoboCup.

[64]  Pierre-Yves Oudeyer,et al.  Robotic clicker training , 2002, Robotics Auton. Syst..

[65]  Oskar von Stryk,et al.  The Role of Motion Dynamics in the Design, Control and Stability of Bipedal and Quadrupedal Robots , 2002, RoboCup.

[66]  Andreas Birk,et al.  RoboCup 2001: Robot Soccer World Cup V , 2002, Lecture Notes in Computer Science.

[67]  Raúl Rojas,et al.  RoboCup 2002: Robot Soccer World Cup VI , 2002, Lecture Notes in Computer Science.

[68]  Dongbing Gu,et al.  Reinforcement learning of fuzzy logic controllers for quadruped walking robots , 2002 .

[69]  Dustin Boswell,et al.  Introduction to Support Vector Machines , 2002 .

[70]  Huosheng Hu,et al.  A hybrid evolutionary algorithm for gait generation of Sony legged robots , 2002, IEEE 2002 28th Annual Conference of the Industrial Electronics Society. IECON 02.

[71]  Pierre-Yves Oudeyer,et al.  Motivational principles for visual know-how development , 2003 .

[72]  S. Bayhan,et al.  CERBERUS 2003 TEAM REPORT , 2003 .

[73]  S. Shankar Sastry,et al.  Autonomous Helicopter Flight via Reinforcement Learning , 2003, NIPS.

[74]  Javier Ruiz-del-Solar,et al.  Evolving Visual Object Recognition for Legged Robots , 2003, RoboCup.

[75]  Brett Browning,et al.  Plays as Team Plans for Coordination and Adaptation , 2003, RoboCup.

[76]  Gordon Wyeth,et al.  A Helicopter named Dolly : behavioural cloning for autonomous helicopter control , 2003 .

[77]  William T. B. Uther,et al.  Automatic Gait Optimisation for Quadruped Robots , 2003 .

[78]  Martin A. Riedmiller,et al.  Using Machine Learning Techniques in Complex Multi-Agent Domains , 2003 .

[79]  Stephan K. Chalup,et al.  Traction Monitoring for Collision Detection with Legged Robots , 2003, RoboCup.

[80]  Stephan K. Chalup,et al.  Techniques for Improving Vision and Locomotion on the Sony AIBO Robot , 2003 .

[81]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[82]  Manuela M. Veloso,et al.  Simultaneous Adversarial Multi-Robot Learning , 2003, IJCAI.

[83]  Claude Sammut,et al.  Goal-directed Learning to Fly , 2003, ICML.

[84]  Dongbing Gu,et al.  GA-based learning in behaviour based robotics , 2003, Proceedings 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation. Computational Intelligence in Robotics and Automation for the New Millennium (Cat. No.03EX694).

[85]  Stephan K. Chalup,et al.  Application of SVMs for Colour Classification and Collision Detection with AIBO Robots , 2003, NIPS.

[86]  Claude Sammut.  Robot Soccer: Science or Just Fun and Games? , 2003, Australian Conference on Artificial Intelligence.

[87]  Huosheng Hu,et al.  An Adaptive Color Segmentation Algorithm for Sony Legged Robots , 2003, Applied Informatics.

[88]  Brett Browning,et al.  RoboCup 2003: Robot Soccer World Cup VII , 2003, Lecture Notes in Computer Science.

[89]  W. Press,et al.  Numerical Recipes in C++: The Art of Scientific Computing (2nd edn)1 Numerical Recipes Example Book (C++) (2nd edn)2 Numerical Recipes Multi-Language Code CD ROM with LINUX or UNIX Single-Screen License Revised Version3 , 2003 .

[90]  Patrick Doherty,et al.  Issues in Designing Physical Agents for Dynamic Real-Time Environments: World Modeling, Planning, Learning, and Communicating , 2004, AI Mag..

[91]  Thomas Röfer,et al.  Evolutionary Gait-Optimization Using a Fitness Function Based on Proprioception , 2004, RoboCup.

[92]  Javier Ruiz-del-Solar,et al.  UCHILSIM: A Dynamically and Visually Realistic Simulator for the RoboCup Four Legged League , 2004, RoboCup.

[93]  Richard H. Middleton,et al.  An Application of the Extended Kalman Filter to Robot Soccer Localisation and World Modelling , 2004 .

[94]  Peggy Fidelman,et al.  Learning Ball Acquisition on a Physical Robot , 2004 .

[95]  Ayumi Shinohara,et al.  Jolly Pochie 2004 in the Four Legged Robot League , 2004 .

[96]  Dieter Fox,et al.  Reinforcement learning for sensing strategies , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[97]  Manuela Veloso,et al.  Learning from accelerometer data on a legged robot , 2004 .

[98]  Peter Stone,et al.  Policy gradient reinforcement learning for fast quadrupedal locomotion , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[99]  Javier Ruiz-del-Solar,et al.  Learning to Kick the Ball Using Back to Reality , 2004, RoboCup.

[100]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[101]  Pierre-Yves Oudeyer,et al.  Intelligent Adaptive Curiosity: a source of Self-Development , 2004 .

[102]  Peter Stone,et al.  Towards Illumination Invariance in the Legged League , 2005, RoboCup.

[103]  Javier Ruiz-del-Solar,et al.  Back to reality: Crossing the reality gap in evolutionary robotics , 2004 .

[104]  Dieter Fox,et al.  Map-Based Multiple Model Tracking of a Moving Object , 2004, RoboCup.

[105]  Stephan K. Chalup,et al.  Machine Learning in the Four-Legged League , 2004 .

[106]  Javier Ruiz-del-Solar,et al.  On-line Learning of an Object Manipulation Behavior for Legged Robots , 2004 .

[107]  Masahiro Fujita,et al.  On activating human communications with pet-type robot AIBO , 2004, Proceedings of the IEEE.

[108]  Manuela M. Veloso,et al.  An evolutionary approach to gait learning for four-legged robots , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[109]  Dieter Fox,et al.  Bayesian color estimation for adaptive vision-based robot localization , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[110]  Peter Stone,et al.  Machine Learning for Fast Quadrupedal Locomotion , 2004, AAAI.

[111]  Thomas Röfer,et al.  Realtime Object Recognition Using Decision Tree Learning , 2004, RoboCup.

[112]  Pierre-Yves Oudeyer,et al.  The Playground Experiment: Task-Independent Development of a Curious Robot , 2005 .

[113]  Peter Stone,et al.  Reinforcement Learning for RoboCup Soccer Keepaway , 2005, Adapt. Behav..

[114]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[115]  Frédéric Kaplan,et al.  Learning to Interpret Pointing Gestures: Experiments with Four-Legged Autonomous Robots , 2005, Biomimetic Neural Learning for Intelligent Robots.

[116]  T. Poggio,et al.  The Mathematics of Learning: Dealing with Data , 2005, 2005 International Conference on Neural Networks and Brain.

[117]  Peter Stone,et al.  Simultaneous Calibration of Action and Sensor Models on a Mobile Robot , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[118]  Martin A. Riedmiller,et al.  RoboCup 2004: Robot Soccer World Cup VIII , 2005, RoboCup.

[119]  Nicholas K. Jong,et al.  The UT Austin Villa 2004 RoboCup Four-Legged Team: Coming of Age , .