Keyframe Sampling, Optimization, and Behavior Integration: Towards Long-Distance Kicking in the RoboCup 3D Simulation League

Even with improvements in machine learning enabling robots to quickly optimize and perfect their skills, developing a seed skill from which to begin an optimization remains a necessary challenge for large action spaces. This paper proposes a method for creating and using such a seed by (i) observing the effects of the actions of another robot, (ii) further optimizing the skill starting from this seed, and (iii) embedding the optimized skill in a full behavior. Called KSOBI, this method is fully implemented and tested in the complex RoboCup 3D simulation domain. To the best of our knowledge, the resulting skill kicks the ball farther in this simulator than has been previously documented.

[1]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[2]  Peter Stone,et al.  MARIOnET: motion acquisition for robots through iterative online evaluative training , 2010, AAMAS.

[3]  Jesus Savage,et al.  RoboCup 2011: Robot Soccer World Cup XV , 2012, Lecture Notes in Computer Science.

[4]  Patrick MacAlpine,et al.  UT Austin Villa: RoboCup 2012 3D Simulation League Champion , 2012, RoboCup.

[5]  Peter Stone,et al.  Transfer Learning for Reinforcement Learning Domains: A Survey , 2009, J. Mach. Learn. Res..

[6]  Daniel Urieli,et al.  UT Austin Villa 2011: a champion agent in the RoboCup 3D soccer simulation competition , 2012, AAMAS.

[7]  Mike Depinet.  Keyframe Sampling, Optimization, and Behavior Integration: A New Longest Kick in the RoboCup 3D Simulation League , 2014 .

[8]  Ubbo Visser,et al.  Motion Capture and Contemporary Optimization Algorithms for Robust and Stable Motions on Simulated Biped Robots , 2012, RoboCup.

[9]  Luís Paulo Reis,et al.  Development of an Omnidirectional Kick for a NAO Humanoid Robot , 2012, IBERAMIA.

[10]  Luis Cruz,et al.  Optimization Approach for the Development of Humanoid Robots' Behaviors , 2012, IBERAMIA.

[11]  Xiaoping Chen,et al.  RoboCup 2012: Robot Soccer World Cup XVI , 2013, Lecture Notes in Computer Science.

[12]  Daniel Urieli,et al.  WrightEagle and UT Austin Villa: RoboCup 2011 Simulation League Champions , 2012, RoboCup.

[13]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Tutorial , 2016, ArXiv.

[14]  Daniel Urieli,et al.  Design and Optimization of an Omnidirectional Humanoid Walk: A Winning Approach at the RoboCup 2011 3D Simulation Competition , 2012, AAAI.

[15]  Maya Cakmak,et al.  Keyframe-based Learning from Demonstration , 2012, Int. J. Soc. Robotics.