Robot-learning - Three case studies in robotics and machine learning

This paper describes methodologies applied and results achieved in the framework of the ESPRIT Basic Research Action B-Learn II (project no. 7274). B-Learn II is one of the rst projects working towards an application of Machine Learning techniques in elds of industrial relevance, which are much more complex than the domains usually treated in ML research. In particular, B-Learn II aims at easing the programming of robots and enhancing their ability to cooperate with humans. The paper gives a short introduction to learning in robotics and to the three applications under consideration in B-Learn II. Afterwards, learning methodologies used in each of the applications, the experimental setups, and the results obtained are described. In general, it can be found that providing good examples and a good interface between the learning and the performance components is crucial for success, so the extension of the "Programming by Demonstration" paradigm to robotics has become one of the key aspects of B-Learn II.

[1]  M. Nuttin,et al.  Fuzzy controller synthesis in robotic assembly: procedure and experiments , 1994, Proceedings of 1994 IEEE 3rd International Fuzzy Systems Conference.

[2]  Charles W. Anderson,et al.  Learning and problem-solving with multilayer connectionist systems (adaptive, strategy learning, neural networks, reinforcement learning) , 1986 .

[3]  Kostas J. Kyriakopoulos,et al.  Sensor-based self-localization for wheeled mobile robots , 1995, J. Field Robotics.

[4]  L. Basañez,et al.  Assembly contact force domains in the presence of uncertainty , 1994 .

[5]  J. Shavlik,et al.  Re nement of Approximate Domain Theories byKnowledge-Based Neural Networks , 1990 .

[6]  L. Basañez,et al.  Uncertainty Modelling in Configuration Space for Robotic Motion Planning , 1991 .

[7]  J. Hopfield,et al.  The Logic of Limax Learning , 1985 .

[8]  Madan M. Gupta,et al.  On the principles of fuzzy neural networks , 1994 .

[9]  Translator-IEEE Expert staff.  Machine Learning: A Theoretical Approach , 1992, IEEE Expert.

[10]  Sheng Liu,et al.  Teaching and learning of deburring robots using neural networks , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[11]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[12]  Long-Ji Lin,et al.  Reinforcement learning for robots using neural networks , 1992 .

[13]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[14]  Matthew T. Mason,et al.  Compliance and Force Control for Computer Controlled Manipulators , 1981, IEEE Transactions on Systems, Man, and Cybernetics.

[15]  T. Suehiro,et al.  A model-based manipulation system with skill-based execution in unstructured environment , 1991 .

[16]  Cristina Baroglio,et al.  Learning Simple Recursive Theories , 1993, ISMIS.

[17]  Pat Langley,et al.  Models of Incremental Concept Formation , 1990, Artif. Intell..

[18]  A. Klopf.  A neuronal model of classical conditioning , 1988 .

[19]  Joris De Schutter,et al.  Model based and sensor based programming of compliant motion tasks , 1993 .

[20]  Carme Torras.  Neural Learning for Robot Control , 1994, ECAI.

[21]  James S. Albus,et al.  Theory and Practice of Hierarchical Control , 1981 .

[22]  Roderic A. Grupen,et al.  Learning reactive admittance control , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[23]  James L. Crowley,et al.  Dynamic Modeling of Free-Space for a Mobile Robot , 1989, Proceedings. IEEE/RSJ International Workshop on Intelligent Robots and Systems '. (IROS '89) 'The Autonomous Mobile Robots and Its Applications.

[24]  Yoichi Hayashi,et al.  A Neural Expert System with Automated Extraction of Fuzzy If-Then Rules , 1990, NIPS.

[25]  Klaus Landzettel,et al.  Multisensory shared autonomy and tele-sensor-programming-Key issues in the space robot technology experiment ROTEX , 1993, Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '93).

[26]  Jurgen Kreuziger,et al.  Application Of Machine Learning To Robotics - An Analysis , 1992 .

[27]  Joris De Schutter,et al.  Rosi : A task specification and simulation tool for force sensor based robot control , 1993 .

[28]  Ingemar J. Cox,et al.  Autonomous Robot Vehicles , 1990, Springer New York.

[29]  Jean-Claude Latombe,et al.  An Approach to Automatic Robot Programming Based on Inductive Learning , 1984 .

[30]  Luis M. Camarinha-Matos,et al.  Execution monitoring in assembly with learning capabilities , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[31]  Jorg-uwe Kietz,et al.  Controlling the Complexity of Learning in Logic through Syntactic and Task-Oriented Models , 1992 .

[32]  李幼升,et al.  Ph , 1989 .

[33]  Utpal Roy,et al.  Feature-based representational scheme of a solid modeler for providing dimensioning and tolerancing information , 1988 .

[34]  Narendra Ahuja,et al.  Gross motion planning—a survey , 1992, CSUR.

[35]  John J. Leonard,et al.  Directed Sonar Sensing for Mobile Robot Navigation , 1992 .

[36]  Henry Lieberman,et al.  Watch what I do: programming by demonstration , 1993 .

[37]  Tom Tollenaere,et al.  SuperSAB: Fast adaptive back propagation with good scaling properties , 1990, Neural Networks.

[38]  Peter Weckesser,et al.  Calibration of the active stereo vision system KASTOR with standardized perspective matrices , 1994, Other Conferences.

[39]  Rüdiger Dillmann,et al.  Integration of topological and geometrical planning in a learning mobile robot , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).

[40]  Long Ji Lin,et al.  Programming Robots Using Reinforcement Learning and Teaching , 1991, AAAI.

[41]  Marnix Nuttin,et al.  On the reduction of costs for robot controller synthesis , 1994 .

[42]  Jude W. Shavlik,et al.  Using Symbolic Learning to Improve Knowledge-Based Neural Networks , 1992, AAAI.

[43]  Nathan Delson,et al.  Robot programming by human demonstration: Subtask compliance controller identification , 1993, Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '93).

[44]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[45]  M. Kaiser,et al.  Time-delay neural networks for control , 1994 .

[46]  Richard P. Paul,et al.  A robot compliant wrist system for automated assembly , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[47]  Sebastian Thrun,et al.  Explanation-Based Neural Network Learning for Robot Control , 1992, NIPS.

[48]  Stefan Wrobel,et al.  Towards a Model of Grounded Concept Formation , 1991, IJCAI.

[49]  Dr.-Ing. U. Rembold.  Integration of Symbolic and Connectionist Learning to ease Robot Programming and Control , 1994 .

[50]  Lashon B. Booker,et al.  Intelligent Behavior as an Adaptation to the Task Environment , 1982 .

[51]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[52]  Paul H. Andersson,et al.  A concept for maintaining quality in flexible production , 1991 .

[53]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[54]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[55]  Satinder P. Singh,et al.  Reinforcement Learning with a Hierarchy of Abstract Models , 1992, AAAI.

[56]  László Monostori,et al.  Neural networks—Their applications and perspectives in intelligent machining , 1991 .

[57]  Stephen C.-Y. Lu,et al.  Machine learning in engineering automation—the present and the future , 1991 .

[58]  Jenq-Neng Hwang,et al.  Neural network architectures for robotic applications , 1989, IEEE Trans. Robotics Autom..

[59]  Hendrik Van Brussel,et al.  Compliant Robot Motion I. A Formalism for Specifying Compliant Motion Tasks , 1988, Int. J. Robotics Res..

[60]  Sridhar Mahadevan,et al.  Automatic Programming of Behavior-Based Robots Using Reinforcement Learning , 1991, Artif. Intell..

[61]  Richard S. Sutton,et al.  Integrated Modeling and Control Based on Reinforcement Learning and Dynamic Programming , 1990, NIPS 1990.

[62]  D. Guinea,et al.  Multi-sensor integration—An automatic feature selection and state identification methodology for tool wear estimation , 1991 .

[63]  José del Rocio Millán Ruiz.  A reinforcement connectionist learning approach to robot path finding , 1993 .

[64]  Stephen J. Buckley,et al.  Teaching compliant motion strategies , 1989, IEEE Trans. Robotics Autom..

[65]  Lorenza Saitta,et al.  Machine learning - an integrated framework and its applications , 1991, Ellis Horwood series in artificial intelligence.

[66]  Leslie Pack Kaelbling,et al.  Learning in embedded systems , 1993 .

[67]  Alberta Maria Segre,et al.  Machine Learning of Robot Assembly Plans , 1988 .

[68]  Ryszard S. Michalski,et al.  Machine learning: an artificial intelligence approach volume III , 1990 .

[69]  Francesco G. B. De Natale,et al.  Adaptive Control in Visual Sensing , 1994 .

[70]  Hyongsuk Kim,et al.  USE OF CMAC NEURAL NETWORKS IN REINFORCEMENT SELF-LEARNING CONTROL , 1991 .

[71]  R. Sutton,et al.  Simulation of the classically conditioned nictitating membrane response by a neuron-like adaptive element: Response topography, neuronal firing, and interstimulus intervals , 1986, Behavioural Brain Research.

[72]  Long Ji Lin,et al.  Scaling Up Reinforcement Learning for Robot Control , 1993, International Conference on Machine Learning.

[73]  Rüdiger Dillmann,et al.  PRIAMOS: An experimental platform for reflexive navigation , 1993, Robotics Auton. Syst..

[74]  Roy Rada,et al.  Machine learning - applications in expert systems and information retrieval , 1986, Ellis Horwood series in artificial intelligence.

[75]  H. Harry Asada,et al.  Skill acquisition from human experts through pattern processing of teaching data , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[76]  Hendrik Van Brussel,et al.  The adaptable compliance concept and its use for automatic assembly by active force feedback accommodations , 1979 .

[77]  M. Kaiser.  A Framework for the Generation of Robot Controllers from Examples , 1994 .

[78]  J. De Schutter,et al.  Compliant Robot Motion II. A Control Approach Based on External Control Loops , 1988 .

[79]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[80]  Rosanna Heise.  Demonstration instead of programming: focussing attention in robot task acquisition , 1989 .

[81]  Karsten Berns,et al.  Reinforcement-learning For The Control Of An Autonomous Mobile Robot , 1992, Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems.

[82]  C. Anderson.  Strategy Learning with Multilayer Connectionist Representations 1 , 1987 .

[83]  Marnix Nuttin,et al.  Integrated Acquisition, Execution, Evaluation, and Tuning of Elementary Skills for Intelligent robots , 1994, AIRTC.

[84]  Sridhar Mahadevan,et al.  Enhancing Transfer in Reinforcement Learning by Building Stochastic Models of Robot Actions , 1992, ML.

[85]  Satinder Singh,et al.  Learning to Solve Markovian Decision Processes , 1993 .

[86]  Tien C. Hsia,et al.  Adaptive control of robot manipulators - A review , 1986, Proceedings. 1986 IEEE International Conference on Robotics and Automation.

[87]  Eduardo D. Sontag,et al.  Some Topics in Neural Networks and Control , 1993 .

[88]  Jeff G. Schneider,et al.  Robot skill learning, basis functions, and control regimes , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[89]  John H. Holland,et al.  Induction: Processes of Inference, Learning, and Discovery , 1987, IEEE Expert.

[90]  Yangsheng Xu,et al.  Hidden Markov model approach to skill learning and its application to telerobotics , 1993, IEEE Trans. Robotics Autom..

[91]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[92]  Ian H. Witten,et al.  An Adaptive Optimal Controller for Discrete-Time Markov Environments , 1977, Inf. Control..

[93]  A. Steiger-Garcao,et al.  Sensor Integration For Expert CNC Machines Supervision , 1992, IEEE International Workshop on Emerging Technologies and Factory Automation,.

[94]  Carme Torras i Genís,et al.  Symbolic Planning versus Neural Control in Robots , 1993 .

[95]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[96]  Jing Peng,et al.  Efficient Learning and Planning Within the Dyna Framework , 1993, Adapt. Behav..

[97]  Larry H. Matthies,et al.  Integration of sonar and stereo range data using a grid-based representation , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[98]  Martin Anthony,et al.  Computational learning theory: an introduction , 1992 .

[99]  Randy C. Brost,et al.  Graphical analysis of planar rigid-body dynamics with multiple frictional contacts , 1991 .

[100]  José del R. Millán,et al.  Efficient reinforcement learning of navigation strategies in an autonomous robot , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).

[101]  James S. Albus,et al.  I A New Approach to Manipulator Control: The I Cerebellar Model Articulation Controller , 1975 .

[102]  Rodolfo Zunino,et al.  On the need for integrated approaches to image understanding , 1992, Eur. Trans. Telecommun..

[103]  Matthew R. Stein,et al.  Behavior-based control for time-delayed teleoperation , 1994 .

[104]  Hamid R. Berenji,et al.  A reinforcement learning--based architecture for fuzzy logic control , 1992, Int. J. Approx. Reason..

[105]  H. Harry Asada,et al.  Teaching and learning of compliance using neural nets: representation and generation of nonlinear compliance , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[106]  Luis M. Camarinha-Matos,et al.  Learning in Assembly Task Execution , 1993 .

[107]  Charles W. Anderson,et al.  Strategy Learning with Multilayer Connectionist Representations , 1987 .

[108]  Jude Shavlik,et al.  Refinement ofApproximate Domain Theories by Knowledge-Based Neural Networks , 1990, AAAI.

[109]  Marco Botta,et al.  SMART+: A Multi-Strategy Learning Tool , 1993, IJCAI.

[110]  J. Peng,et al.  Efficient Learning and Planning Within the Dyna Framework , 1993, IEEE International Conference on Neural Networks.

[111]  Vijaykumar Gullapalli,et al.  Reinforcement learning and its application to control , 1992 .