Discovery of High-Level Behavior From Observation of Human Performance in a Strategic Game

This paper explores the issues faced in creating a system that can learn tactical human behavior merely by observing a human perform the behavior in a simulation. More specifically, this paper describes a technique based on fuzzy ARTMAP (FAM) neural networks to discover the criteria that cause a transition between contexts during a strategic game simulation. The approach depends on existing context templates that can identify the high-level action of the human, given a description of the situation along with his action. The learning task then becomes the identification and representation of the context sequence executed by the human. In this paper, we present the FAM/template-based interpretation learning engine (FAMTILE). This system seeks to achieve this learning task by constructing rules that govern the context transitions made by the human. To evaluate FAMTILE, six test scenarios were developed to achieve three distinct evaluation goals: 1) to assess the learning capabilities of FAM; 2) to evaluate the ability of FAMTILE to correctly predict human and context selections, given an observation; and 3) more fundamentally, to create a model of the human's behavior that can perform the high-level task at a comparable level of proficiency.

[1]  Avelino J. Gonzalez,et al.  A Framework for Learning Implicit Expert Knowledge through Observation , 2000 .

[2]  Rui Camacho,et al.  Inducing Models of human Control Skills , 1998, ECML.

[3]  Jonathan Schaeffer,et al.  Poker as Testbed for AI Research , 1998, Canadian Conference on AI.

[4]  Allen Newell,et al.  Chess-Playing Programs and the Problem of Complexity , 1958, IBM J. Res. Dev..

[5]  Merriam-Webster.  Webster's New World Dictionary of the American Language , 1976 .

[6]  John E. Laird,et al.  Learning goal hierarchies from structured observations and expert annotations , 2006, Machine Learning.

[7]  N. Charness,et al.  Component processes in bridge bidding and novel problem-solving tasks. , 1987, Canadian journal of psychology.

[8]  John R. Anderson,et al.  MACHINE LEARNING An Artificial Intelligence Approach , 2009 .

[9]  T.A. Sidani,et al.  Learning situational knowledge through observation of expert performance in a simulation-based environment , 1994, Conference Record Southcon.

[10]  Christopher G. Atkeson,et al.  Learning from observation using primitives , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[11]  Stefan Schaal,et al.  Is imitation learning the route to humanoid robots? , 1999, Trends in Cognitive Sciences.

[12]  Monty Newborn.  The 20th annual ACM North American computer chess championship , 1990, CACM.

[13]  J. D. Morrison,et al.  Realtime Learning of Doctrine and Tactics Using Neural Networks and Combat Simulations , 1996 .

[14]  Avelino J. Gonzalez,et al.  Neural network based movement models to improve the predictive utility of entity state synchronization methods for distributed simulations , 2001 .

[15]  A. Pentland,et al.  Toward augmented control systems , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[16]  Francesco Mondada,et al.  Automatic creation of an autonomous agent: genetic evolution of a neural-network driven robot , 1994 .

[17]  Geir Hovland,et al.  Skill acquisition from human demonstration using a hidden Markov model , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[18]  Alexandros Moukas,et al.  Synthetic Robotic Language Acquisition by Observation , 1996 .

[19]  Paul S. Rosenbloom,et al.  Universal Subgoaling and Chunking , 1986 .

[20]  C G Gingrich,et al.  Modeling human operators using neural networks. , 1992, ISA transactions.

[21]  T. Fukuda,et al.  Acquisition of human skills for robotic systems , 1991, Proceedings of the 1991 IEEE International Symposium on Intelligent Control.

[22]  Sukhan Lee,et al.  Skill learning from observations , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[23]  John E. Laird,et al.  Learning Goal Hierarchies from Structured Observations and Expert Annotations , 2004, ILP.

[24]  Avelino J. Gonzalez,et al.  Real-time synchronization of behavioral models with human performance in a simulation , 2001 .

[25]  Rüdiger Dillmann,et al.  Building elementary robot skills from human demonstration , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[26]  Henry A. Kautz,et al.  Inferring High-Level Behavior from Low-Level Sensors , 2003, UbiComp.

[27]  Donald A. Waterman,et al.  Generalization Learning Techniques for Automating the Learning of Heuristics , 1970, Artif. Intell..

[28]  Neil Charness,et al.  Expertise and age effects on knowledge activation in chess. , 2006, Psychology and aging.

[29]  Neil Charness,et al.  Components of skill in bridge. , 1979 .

[30]  Avelino J. Gonzalez,et al.  Automating the Acquisition of Tactical Knowledge for Military Missions , 2005 .

[31]  Ronald A. Hess,et al.  A Model of Driver Steering Control Behavior for Use in Assessing Vehicle Handling Qualities , 1993 .

[32]  Claude E. Shannon,et al.  Programming a computer for playing chess , 1950 .

[33]  Peter Drewes.  Automated student performance monitoring in training simulation , 1997 .

[34]  Dean A. Pomerleau,et al.  AVCS RESEARCH AT CARNEGIE MELLON UNIVERSITY , 1994 .

[35]  David Sklansky,et al.  Hold'Em Poker for Advanced Players , 1999 .

[36]  Feng-Hsiung Hsu,et al.  Behind Deep Blue: Building the Computer that Defeated the World Chess Champion , 2002 .

[37]  Michael Georgiopoulos,et al.  Applications of Neural Networks in Electromagnetics , 2001 .

[38]  Claude E. Shannon,et al.  XXII. Programming a Computer for Playing Chess 1 , 1950 .

[39]  Rüdiger Dillmann,et al.  What Can Robots Learn from Humans , 1996 .

[40]  Randolph M. Jones,et al.  Constraints on the Design of a High-Level Model of Cognition , 1999 .

[41]  Bruce Wilcox,et al.  Reflections on building two Go programs , 1985, SGAR.

[42]  Stephen Grossberg,et al.  Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps , 1992, IEEE Trans. Neural Networks.

[43]  H. Harry Asada,et al.  Hybrid linguistic/numeric control of deburring robots based on human skills , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[44]  Avelino J. Gonzalez,et al.  Context-based representation of intelligent behavior in training simulations , 1998 .

[45]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[46]  John E. Laird,et al.  Learning Hierarchical Performance Knowledge by Observation , 1999, ICML.

[47]  Albert Lindsey Zobrist,et al.  Feature extraction and representation for pattern recognition and the game of go , 1970 .

[48]  N. Charness.  The impact of chess research on cognitive science , 1992 .

[49]  David Sklansky,et al.  The Theory of Poker , 1999 .

[50]  John E. Laird,et al.  Learning procedural knowledge through observation , 2001, K-CAP '01.

[51]  Anders Kierulf,et al.  Smart game board and Go explorer: a case study in software and knowledge engineering , 1989 .

[52]  Ken Chen,et al.  Smart game board and go explorer: a study in software and knowledge engineering , 1990, Commun. ACM.

[53]  Sheng Ma,et al.  Toward Applying Machine Learning to Design Rule Acquisition for Automated Graphics Generation , 1999 .

[54]  T. Damarla,et al.  Methodology for human decision making using fuzzy ARTMAP neural networks , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[55]  Jonathan Leonard Ryder,et al.  Heuristic analysis of large trees as generated in the game of Go , 1971 .

[56]  Brian S. Stensrud,et al.  Famtile: an algorithm for learning high-level tactical behavior from observation , 2005 .

[57]  E. Fix.  Neural network based human performance modeling , 1990, IEEE Conference on Aerospace and Electronics.

[58]  Roni Khardon,et al.  Learning to Take Actions , 1996, Machine Learning.

[59]  Randolph M. Jones,et al.  Automated Intelligent Pilots for Combat Flight Simulation , 1998, AI Mag..

[60]  Yangsheng Xu,et al.  Learning and Transfer of Human Real-Time Control Strategies , 1997, J. Adv. Comput. Intell. Intell. Informatics.

[61]  S. King.  Learning to fly. , 1998, Nursing Times.

[62]  Claude Sammut,et al.  Goal-directed Learning to Fly , 2003, ICML.

[63]  Sukhan Lee,et al.  Robot skill discovery based on observed data(003) 5335512 , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[64]  Steven Walczak,et al.  A quantitative analysis of pattern production and its relationship to expert performance , 1997, J. Exp. Theor. Artif. Intell..

[65]  Avelino J. Gonzalez,et al.  Learning tactical human behavior through observation of human performance , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[66]  Yangsheng Xu,et al.  Cascade neural networks with node-decoupled extended Kalman filtering , 1997, Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA'97. 'Towards New Computational Principles for Robotics and Automation'.

[67]  Pattie Maes,et al.  Synthetic Robotic Language Acquisition by Observation , 1996 .