Representing and Learning Routine Activities

A routine is a habitually repeated performance of some actions. Agents use routines to guide their everyday activities and to enrich their abstract concepts about acts. This dissertation addresses the question of how an agent who is engaged in ordinary, routine activities changes its behavior over time, how the agent's internal representations about the world is a ected by its interactions, and what is a good agent architecture for learning routine interactions with the world. In it, I develop a theory that proposes several key processes: (1) automaticity, (2) habituation and skill re nement, (3) abstraction-bychunking, and (4) discovery of new knowledge chunks. The process of automaticity caches the agent's knowledge about actions into a at stimulus-response data structure that eliminates knowledge of action consequences. The stimulus-response data structure produces a response to environmental stimuli in constant time. The process of habituation and skill re nement uses environmental clues as rewards. Rewards are used to develop a bias for the agent's actions in competing action-situation pairs where the agent did not have prior choices. The process of abstraction-by-chunking monitors the agent's increased reliance on stimulusresponse data structures and turns the agent's complex actions into primitive acts, thereby eliminating the need for the agent to elaborate its plans beyond a certain level. The process of discovering knowledge-chunks monitors the agent's use of stimulus-action data structures and constructs knowledge about action preconditions and consequences and constructs patterns of interactions into plans. I have implemented several agents that demonstrate parts of my theory using an agent architecture I developed called GLAIR. GLAIR models agents that function in the world. Beyond my theory about routines, GLAIR is used to demonstrate situated actions as well as deliberative actions. Each of GLAIR's three levels (Sensori-actuator, Perceptuo-motor, and Knowledge) uses di erent representations and models di erent components of intelligent agency. GLAIR levels operate semi-autonomously. Whereas intra-level learning improves an agent's performance, inter-level learning migrates know-how from one level to another. Using the GLAIR architecture, I model agents that engage in routines, use these routines to guide their behavior, and learn more concepts having to do with acts. The signi cance of my theory, architectures, and the agents are to illustrate that computational agents can autonomously learn know-how from their own routine interactions in the world as well as learn to improve their performance. 5

[1]  Deepak Kumar.  From beliefs and goals to intentions and actions: an amalgamated model of inference and acting , 1993 .

[2]  Geoffrey E. Hinton,et al.  The appeal of parallel distributed processing , 1986 .

[3]  Roland Hausser,et al.  Computation of language - an essay on syntax, semantics and pragmatics in natural man-machine communication , 1989, Symbolic computation.

[4]  David W. Payton,et al.  An architecture for reflexive autonomous vehicle control , 1986, Proceedings. 1986 IEEE International Conference on Robotics and Automation.

[5]  Walter Schneider,et al.  Controlled and automatic human information processing: II. Perceptual learning, automatic attending and a general theory. , 1977 .

[6]  Pattie Maes,et al.  Situated agents can have goals , 1990, Robotics Auton. Syst..

[7]  J. Lammens,et al.  Behavior Based Ai, Cognitive Processes, and Emergent Behaviors in Autonomous Agents , 1993 .

[8]  Steven Douglas Whitehead,et al.  Reinforcement learning for the adaptive control of perception and action , 1992 .

[9]  Lucy A. Suchman,et al.  Plans and Situated Actions: The Problem of Human-Machine Communication (Learning in Doing: Social, , 1987 .

[10]  Stuart C. Shapiro,et al.  An Autonomous Agent Architecture for Integrating "Unconscious" and "Conscious", Reasoned Behaviors , 1993, 1993 Computer Architectures for Machine Perception.

[11]  Gerald DeJong,et al.  Explanation-Based Learning of Reactive Operations , 1989, ML.

[12]  Tom M. Mitchell,et al.  On Becoming Reactive , 1989, ML.

[13]  Leslie Pack Kaelbling,et al.  The Synthesis of Digital Machines With Provable Epistemic Properties , 1986, TARK.

[14]  Marcel Joachim Schoppers,et al.  Representation and automatic synthesis of reaction plans , 1989 .

[15]  John R. Anderson,et al.  Rules of the Mind , 1993 .

[16]  D. N. Teller,et al.  The Minds of Robots , 1965 .

[17]  Rodney A. Brooks.  PLANNING IS JUST A WAY OF AVOIDING FIGURING OUT WHAT TO DO NEXT , 1987 .

[19]  Drew McDermott,et al.  Robot Planning , 1991, AI Mag..

[20]  Richard Fikes,et al.  Learning and Executing Generalized Robot Plans , 1993, Artif. Intell..

[21]  Allen Newell,et al.  Chunking in Soar , 1986 .

[22]  T. Odlin.  Women, Fire, and Dangerous Things: What Categories Reveal about the Mind , 1988 .

[23]  Richard Fikes,et al.  STRIPS: A New Approach to the Application of Theorem Proving to Problem Solving , 1971, IJCAI.

[24]  Kristian J. Hammond,et al.  Case-based planning: an integrated theory of planning, learning and memory , 1986 .

[25]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .

[26]  D. Norman.  Categorization of action slips. , 1981 .

[27]  R. James Firby,et al.  An Investigation into Reactive Planning in Complex Domains , 1987, AAAI.

[28]  Stuart C. Shapiro,et al.  Cables, Paths, and "Subconscious" Reasoning in Propositional Semantic Networks , 1991, Principles of Semantic Networks.

[29]  David E. Wilkins,et al.  Domain-Independent Planning: Representation and Plan Generation , 1984, Artif. Intell..

[30]  B. Habibi,et al.  Pengi : An Implementation of A Theory of Activity , 1998 .

[31]  David Kortenkamp,et al.  Issues on building software for hardware agents , 1995, The Knowledge Engineering Review.

[32]  Tom M. Mitchell,et al.  Becoming Increasingly Reactive , 1990, AAAI.

[33]  Rodney A. Brooks,et al.  Learning to Coordinate Behaviors , 1990, AAAI.

[34]  Pat Langley,et al.  A design for the ICARUS architecture , 1991, SGAR.

[35]  David Kortenkamp,et al.  Characterizing an Architecture for Intelligent, Reactive Agents , 1995 .

[36]  Richard Alterman,et al.  An Adaptive Planner , 1986, AAAI.

[37]  Henry Hexmoor,et al.  Methods for deciding what to do next and learning , 1992 .

[38]  James S. Albus,et al.  Theory and Practice of Hierarchical Control , 1981 .

[39]  Jonathan H. Connell,et al.  SSS: a hybrid architecture applied to robot navigation , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[40]  Maja J. Matarić,et al.  Designing emergent behaviors: from local interactions to collective intelligence , 1993 .

[41]  Stuart C. Shapiro,et al.  SNePS Considered as a Fully Intensional Propositional Semantic Network , 1986, AAAI.

[42]  M. F.,et al.  Bibliography , 1985, Experimental Gerontology.

[43]  Erann Gat,et al.  Reliable goal-directed reactive control of autonomous mobile robots , 1991 .

[44]  M. Frese,et al.  Action as the core of work psychology: A German approach. , 1994 .

[45]  Gregory T. Stevens,et al.  Emergence, Self-Organization, and Social Interaction: Arousal-Dependent Structure in Social Systems* , 1996 .

[46]  G. Lakoff.  Women, fire, and dangerous things : what categories reveal about the mind , 1989 .

[47]  Leslie Pack Kaelbling,et al.  An Architecture for Intelligent Reactive Systems , 1987 .

[48]  David Chapman,et al.  What are plans for? , 1990, Robotics Auton. Syst..

[49]  Allen Newell,et al.  The Knowledge Level , 1989, Artif. Intell..

[50]  Nils J. Nilsson,et al.  Toward agent programs with circuit semantics , 1992 .

[51]  Huosheng Hu,et al.  Software and Hardware Architecture of a Mobile Robot for Manufacturing , 1995 .

[52]  Leslie Pack Kaelbling,et al.  Learning in embedded systems , 1993 .

[53]  Long Ji Lin,et al.  Programming Robots Using Reinforcement Learning and Teaching , 1991, AAAI.

[54]  Gerald DeJong,et al.  Explanation-based manipulator learning: Acquisition of planning ability through observation , 1985, Proceedings. 1985 IEEE International Conference on Robotics and Automation.

[55]  J. Fodor.  The Modularity of mind. An essay on faculty psychology , 1986 .

[56]  F. W. Irwin,et al.  Habits, Their Making and Unmaking , 1933, The Psychological Clinic.

[57]  Philip E. Agre,et al.  The dynamic structure of everyday life , 1988 .

[58]  Melinda T. Gervasio,et al.  Learning General Completable Reactive Plans , 1990, AAAI.

[59]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[60]  Leslie Pack Kaelbling,et al.  Action and planning in embedded agents , 1990, Robotics Auton. Syst..

[61]  Pat Langley,et al.  Reactive and Automatic Behavior in Plan Execution , 1994, AIPS.

[62]  David Chapman,et al.  Vision, instruction, and action , 1990 .

[63]  Richard S. Sutton,et al.  Temporal credit assignment in reinforcement learning , 1984 .

[64]  Deepak Kumar.  An Integrated Model of Acting and Inference , 1989, SNePS.

[65]  James S. Albus,et al.  RCS: A Reference Model Architecture for Intelligent Systems , 1995 .

[66]  Michael Hucka,et al.  Robo-Soar: An integration of external interaction, planning, and learning using Soar , 1991, Robotics Auton. Syst..

[67]  Sridhar Mahadevan,et al.  Scaling Reinforcement Learning to Robotics by Exploiting the Subsumption Architecture , 1991, ML.

[68]  Eric A. Jenkins,et al.  How Children Discover New Strategies , 1989 .

[69]  Stuart J. Russell.  An architecture for bounded rationality , 1991, SGAR.

[70]  Henry Hexmoor.  Smarts are in the architecture , 1995 .

[71]  Ellin Kofsky Scholnick,et al.  The planning construct in the psychological literature. , 1987 .

[72]  Patrick Henry Winston,et al.  Learning structural descriptions from examples , 1970 .

[73]  J. Adams,et al.  A closed-loop theory of motor learning. , 1971, Journal of motor behavior.

[74]  Stuart C. Shapiro,et al.  Of Elephants and Men , 1995 .

[75]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[76]  Stuart C. Shapiro,et al.  Autonomous Agent Architecture for Integrating Perception and Acting with Grounded Embodied Symbolic Reasoning , 1992 .

[77]  Stuart C. Shapiro,et al.  Modelling a Rational Cognitive Agent in SNePS , 1991, EPIA.

[78]  Mark Drummond,et al.  Situated Control Rules , 1989, KR.

[79]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[80]  D. Kumar.  Current Trends in SNePS — Semantic Network Processing System , 1990, Lecture Notes in Computer Science.

[81]  Alan D. Christiansen,et al.  Learning reliable manipulation strategies without initial physical models , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[82]  Richard L. Francis,et al.  Artificial Experts: Social Knowledge and Intelligent Machines , 1991 .

[83]  Leslie Pack Kaelbling,et al.  Goals as Parallel Program Specifications , 1988, AAAI.