Monitoring Strategies for Embedded Agents: Experiments and Analysis

Monitoring is an important activity for any embedded agent. To operate effectively, agents must gather information about their environment. The policy by which they do this is called a monitoring strategy. Our work has focused on classifying different types of monitoring strategies and understanding how strategies depend on features of the task and environment. We have discovered only a few general monitoring strategies, in particular periodic and interval reduction, and speculate that there are no more. The relative advantages and generality of each strategy will be discussed in detail. The wide applicability of interval reduction will be demonstrated both empirically and analytically. We conclude with a number of general laws that state when a strategy is most appropriate.

[1]  Neville Moray,et al.  Monitoring behavior and supervisory control , 1986 .

[2]  Tarald O. Kvålseth,et al.  The Effect of Cost on the Sampling Behavior of Human Instrument Monitors , 1976 .

[3]  Alberto Rovetta,et al.  Robot motion planning with parallel systems , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[4]  Eric A. Hansen,et al.  Cost-Effective Sensing during Plan Execution , 1994, AAAI.

[5]  R. Bajcsy.  Active perception , 1988 .

[6]  David Chapman,et al.  Pengi: An Implementation of a Theory of Activity , 1987, AAAI.

[7]  C F Lowe,et al.  The role of verbal behavior in human learning: infant performance on fixed-interval schedules. , 1983, Journal of the experimental analysis of behavior.

[8]  C F Lowe,et al.  Species differences in temporal control of behavior II: human performance. , 1978, Journal of the experimental analysis of behavior.

[9]  B A Schneider,et al.  A two-state analysis of fixed-interval responding in the pigeon. , 1969, Journal of the experimental analysis of behavior.

[10]  Drew McDermott,et al.  Robot Planning , 1991, AI Mag..

[11]  David Atkinson,et al.  Generating Perception Requests and Expectations to Verify the Execution of Plans , 1986, AAAI.

[12]  L. Rowell Huesmann,et al.  The thinking computer : mind inside matter , 1977 .

[13]  Richard Fikes,et al.  Learning and Executing Generalized Robot Plans , 1993, Artif. Intell..

[14]  S. Ceci,et al.  "Don't forget to take the cupcakes out of the oven": prospective memory, strategic time-monitoring, and context. , 1985, Child development.

[15]  C. Lowe,et al.  Temporal control of behavior and the power law. , 1979, Journal of the experimental analysis of behavior.

[16]  Paul R. Cohen,et al.  Learning Monitoring Strategies , 1994 .

[17]  James A. Hendler,et al.  A model of reaction for planning in dynamic environments , 1988, Artif. Intell. Eng..

[18]  Kalyanmoy Deb,et al.  A Comparative Analysis of Selection Schemes Used in Genetic Algorithms , 1990, FOGA.

[19]  M. Atkin,et al.  Genetric Programming to Learn an Agent''s Monitoring Strategy , 1993 .

[20]  Paul R. Cohen,et al.  Intelligent, Real-Time Problem Solving , 1990 .

[21]  Paul R. Cohen,et al.  Learning monitoring strategies: a difficult genetic programming application , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[22]  G. Pyke.  Optimal Foraging Theory: A Critical Review , 1984 .

[23]  Thomas B. Sheridan,et al.  Monitoring Behavior and Supervisory Control , 1976 .

[24]  Reid G. Simmons,et al.  Sensible Planning: Focusing Perceptual Attention , 1991, AAAI.

[25]  Amy L. Lansky,et al.  Reactive Reasoning and Planning , 1987, AAAI.

[26]  Jean-Claude Latombe,et al.  Robot Motion Planning with Uncertainty in Control and Sensing , 1991, Artif. Intell..

[27]  Richard S. Sutton,et al.  Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming , 1990, ML.

[28]  J. Wearden,et al.  The Power Law and Weber's Law in Fixed-Interval Postreinforcement Pausing: A Scalar Timing Model , 1985 .

[29]  Marcel Schoppers,et al.  Universal Plans for Reactive Robots in Unpredictable Environments , 1987, IJCAI.

[30]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[31]  Paul R. Cohen,et al.  Learning a Decision Rule for Monitoring Tasks with Deadlines , 1992 .

[32]  Jaime R. Carbonell,et al.  A Queueing Model of Visual Sampling Experimental Validation , 1968 .

[33]  Stephen J. Ceci,et al.  Don't forget to take the cupcakes out of the oven": prospective memory, strategic time-monitoring, and context. , 1985 .

[34]  P. Killeen.  On the temporal control of behavior. , 1975 .

[35]  D. E. Goldberg,et al.  Genetic Algorithms in Search, Optimization & Machine Learning , 1989 .

[36]  N. Mackintosh.  The psychology of animal learning , 1974 .

[37]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[38]  Drew McDermott,et al.  Planning and Acting , 1978, Cogn. Sci..

[39]  Sam Steel,et al.  Integrating Planning, Execution and Monitoring , 1988, AAAI.

[40]  C. B. Ferster,et al.  Schedules of reinforcement , 1957 .

[41]  J. R. Carbonell.  A Queueing Model of Many-Instrument Visual Sampling , 1966 .

[42]  Thomas B. Sheridan,et al.  On How Often the Supervisor Should Sample , 1970, IEEE Trans. Syst. Sci. Cybern..

[43]  Paul R. Cohen,et al.  The Interval Reduction Strategy for Monitoring Cupcake Problems , 1994 .

[44]  Pattie Maes,et al.  Designing autonomous agents: Theory and practice from biology to engineering and back , 1990, Robotics Auton. Syst..

[45]  Leslie Pack Kaelbling,et al.  An Architecture for Intelligent Reactive Systems , 1987 .

[46]  John R. Koza,et al.  Evolution of Food-Foraging Strategies for the Caribbean Anolis Lizard Using Genetic Programming , 1992, Adapt. Behav..

[47]  Graham H. Pyke,et al.  Optimal Foraging: A Selective Review of Theory and Tests , 1977, The Quarterly Review of Biology.

[48]  John R. Koza,et al.  Automatic Programming of Robots Using Genetic Programming , 1992, AAAI.

[49]  R. James Firby,et al.  An Investigation into Reactive Planning in Complex Domains , 1987, AAAI.

[50]  David Chapman,et al.  What are plans for? , 1990, Robotics Auton. Syst..