Learning to Control Forest Fires

Forest fires are an important environmental problem. This paper describes a methodology for constructing an intelligent system which aims to support the human expert's decision making in fire control. The idea is based on first implementing a fire spread simulator and on searching for good decision policies by reinforcement learning (RL). RL algorithms optimize policies by letting the agents interact with the simulator and learn from their experiences. Finally, we observe different problems and propose solutions for solving them. Among these problems are storing policies for huge state spaces and coping with multiple agents which need to learn cooperative strategies.

[1]  Jürgen Schmidhuber,et al.  CMAC models learn to play soccer , 1998 .

[2]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[3]  Tran van Nao.  Forest Fires in North America , 1982 .

[4]  R. Bellman,et al.  V. Adaptive Control Processes , 1964 .

[5]  Luca Maria Gambardella,et al.  Ant colony system: a cooperative learning approach to the traveling salesman problem , 1997, IEEE Trans. Evol. Comput..

[6]  Tom Beer.  The Australian National Bushfire model project , 1990 .

[7]  James S. Albus,et al.  New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)1 , 1975 .

[8]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[9]  Tom Beer,et al.  The interaction of wind and fire , 1991 .

[10]  Andrew G. Barto,et al.  Improving Elevator Performance Using Reinforcement Learning , 1995, NIPS.

[11]  C. Ball,et al.  Society of American Foresters , 1991 .

[12]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[13]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[14]  G. Calabri.  Recent Evolution and Prospects for the Mediterranean Region , 1982 .

[15]  Richard S. Sutton,et al.  Generalization in ReinforcementLearning : Successful Examples UsingSparse Coarse , 1996 .

[16]  P. Avesani,et al.  Combining CBR and Constraint Reasoning in Planning Forest Fire Fighting1 , 2000 .

[17]  Marco Colombetti,et al.  Robot Shaping: An Experiment in Behavior Engineering , 1997 .

[18]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[19]  Jürgen Schmidhuber,et al.  Efficient model-based exploration , 1998 .

[20]  Toshio Odanaka,et al.  ADAPTIVE CONTROL PROCESSES , 1990 .

[21]  Andrea Emilio Rizzoli,et al.  Delivering environmental decision support systems: Software tools and techniques , 1997 .