Scan Strategies for Meteorological Radars

We address the problem of adaptive sensor control in dynamic resource-constrained sensor networks. We focus on a meteorological sensing network comprising radars that can perform sector scanning rather than always scanning 360 degrees. We compare three sector scanning strategies. The sit-and-spin strategy always scans 360 degrees. The limited lookahead strategy additionally uses the expected environmental state K decision epochs in the future, as predicted from Kalman filters, in its decision-making. The full lookahead strategy uses all expected future states by casting the problem as a Markov decision process and using reinforcement learning to estimate the optimal scan strategy. We show that the main benefits of using a lookahead strategy are when there are multiple meteorological phenomena in the environment, and when the maximum radius of any phenomenon is sufficiently smaller than the radius of the radars. We also show that there is a trade-off between the average quality with which a phenomenon is scanned and the number of decision epochs before which a phenomenon is rescanned.

[1]  Kevin P. Murphy,et al.  A Survey of POMDP Solution Techniques , 2000 .

[2]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[3]  Dieter Fox,et al.  Reinforcement learning for sensing strategies , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[4]  Sridhar Mahadevan,et al.  Switching kalman filters for prediction and tracking in an adaptive meteorological sensing network , 2005, 2005 Second Annual IEEE Communications Society Conference on Sensor and Ad Hoc Communications and Networks, 2005. IEEE SECON 2005..

[5]  Alfred O. Hero,et al.  Non-myopic approaches to scheduling agile sensors for multistage detection, tracking and identification , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[6]  David R. Cox,et al.  A simple spatial-temporal model of rainfall , 1988, Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences.

[7]  Michael Zink,et al.  An End-User-Responsive Sensor Network Architecture for Hazardous Weather Detection, Prediction and Response , 2006, AINTEC.

[8]  Richard S. Sutton,et al.  Generalization in ReinforcementLearning : Successful Examples UsingSparse Coarse , 1996 .

[9]  Brian C. Donovan,et al.  Improved radar sensitivity through limited sector scanning: The DCAS approach , 2005 .

[10]  Peter Stone,et al.  Reinforcement Learning for RoboCup Soccer Keepaway , 2005, Adapt. Behav..

[11]  Barbara F. La Scala,et al.  Multi step ahead beam and waveform scheduling for tracking of manoeuvering targets in clutter , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[12]  P. S. Eagleson,et al.  Mathematical models of rainstorm events in space and time , 1987 .

[13]  Roderic A. Grupen,et al.  A feedback control structure for on-line learning tasks , 1997, Robotics Auton. Syst..

[14]  Ben Tse,et al.  Autonomous Inverted Helicopter Flight via Reinforcement Learning , 2004, ISER.