Reinforcement Learning Approach to Managing Distributed Data Processing Tasks in Wireless Sensing Networks

As data processing capabilities and techniques continue to rapidly improve across disciplines, the modern engineering community has become increasingly reliant on sensor data to provide an accurate assessment of system behavior and performance [8, 12, 13]. However, because of the high cost of installing and maintaining data cables in large engineered systems, it is often impractical to install sensing transducers in sufficient numbers. As such, wireless sensing systems, which can be deployed at less than one-tenth of the cost of traditional tethered systems, are being explored as a new interface between sensing transducer and data repository. In addition their low costs, wireless sensing networks (WSNs) have also shown great promise because of their ability to process sensor data locally at each wireless node. Local data processing is especially advantageous when confronted with the huge amounts of data commonly associated with dense networks of sensors: transmitting only processed results (instead of raw sensor data) can drastically reduce the amount of data needing to be communicated. As such, many different architectures have been developed for embedded data processing using wireless sensors [4, 9, 10]. Recently, the WSN community has begun investigating increasingly parallel methods of in-network processing. For example, data aggregation and fusion techniques [11], query processing [14], and explicitly parallel architectures [21] have all been adopted in an attempt to create a framework for the autonomous, in-network processing of large tracts of sensor data. With these advances in mind, one of the key challenges yet to be overcome is that within a wireless environment many system resources (such as processing power and wireless bandwidth) required to perform complex computational tasks are available only in a limited manner. As such, in networks where multiple computational tasks may need to be executed simultaneously, it is important to devise an autonomous method of distributing these scarce system resources across multiple computational objectives. The goal of this study is to apply reinforcement learning (RL) techniques to create offline-trainable online-learning agents which can successfully allocate a WSN’s scarce resources across a set of simultaneous computational objectives.