QoS-Oriented Wireless Routing for Smart Meter Data Collection: Stochastic Learning on Graph

To ensure resilient and reliable meter data collection that is essential for the smart grid operation, we propose a QoS-oriented wireless routing scheme. Specifically tailored for the heterogeneity of the meter data traffic in the smart grid, we first design a novel utility function that not only jointly accounts for system throughput and transmission latency, but also allows for flexible tradeoff between the two with a strict transmission latency constraint, as desired by various smart meter applications. Then, we model the interactions among smart meter data concentrators as a mixed-strategy network formation game. To avoid potential information exchange which is not always practical in meter data collection scenario, a stochastic reinforcement learning algorithm with only private and incomplete information is proposed to solve the network formation problem. Such a problem formulation, together with our proposed stochastic learning algorithm on graph, results in a steady probabilistic route. Both contributions are novel and unique in comparison with existing work on this topic. Another distinct feature of our approach is its capability of effectively maintaining the QoS of smart meter data collection, even when the network is under fault or attack, as verified by simulations.

[1]  Mazen O. Hasna,et al.  Outage probability of multihop transmission over Nakagami fading channels , 2003, IEEE Communications Letters.

[2]  Quanyan Zhu,et al.  Hierarchical Network Formation Games in the Uplink of Multi-Hop Wireless Networks , 2009, GLOBECOM 2009 - 2009 IEEE Global Telecommunications Conference.

[3]  Walid Saad,et al.  Network Formation Games Among Relay Stations in Next Generation Wireless Networks , 2011, IEEE Transactions on Communications.

[4]  Xianfu Chen,et al.  Improving energy efficiency in Green femtocell networks: A hierarchical reinforcement learning framework , 2012, 2013 IEEE International Conference on Communications (ICC).

[5]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[6]  Michael P. Wellman,et al.  Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm , 1998, ICML.

[7]  Yi Yang,et al.  Nonintrusive, Self-Organizing, and Probabilistic Classification and Identification of Plugged-In Electric Loads , 2013, IEEE Transactions on Smart Grid.

[8]  Nei Kato,et al.  A Lightweight Message Authentication Scheme for Smart Grid Communications , 2011, IEEE Transactions on Smart Grid.

[9]  Lei Zheng,et al.  Reliable Wireless Communication Networks for Demand Response Control , 2013, IEEE Transactions on Smart Grid.

[10]  Husheng Li,et al.  QoS Routing in Smart Grid , 2010, 2010 IEEE Global Telecommunications Conference GLOBECOM 2010.

[11]  Christian Wietfeld,et al.  Implementation design and performance analysis of an IEC 61850 MMS server , 2012, 2012 IEEE Third International Conference on Smart Grid Communications (SmartGridComm).

[12]  Xiaoyan Hong,et al.  Delay analysis of Wireless Ad Hoc networks: Single vs. multiple radio , 2010, IEEE Local Computer Network Conference.

[13]  M. Tidball,et al.  Adapting behaviors through a learning process , 2006 .

[14]  Wenpeng Luan,et al.  Smart grid communication network capacity planning for power utilities , 2010, IEEE PES T&D 2010.

[15]  Nei Kato,et al.  Toward intelligent machine-to-machine communications in smart grid , 2011, IEEE Communications Magazine.

[16]  Xiang Cheng,et al.  Interference-aware graph based resource sharing for device-to-device communications underlaying cellular networks , 2013, 2013 IEEE Wireless Communications and Networking Conference (WCNC).

[17]  Aram Galstyan,et al.  Dynamics of Boltzmann Q learning in two-player two-action games. , 2011, Physical review. E, Statistical, nonlinear, and soft matter physics.

[18]  Rose Qingyang Hu,et al.  Scalable Distributed Communication Architectures to Support Advanced Metering Infrastructure in Smart Grid , 2012, IEEE Transactions on Parallel and Distributed Systems.

[19]  Michael P. Wellman,et al.  Conjectural Equilibrium in Multiagent Learning , 1998, Machine Learning.

[20]  Xiang Cheng,et al.  A QoS-oriented routing game for smart meter data collection , 2013, 2013 IEEE Globecom Workshops (GC Wkshps).

[21]  Dusit Niyato,et al.  Optimizations of Power Consumption and Supply in the Smart Grid: Analysis of the Impact of Data Communication Reliability , 2013, IEEE Transactions on Smart Grid.

[22]  Walid Saad,et al.  Tree Formation with Physical Layer Security Considerations in Wireless Multi-Hop Networks , 2012, IEEE Transactions on Wireless Communications.

[23]  Yi Xu,et al.  A survey on the communication architectures in smart grid , 2011, Comput. Networks.

[24]  Tansu Alpcan,et al.  Security Risk Management via Dynamic Games with Learning , 2011, 2011 IEEE International Conference on Communications (ICC).

[25]  Csaba Szepesvári,et al.  A Unified Analysis of Value-Function-Based Reinforcement-Learning Algorithms , 1999, Neural Computation.

[26]  Walid Saad,et al.  A Game Theoretic Approach for Multi-hop Power Line Communications , 2011, GAMENETS.

[27]  Liuqing Yang,et al.  Relay-Aided Amplify-and-Forward Powerline Communications , 2013, IEEE Transactions on Smart Grid.

[28]  Jeff S. Shamma,et al.  Unified convergence proofs of continuous-time fictitious play , 2004, IEEE Transactions on Automatic Control.

[29]  Bernd Girod,et al.  Adaptive playout scheduling and loss concealment for voice communication over IP networks , 2003, IEEE Trans. Multim..

[30]  Quanyan Zhu,et al.  Interference Aware Routing Game for Cognitive Radio Multi-Hop Networks , 2012, IEEE Journal on Selected Areas in Communications.

[31]  Sheng Zhong,et al.  Sprite: a simple, cheat-proof, credit-based system for mobile ad-hoc networks , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[32]  Yi Yang,et al.  Front-End Electronic Circuit Topology Analysis for Model-Driven Classification and Monitoring of Appliance Loads in Smart Buildings , 2012, IEEE Transactions on Smart Grid.

[33]  Zhu Han,et al.  Coalitional Graph Games for Popular Content Distribution in Cognitive Radio VANETs , 2013, IEEE Transactions on Vehicular Technology.

[34]  Xianfu Chen,et al.  Stochastic Power Adaptation with Multiagent Reinforcement Learning for Cognitive Wireless Mesh Networks , 2013, IEEE Transactions on Mobile Computing.

[35]  Liuqing Yang,et al.  Phasor state estimation from PMU measurements with bad data , 2011, 2011 4th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[36]  James MacGregor Smith,et al.  Properties and performance modelling of finite buffer M/G/1/K networks , 2011, Comput. Oper. Res..

[37]  Xianfu Chen,et al.  On Improving Energy Efficiency within Green Femtocell Networks: A Hierarchical Reinforcement Learning Approach , 2013, ArXiv.

[38]  Samson Lasaulce,et al.  Learning equilibria with partial information in decentralized wireless networks , 2011, IEEE Communications Magazine.

[39]  Yi Xu,et al.  Wireless Mesh Network in Smart Grid: Modeling and Analysis for Time Critical Communications , 2013, IEEE Transactions on Wireless Communications.