A Survey of Decision-Theoretic Models for Cognitive Internet of Things (CIoT)

Communication technology and wide spread usage of Internet of Things (IoT) are rapidly becoming the core enabler for consumers to use their smart devices in their daily routine. These smart devices are gradually transforming the IoT scenario into a new paradigm called cognitive Internet of Things (CIoT). We believe that CIoT will revolutionize many service sectors including smart cities, transportation, health-care, and environmental monitoring. The current development in CIoT is still elusive as it has to achieve effective interoperability and autonomous decision making. Most importantly, the decision theoretic models are seen as an enabler to achieve effective interoperability among heterogeneous CIoT objects. In this paper, we provide a survey of the existing decision theoretic models and their usage for CIoT, an architectural CIoT framework is also proposed to discuss the open issues and solution for potential challenges emerging in the area of CIoT research.

[1]  Qihui Wu,et al.  Game Theoretic Channel Selection for Opportunistic Spectrum Access with Unknown Prior Information , 2011, 2011 IEEE International Conference on Communications (ICC).

[2]  Husheng Li,et al.  Multi-Agent Q-Learning for Competitive Spectrum Access in Cognitive Radio Systems , 2010, 2010 Fifth IEEE Workshop on Networking Technologies for Software Defined Radio Networks (SDR).

[3]  Sungwook Kim.  New Game Paradigm for IoT Systems , 2018 .

[4]  Lang Tong,et al.  Maximum Throughput Region of Multiuser Cognitive Access of Continuous Time Markovian Channels , 2011, IEEE Journal on Selected Areas in Communications.

[5]  Mingyan Liu,et al.  Atomic Congestion Games on Graphs and Their Applications in Networking , 2012, IEEE/ACM Transactions on Networking.

[6]  Mingyan Liu,et al.  Optimal Channel Probing and Transmission Scheduling for Opportunistic Spectrum Access , 2007, IEEE/ACM Transactions on Networking.

[7]  Alagan Anpalagan,et al.  Opportunistic Spectrum Access in Unknown Dynamic Environment: A Game-Theoretic Stochastic Learning Solution , 2012, IEEE Transactions on Wireless Communications.

[8]  Mihaela van der Schaar,et al.  Repeated Games with Intervention: Theory and Applications in Communications , 2011, IEEE Transactions on Communications.

[9]  J. Gittins.  Bandit processes and dynamic allocation indices , 1979 .

[10]  L. Shapley,et al.  Potential Games , 1994 .

[11]  Mihaela van der Schaar,et al.  Learning to Compete for Resources in Wireless Stochastic Games , 2009, IEEE Transactions on Vehicular Technology.

[12]  Gabriel-Miro Muntean,et al.  Game Theory-Based Network Selection: Solutions and Challenges , 2012, IEEE Communications Surveys & Tutorials.

[13]  P. Whittle.  Restless bandits: activity allocation in a changing world , 1988, Journal of Applied Probability.

[14]  Dusit Niyato,et al.  Optimal Channel Access Management with QoS Support for Cognitive Vehicular Networks , 2011, IEEE Transactions on Mobile Computing.

[15]  Mehul Motani,et al.  Price-Based Resource Allocation for Spectrum-Sharing Femtocell Networks: A Stackelberg Game Approach , 2012, 2011 IEEE Global Telecommunications Conference - GLOBECOM 2011.

[16]  Alagan Anpalagan,et al.  Energy-efficient exploration and exploitation of multichannel diversity in spectrum sharing systems , 2012, Trans. Emerg. Telecommun. Technol..

[17]  Shaojie Tang,et al.  Optimal Frequency-Temporal Opportunity Exploitation for Multichannel Ad Hoc Networks , 2012, IEEE Transactions on Parallel and Distributed Systems.

[18]  Ananthram Swami,et al.  Decentralized cognitive MAC for opportunistic spectrum access in ad hoc networks: A POMDP framework , 2007, IEEE Journal on Selected Areas in Communications.

[19]  Choong Seon Hong,et al.  An Autonomic SLA Management for IoT Networks , 2016 .

[20]  Mehdi Bennis,et al.  A Q-learning based approach to interference avoidance in self-organized femtocell networks , 2010, 2010 IEEE Globecom Workshops.

[21]  Tommi Mikkonen,et al.  A Roadmap to the Programmable World: Software Challenges in the IoT Era , 2017, IEEE Software.

[22]  Peter Friess,et al.  Internet of Things Strategic Research Roadmap , 2011 .

[23]  Qihui Wu,et al.  Cognitive Internet of Things: A New Paradigm Beyond Connection , 2014, IEEE Internet of Things Journal.

[24]  Ashutosh Sabharwal,et al.  Opportunistic spectral usage: bounds and a multi-band CSMA/CA protocol , 2007, TNET.

[25]  Vera Stavroulaki,et al.  Cognitive Management for the Internet of Things: A Framework for Enabling Autonomous Applications , 2013, IEEE Vehicular Technology Magazine.

[26]  H. Vincent Poor,et al.  Optimal selection of channel sensing order in cognitive radio , 2009, IEEE Transactions on Wireless Communications.

[27]  Mingyan Liu,et al.  Online learning in opportunistic spectrum access: A restless bandit approach , 2010, 2011 Proceedings IEEE INFOCOM.

[28]  Brian M. Sadler,et al.  Opportunistic Spectrum Access via Periodic Channel Sensing , 2008, IEEE Transactions on Signal Processing.

[29]  T. L. Lai Andherbertrobbins.  Asymptotically Efficient Adaptive Allocation Rules , 1985 .

[30]  Xu Chen,et al.  Evolutionarily Stable Spectrum Access , 2012, IEEE Transactions on Mobile Computing.

[31]  Neil Immerman,et al.  The Complexity of Decentralized Control of Markov Decision Processes , 2000, UAI.

[32]  Qihui Wu,et al.  Multichannel Opportunistic Spectrum Access in Fading Environment Using Optimal Stopping Rule , 2011 .

[33]  Sungwook Kim.  Learning-Based QoS Control Algorithms for Next Generation Internet of Things , 2015, Mob. Inf. Syst..

[34]  Alagan Anpalagan,et al.  Distributed Channel Selection in CRAHNs with Heterogeneous Spectrum Opportunities: A Local Congestion Game Approach , 2012, IEICE Trans. Commun..

[35]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[36]  Jai E. Jung,et al.  Game theoretic approach on Real‐time decision making for IoT‐based traffic light control , 2017, Concurr. Comput. Pract. Exp..

[37]  Stephen S. Yau,et al.  Intelligent Planning for Developing Mobile IoT Applications Using Cloud Systems , 2014, 2014 IEEE International Conference on Mobile Services.

[38]  D. E. Matthews.  Evolution and the Theory of Games , 1977 .

[39]  Zhi-Quan Luo,et al.  A Stackelberg game approach to distributed spectrum management , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[40]  Qing Zhao,et al.  Cooperative Game in Dynamic Spectrum Access with Unknown Model and Imperfect Sensing , 2012, IEEE Transactions on Wireless Communications.

[41]  Andrea Montanari,et al.  Convergence to Equilibrium in Local Interaction Games , 2009, FOCS.

[42]  Yi Gai,et al.  Decentralized Online Learning Algorithms for Opportunistic Spectrum Access , 2011, 2011 IEEE Global Telecommunications Conference - GLOBECOM 2011.

[43]  V. V. Phansalkar,et al.  Decentralized Learning of Nash Equilibria in Multi-Person Stochastic Games With Incomplete Information , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[44]  Athanasios V. Vasilakos,et al.  Game Dynamics and Cost of Learning in Heterogeneous 4G Networks , 2012, IEEE Journal on Selected Areas in Communications.

[45]  Sungwook Kim.  R-learning-based team game model for Internet of things quality-of-service control scheme , 2017, Int. J. Distributed Sens. Networks.

[46]  Jason R. Marden,et al.  Cooperative Control and Potential Games , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[47]  Alagan Anpalagan,et al.  A game-theoretic perspective on self-organizing optimization for cognitive small cells , 2015, IEEE Communications Magazine.

[48]  Alagan Anpalagan,et al.  Decision-Theoretic Distributed Channel Selection for Opportunistic Spectrum Access: Strategies, Challenges and Solutions , 2013, IEEE Communications Surveys & Tutorials.

[49]  Peter Friess,et al.  Internet of Things Applications - From Research and Innovation to Market Deployment , 2014 .

[50]  Craig Boutilier,et al.  Sequential Optimality and Coordination in Multiagent Systems , 1999, IJCAI.

[51]  Mérouane Debbah,et al.  Coalition Formation Games for Collaborative Spectrum Sensing , 2010, IEEE Transactions on Vehicular Technology.

[52]  Ao Tang,et al.  Opportunistic Spectrum Access with Multiple Users: Learning under Competition , 2010, 2010 Proceedings IEEE INFOCOM.

[53]  Jonathan P. How,et al.  Decentralized control of partially observable Markov decision processes , 2015, 52nd IEEE Conference on Decision and Control.

[54]  Weihua Zhuang,et al.  Simple Channel Sensing Order in Cognitive Radio Networks , 2011, IEEE Journal on Selected Areas in Communications.

[55]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[56]  Alagan Anpalagan,et al.  Optimal Energy-Efficient Channel Exploration for Opportunistic Spectrum Usage , 2012, IEEE Wireless Communications Letters.

[57]  Ariel Rubinstein,et al.  A Course in Game Theory , 1995 .

[58]  Mingyan Liu,et al.  Spectrum Sharing as Spatial Congestion Games , 2010, ArXiv.

[59]  Joelle Pineau,et al.  Online Planning Algorithms for POMDPs , 2008, J. Artif. Intell. Res..

[60]  Setareh Maghsudi,et al.  Game Theoretic Mechanisms for Resource Management in Massive Wireless IoT Systems , 2017, IEEE Communications Magazine.

[61]  M. Shamim Hossain,et al.  SMDP-Based Radio Resource Allocation Scheme in Software-Defined Internet of Things Networks , 2016, IEEE Sensors Journal.

[62]  Hwee Pink Tan,et al.  Markov Decision Processes With Applications in Wireless Sensor Networks: A Survey , 2015, IEEE Communications Surveys & Tutorials.

[63]  Qing Zhao,et al.  Distributed Learning in Multi-Armed Bandit With Multiple Players , 2009, IEEE Transactions on Signal Processing.

[64]  Alagan Anpalagan,et al.  Exploiting Multichannel Diversity in Spectrum Sharing Systems Using Optimal Stopping Rule , 2012 .

[65]  Alexander Gluhak,et al.  The SENSEI Real World Internet Architecture , 2010, Future Internet Assembly.

[66]  Sandeep K. Sood,et al.  A Game Theoretic Approach for an IoT-Based Automated Employee Performance Evaluation , 2017, IEEE Systems Journal.

[67]  Aurélien Garivier,et al.  Optimally Sensing a Single Channel Without Prior Information: The Tiling Algorithm and Regret Bounds , 2011, IEEE Journal of Selected Topics in Signal Processing.

[68]  Zhu Han,et al.  Competitive Spectrum Access in Cognitive Radio Networks: Graphical Game and Learning , 2010, 2010 IEEE Wireless Communication and Networking Conference.

[69]  Klaus Moessner,et al.  Enabling smart cities through a cognitive management framework for the internet of things , 2013, IEEE Communications Magazine.

[70]  Lin Chen,et al.  On Optimality of Myopic Policy for Restless Multi-Armed Bandit Problem: An Axiomatic Approach , 2012, IEEE Transactions on Signal Processing.

[71]  J. Walrand,et al.  Asymptotically efficient allocation rules for the multiarmed bandit problem with multiple plays-Part II: Markovian rewards , 1987 .

[72]  H. Vincent Poor,et al.  Cognitive Medium Access: Exploration, Exploitation, and Competition , 2007, IEEE Transactions on Mobile Computing.

[73]  Yonghong Zeng,et al.  Opportunistic spectrum access for energy-constrained cognitive radios , 2009 .

[74]  Sandeep K. Sood,et al.  Game theoretic decision making in IoT-assisted activity monitoring of defence personnel , 2017, Multimedia Tools and Applications.

[75]  Vikram Krishnamurthy,et al.  Transmission control in cognitive radio as a Markovian dynamic game: Structural result on randomized threshold policies , 2010, IEEE Transactions on Communications.

[76]  Georgios B. Giannakis,et al.  Sequential and cooperative sensing for multi-channel cognitive radios , 2010, IEEE Trans. Signal Process..

[77]  Alagan Anpalagan,et al.  Opportunistic Spectrum Access in Cognitive Radio Networks: Global Optimization Using Local Interaction Games , 2012, IEEE Journal of Selected Topics in Signal Processing.

[78]  Rajarathnam Chandramouli,et al.  Distributed Learning in Secondary Spectrum Sharing Graphical Game , 2011, 2011 IEEE Global Telecommunications Conference - GLOBECOM 2011.

[79]  Xuemin Shen,et al.  HC-MAC: A Hardware-Constrained Cognitive MAC for Efficient Spectrum Management , 2008, IEEE Journal on Selected Areas in Communications.

[80]  Mohsen Guizani,et al.  Internet of Things Architecture: Recent Advances, Taxonomy, Requirements, and Open Challenges , 2017, IEEE Wireless Communications.

[81]  Berthold Vöcking.  Congestion Games: Optimization in Competition , 2006, ACiD.

[82]  Mubashir Husain Rehmani,et al.  Cognitive-Radio-Based Internet of Things: Applications, Architectures, Spectrum Related Functionalities, and Future Research Directions , 2017, IEEE Wireless Communications.

[83]  Ananthram Swami,et al.  Distributed Algorithms for Learning and Cognitive Medium Access with Logarithmic Regret , 2010, IEEE Journal on Selected Areas in Communications.

[84]  Hamid Aghvami,et al.  Cognitive Machine-to-Machine Communications for Internet-of-Things: A Protocol Stack Perspective , 2015, IEEE Internet of Things Journal.

[85]  Ananthram Swami,et al.  Joint Design and Separation Principle for Opportunistic Spectrum Access in the Presence of Sensing Errors , 2007, IEEE Transactions on Information Theory.

[86]  Mingyan Liu,et al.  Mining Spectrum Usage Data: A Large-Scale Spectrum Measurement Study , 2009, IEEE Transactions on Mobile Computing.

[87]  Venugopal V. Veeravalli,et al.  Algorithms for Dynamic Spectrum Access With Learning for Cognitive Radio , 2008, IEEE Transactions on Signal Processing.