Reinforcement learning‐based clustering protocols for a self‐organising cognitive radio network

This paper presents a methodology on how cognitive radio networks can form clusters by exploiting reinforcement learning-based principles. Each node repeatedly senses the received signal strength indicator beacons given off by other nodes in the network. This information can be used by the nodes to learn about its positioning significance within the network and whether to become cluster heads thus forming a cluster. Extensive simulation results are presented, and it is shown that on average, the clustering performance packing efficiency can be improved when nodes have the ability to learn. The results are compared with that of a k-means and node degree-based approach for the formation of clusters. It is found that the node degree scheme can result in the clustering performance deteriorating, which is undesirable as it can lead to an increase in the total energy consumption of the network over time. It is shown that in a shadowing environment, that clusters formed via learning through received signal strength indicator can reduce their transmission power by up to 2i¾?dBW achieving a potential power saving of 37i¾?per cent while achieving the same signal-to-noise ratio as that of the no learning and node degree schemes. Further energy conservation can be obtained by restricting beacon transmissions to immediate neighbouring nodes. In certain geographical node distributions, a no learning scheme is preferred for the formation of clusters due to its lower overhead. Copyright © 2015 John Wiley & Sons, Ltd.

[1]  Wendi Heinzelman,et al.  Energy-efficient communication protocol for wireless microsensor networks , 2000, Proceedings of the 33rd Annual Hawaii International Conference on System Sciences.

[2]  Anantha P. Chandrakasan,et al.  An application-specific protocol architecture for wireless microsensor networks , 2002, IEEE Trans. Wirel. Commun..

[3]  Lajos Hanzo,et al.  Green radio: radio techniques to enable energy-efficient wireless networks , 2011, IEEE Communications Magazine.

[4]  Stefano Basagni,et al.  Distributed clustering for ad hoc networks , 1999, Proceedings Fourth International Symposium on Parallel Architectures, Algorithms, and Networks (I-SPAN'99).

[5]  Chih-Yu Wen,et al.  Distributed Clustering With Directional Antennas for Wireless Sensor Networks , 2013, IEEE Sensors Journal.

[6]  Michalis Vazirgiannis,et al.  c ○ 2001 Kluwer Academic Publishers. Manufactured in The Netherlands. On Clustering Validation Techniques , 2022 .

[7]  Robert Tappan Morris,et al.  Span: An Energy-Efficient Coordination Algorithm for Topology Maintenance in Ad Hoc Wireless Networks , 2002, Wirel. Networks.

[8]  David Grace,et al.  RF signal Strength based clustering protocols for a self-organizing cognitive radio network , 2010, 2010 7th International Symposium on Wireless Communication Systems.

[9]  David Grace,et al.  Efficient exploration in reinforcement learning-based cognitive radio spectrum sharing , 2011, IET Commun..

[10]  Qin Wang,et al.  A Realistic Power Consumption Model for Wireless Sensor Network Devices , 2006, 2006 3rd Annual IEEE Communications Society on Sensor and Ad Hoc Communications and Networks.

[11]  Sankalpa Gamwarige,et al.  Cluster Density of Dependent Thinning Distributed Clustering Class of Algorithms in Ad Hoc Deployed Wireless Networks , 2012, J. Comput. Networks Commun..

[12]  Ossama Younis,et al.  HEED: a hybrid, energy-efficient, distributed clustering approach for ad hoc sensor networks , 2004, IEEE Transactions on Mobile Computing.

[13]  Ajith Abraham,et al.  Self Organizing Sensor Networks Using Intelligent Clustering , 2006, ICCSA.

[14]  Mario Gerla,et al.  Multicluster, mobile, multimedia radio network , 1995, Wirel. Networks.

[15]  Maurice K. Wong,et al.  Algorithm AS136: A k-means clustering algorithm. , 1979 .

[16]  Ramon Lawrence,et al.  Cluster head selection using RF signal strength , 2009, 2009 Canadian Conference on Electrical and Computer Engineering.

[17]  Paolo Santi,et al.  A probabilistic analysis for the range assignment problem in ad hoc networks , 2001, MobiHoc.

[18]  L. Kleinrock,et al.  Spatial reuse in multihop packet radio networks , 1987, Proceedings of the IEEE.

[19]  Adrian Perrig,et al.  ACE: An Emergent Algorithm for Highly Uniform Cluster Formation , 2004, EWSN.

[20]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[21]  Ossama Younis,et al.  Node clustering in wireless sensor networks: recent developments and deployment challenges , 2006, IEEE Network.

[22]  M Kobayashi,et al.  Green Small-Cell Networks , 2011, IEEE Vehicular Technology Magazine.

[23]  Christian Bettstetter.  The cluster density of a distributed clustering algorithm in ad hoc networks , 2004, 2004 IEEE International Conference on Communications (IEEE Cat. No.04CH37577).

[24]  David Grace,et al.  Two-stage reinforcement-learning-based cognitive radio with exploration control , 2011, IET Commun..

[25]  Ajith Abraham,et al.  Self Organizing Sensors by Minimization of Cluster Heads Using Intelligent Clustering , 2006, J. Digit. Inf. Manag..

[26]  Jae-Won Choi,et al.  A Cluster Head Selection Algorithm Adopting Sensing-Awareness and Sensor Density for Wireless Sensor Networks , 2007, IEICE Trans. Commun..