Reinforcement learning based anti-jamming with wideband autonomous cognitive radios

This paper presents a design and an implementation of a wideband autonomous cognitive radio (WACR) for anti-jamming. The proposed anti-jamming scheme is aimed at evading a jammer that sweeps across the whole wideband spectrum range in which the WACR is expected to operate. The WACR makes use of its spectrum knowledge acquisition ability to detect and identify the location of the sweeping jammer. This information and reinforcement learning is then used to learn the optimal communications mode to avoid the jammer. In this paper, we discuss a specific reinforcement learning mechanism based on Q-learning to successfully learn such an anti-jamming operation over a several hundred mega-Hz of wide spectrum in realtime. We describe a cognitive anti-jamming communications protocol that selects a spectrum position with enough contiguous idle spectrum uninterfered by both deliberate jammers and inadvertent interferers and transmits till the jammer catches up to it. When the jammer interferes with the cognitive radio's transmission, it switches to a new spectrum band that will lead to the longest possible uninterrupted transmission as learned through Q-learning. We present results of an implementation of the proposed WACR for cognitive anti-jamming and discuss its effectiveness in learning the surrounding RF environment to avoid both deliberate jamming and unintentional interference.