Preserving Privacy of Agents in Reinforcement Learning for Distributed Cognitive Radio Networks

Reinforcement learning (RL) is one of the artificial intelligence approaches that has been deployed effectively to improve performance of distributed cognitive radio networks (DCRNs). However, in existing proposals that involve multi-agents, perceptions of the agents are shared in plain in order to calculate optimal actions. This raises privacy concern where an agent learns private information (e.g. Q-values) of the others, which can then be used to infer, for instance, the actions of these other agents. In this paper, we provide a preliminary investigation and a privacy-preserving protocol on multi-agent RL in DCRNs. The proposed protocol provides RL computations without revealing agents’ private information. We also discuss the security and performance of the protocol.

[1]  Ferhat Özgür Çatak,et al.  Secure Multi-party Computation Based Privacy Preserving Extreme Learning Machine Algorithm Over Vertically Distributed Data , 2015, ICONIP.

[2]  Qun Li,et al.  Preserving secondary users' privacy in cognitive radio networks , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[3]  Ian F. Akyildiz,et al.  NeXt generation/dynamic spectrum access/cognitive radio wireless networks: A survey , 2006, Comput. Networks.

[4]  Riivo Talviste,et al.  Applying Secure Multi-party Computation in Practice , 2016 .

[5]  Tim Clarke,et al.  Multichannel non-persistent CSMA MAC schemes with reinforcement learning for cognitive radio networks , 2011, 2011 11th International Symposium on Communications & Information Technologies (ISCIT).

[6]  Zekeriya Erkin,et al.  Private data aggregation with groups for smart grids in a dynamic setting using CRT , 2015, 2015 IEEE International Workshop on Information Forensics and Security (WIFS).

[7]  Yehuda Lindell,et al.  Secure Multiparty Computation for Privacy-Preserving Data Mining , 2009, IACR Cryptol. ePrint Arch..

[8]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[9]  Ivan Damgård,et al.  Secure Multiparty Computation Goes Live , 2009, Financial Cryptography.

[10]  Raphael C.-W. Phan,et al.  Facial Expression Recognition in the Encrypted Domain Based on Local Fisher Discriminant Analysis , 2013, IEEE Transactions on Affective Computing.

[11]  Dan Bogdanov,et al.  Sharemind: A Framework for Fast Privacy-Preserving Computations , 2008, ESORICS.

[12]  Pascal Paillier,et al.  Public-Key Cryptosystems Based on Composite Degree Residuosity Classes , 1999, EUROCRYPT.

[13]  Qiang Ni,et al.  Application of reinforcement learning for security enhancement in cognitive radio networks , 2015, Appl. Soft Comput..

[14]  Ivan Damgård,et al.  A generalization of Paillier’s public-key system with applications to electronic voting , 2010, International Journal of Information Security.

[15]  Joseph Mitola,et al.  Cognitive radio: making software radios more personal , 1999, IEEE Wirel. Commun..

[16]  Benny Pinkas,et al.  FairplayMP: a system for secure multi-party computation , 2008, CCS.

[17]  Jing Li,et al.  Multi-relay Cooperative Mechanism with Q-Learning in Cognitive Radio Multimedia Sensor Networks , 2011, 2011IEEE 10th International Conference on Trust, Security and Privacy in Computing and Communications.

[18]  Kok-Lim Alvin Yau,et al.  Enhancing network performance in Distributed Cognitive Radio Networks using single-agent and multi-agent Reinforcement Learning , 2010, IEEE Local Computer Network Conference.

[19]  Shigenobu Kobayashi,et al.  Privacy-preserving reinforcement learning , 2008, ICML '08.