Trust-Based Social Networks with Computing, Caching and Communications: A Deep Reinforcement Learning Approach

Social networks have continuously been expanding and trying to be innovative. The recent advances of computing, caching, and communication (3C) can have significant impacts on mobile social networks (MSNs). MSNs can leverage these new paradigms to provide a new mechanism for users to share resources (e.g., information, computation-based services). In this paper, we exploit the intrinsic nature of social networks, i.e., the trust formed through social relationships among users, to enable users to share resources under the framework of 3C. Specifically, we consider the mobile edge computing (MEC), in-network caching and device-to-device (D2D) communications. When considering the trust-based MSNs with MEC, caching and D2D, we apply a novel deep reinforcement learning approach to automatically make a decision for optimally allocating the network resources. The decision is made purely through observing the network's states, rather than any handcrafted or explicit control rules, which makes it adaptive to variable network conditions. Google TensorFlow is used to implement the proposed deep $Q$-learning approach. Simulation results with different network parameters are presented to show the effectiveness of the proposed scheme.

[1]  F. Richard Yu,et al.  Optimization of cache-enabled opportunistic interference alignment wireless networks: A big data deep reinforcement learning approach , 2017, 2017 IEEE International Conference on Communications (ICC).

[2]  Hong Shen Wang,et al.  Finite-state Markov channel-a useful model for radio communication channels , 1995 .

[3]  Victor C. M. Leung,et al.  Deep-Reinforcement-Learning-Based Optimization for Cache-Enabled Opportunistic Interference Alignment Wireless Networks , 2017, IEEE Transactions on Vehicular Technology.

[4]  F. Richard Yu,et al.  Resource Allocation for Information-Centric Virtualized Heterogeneous Networks With In-Network Caching and Mobile Edge Computing , 2017, IEEE Transactions on Vehicular Technology.

[5]  Frank L. Lewis,et al.  Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  Leonard J. Cimini,et al.  MobiCacher: Mobility-aware content caching in small-cell networks , 2014, 2014 IEEE Global Communications Conference.

[7]  Hsiao-Hwa Chen,et al.  Cooperative Device-to-Device Communications: Social Networking Perspectives , 2017, IEEE Network.

[8]  Boleslaw K. Szymanski,et al.  Friendship Based Routing in Delay Tolerant Mobile Social Networks , 2010, 2010 IEEE Global Telecommunications Conference GLOBECOM 2010.

[9]  Hao Chen,et al.  Enabling cyber-physical communication in 5G cellular networks: challenges, spatial spectrum sensing, and cyber-security , 2017, IET Cyper-Phys. Syst.: Theory & Appl..

[10]  Victor C. M. Leung,et al.  Software-Defined Networks with Mobile Edge Computing and Caching for Smart Cities: A Big Data Deep Reinforcement Learning Approach , 2017, IEEE Communications Magazine.

[11]  Khaled Ben Letaief,et al.  Delay-optimal computation task scheduling for mobile-edge computing systems , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[12]  Xing Zhang,et al.  A Survey on Mobile Edge Networks: Convergence of Computing, Caching and Communications , 2017, IEEE Access.

[13]  R. Michael Buehrer,et al.  Learning distributed caching strategies in small cell networks , 2014, 2014 11th International Symposium on Wireless Communications Systems (ISWCS).

[14]  Wha Sook Jeon,et al.  Two-Stage Semi-Distributed Resource Management for Device-to-Device Communication in Cellular Networks , 2014, IEEE Transactions on Wireless Communications.

[15]  Jia Guo,et al.  Trust-Based Service Management for Social Internet of Things Systems , 2016, IEEE Transactions on Dependable and Secure Computing.

[16]  Ke Zhang,et al.  Energy-Efficient Offloading for Mobile Edge Computing in 5G Heterogeneous Networks , 2016, IEEE Access.

[17]  B. Aazhang,et al.  Cellular networks with an overlaid device to device network , 2008, 2008 42nd Asilomar Conference on Signals, Systems and Computers.

[18]  Ioannis Lambadaris,et al.  Trust establishment in cooperative wireless networks , 2010, 2010 - MILCOM 2010 MILITARY COMMUNICATIONS CONFERENCE.

[19]  Deniz Gündüz,et al.  Learning-based optimization of cache content in a small cell base station , 2014, 2014 IEEE International Conference on Communications (ICC).

[20]  Wei Wang,et al.  Proactive storage at caching-enable base stations in cellular networks , 2013, 2013 IEEE 24th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC).

[21]  Marco Di Renzo,et al.  Power-Availability-Aware Cell Association for Energy-Harvesting Small-Cell Base Stations , 2017, IEEE Transactions on Wireless Communications.

[22]  Sergio Barbarossa,et al.  Joint Optimization of Radio and Computational Resources for Multicell Mobile-Edge Computing , 2014, IEEE Transactions on Signal and Information Processing over Networks.

[23]  Zhu Han,et al.  Social-Aware Data Dissemination via Device-to-Device Communications: Fusing Social and Mobile Networks with Incentive Constraints , 2019, IEEE Transactions on Services Computing.

[24]  Zhu Han,et al.  Caching based socially-aware D2D communications in wireless content delivery networks: a hypergraph framework , 2016, IEEE Wireless Communications.

[25]  Zhou Su,et al.  Content distribution over content centric mobile social networks in 5G , 2015, IEEE Communications Magazine.

[26]  Qing Wang,et al.  A Survey on Device-to-Device Communication in Cellular Networks , 2013, IEEE Communications Surveys & Tutorials.

[27]  Qianbin Chen,et al.  Computation Offloading and Resource Allocation in Wireless Cellular Networks With Mobile Edge Computing , 2017, IEEE Transactions on Wireless Communications.

[28]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[29]  Nan Zhao,et al.  Integrated Networking, Caching, and Computing for Connected Vehicles: A Deep Reinforcement Learning Approach , 2018, IEEE Transactions on Vehicular Technology.

[30]  Peter Xiaoping Liu,et al.  Distributed Combined Authentication and Intrusion Detection With Data Fusion in High-Security Mobile Ad Hoc Networks , 2010, IEEE Transactions on Vehicular Technology.

[31]  Song Guo,et al.  Big Data Meet Green Challenges: Big Data Toward Green Applications , 2016, IEEE Systems Journal.

[32]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[33]  Thomas M. Chen,et al.  Dempster-Shafer theory for intrusion detection in ad hoc networks , 2005, IEEE Internet Computing.

[34]  Kun Yang,et al.  Mobile Social Networks: Architectures, Social Properties, and Key Research Challenges , 2013, IEEE Communications Surveys & Tutorials.

[35]  Hao Yi Ong,et al.  Distributed Deep Q-Learning , 2015, ArXiv.

[36]  Mahdi Imani,et al.  Control of Gene Regulatory Networks With Noisy Measurements and Uncertain Inputs , 2017, IEEE Transactions on Control of Network Systems.

[37]  Yang Yi,et al.  Reservoir Computing Meets Smart Grids: Attack Detection Using Delayed Feedback Networks , 2018, IEEE Transactions on Industrial Informatics.

[38]  Pan Hui,et al.  BUBBLE Rap: Social-Based Forwarding in Delay-Tolerant Networks , 2008, IEEE Transactions on Mobile Computing.

[39]  Sherali Zeadally,et al.  Vehicular delay-tolerant networks for smart grid data management using mobile edge computing , 2016, IEEE Communications Magazine.

[40]  Panos M. Pardalos,et al.  Approximate dynamic programming: solving the curses of dimensionality , 2009, Optim. Methods Softw..

[41]  Depeng Jin,et al.  Vehicular Fog Computing: A Viewpoint of Vehicles as the Infrastructures , 2016, IEEE Transactions on Vehicular Technology.

[42]  Wenzhong Li,et al.  Efficient Multi-User Computation Offloading for Mobile-Edge Cloud Computing , 2015, IEEE/ACM Transactions on Networking.

[43]  Song Guo,et al.  Big Data Meet Green Challenges: Greening Big Data , 2016, IEEE Systems Journal.

[44]  F. Richard Yu,et al.  Information-Centric Virtualized Cellular Networks With Device-to-Device Communications , 2016, IEEE Transactions on Vehicular Technology.

[45]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[46]  Zhu Han,et al.  Distributed Interference and Energy-Aware Power Control for Ultra-Dense D2D Networks: A Mean Field Game , 2017, IEEE Transactions on Wireless Communications.

[47]  Carey L. Williamson,et al.  Estimating Instantaneous Cache Hit Ratio Using Markov Chain Analysis , 2013, IEEE/ACM Transactions on Networking.

[48]  Zaher Dawy,et al.  Social Network Aware Device-to-Device Communication in Wireless Networks , 2015, IEEE Transactions on Wireless Communications.

[49]  Xianfu Chen,et al.  Energy-Efficiency Oriented Traffic Offloading in Wireless Networks: A Brief Survey and a Learning Approach for Heterogeneous Cellular Networks , 2015, IEEE Journal on Selected Areas in Communications.

[50]  Qichao Xu,et al.  Security-Aware Resource Allocation for Mobile Social Big Data: A Matching-Coalitional Game Solution , 2017, IEEE Transactions on Big Data.

[51]  Brian L. Mark,et al.  A quantitative trust establishment framework for reliable data packet delivery in MANETs , 2005, SASN '05.