Iris: Deep Reinforcement Learning Driven Shared Spectrum Access Architecture for Indoor Neutral-Host Small Cells

We consider indoor mobile access, a vital use case for current and future mobile networks. For this key use case, we outline a vision that combines a neutral-host-based shared small-cell infrastructure with a common pool of spectrum for dynamic sharing as a way forward to proliferate indoor small-cell deployments and open up the mobile operator ecosystem. Toward this vision, we focus on the challenges pertaining to managing access to shared spectrum [e.g., 3.5-GHz U.S. Citizen Broadband Radio Service (CBRS) spectrum]. We propose Iris, a practical shared spectrum access architecture for indoor neutral-host small-cells. At the core of Iris is a deep reinforcement learning-based dynamic pricing mechanism that efficiently mediates access to shared spectrum for diverse operators in a way that provides incentives for operators and the neutral-host alike. We then present the Iris system architecture that embeds this dynamic pricing mechanism alongside cloud-RAN and RAN slicing design principles in a practical neutral-host design tailored for the indoor small-cell environment. Using a prototype implementation of the Iris system, we present the extensive experimental evaluation results that not only offer insight into the Iris dynamic pricing process and its superiority over alternative approaches but also demonstrate its deployment feasibility.

[1]  Egon Schulz,et al.  Multi-carrier waveform based flexible inter-operator spectrum sharing for 5G systems , 2014, 2014 IEEE International Symposium on Dynamic Spectrum Access Networks (DYSPAN).

[2]  Sergey Levine,et al.  High-Dimensional Continuous Control Using Generalized Advantage Estimation , 2015, ICLR.

[3]  R. M. A. P. Rajatheva,et al.  On spectrum sharing among micro-operators in 5G , 2017, 2017 European Conference on Networks and Communications (EuCNC).

[4]  Toktam Mahmoodi,et al.  Network slicing management & prioritization in 5G mobile systems , 2016 .

[5]  Raymond Knopp,et al.  FlexCRAN: A flexible functional split framework over ethernet fronthaul in Cloud-RAN , 2017, 2017 IEEE International Conference on Communications (ICC).

[6]  Sampath Rangarajan,et al.  CellSlice: Cellular wireless resource slicing for active RAN sharing , 2013, 2013 Fifth International Conference on Communication Systems and Networks (COMSNETS).

[7]  Lei Li,et al.  Recent Progress on C-RAN Centralization and Cloudification , 2014, IEEE Access.

[8]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[9]  Xia Zhou,et al.  eBay in the Sky: strategy-proof wireless spectrum auctions , 2008, MobiCom '08.

[10]  Mahesh K. Marina,et al.  Communication-Free Inter-Operator Interference Management in Shared Spectrum Small Cell Networks , 2019, IEEE Transactions on Cognitive Communications and Networking.

[11]  Irena Trajkovska,et al.  The emergence of operator-neutral small cells as a strong case for cloud computing at the mobile edge , 2016, Trans. Emerg. Telecommun. Technol..

[12]  Petri Ahokangas,et al.  Future micro operators business models in 5 G , 2016 .

[13]  Marco Gramaglia,et al.  Optimising 5G infrastructure markets: The business of network slicing , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[14]  Andrew G. Barto,et al.  Reinforcement Learning and Dynamic Programming , 1995 .

[15]  Bikramjit Singh,et al.  Co-primary inter-operator spectrum sharing over a limited spectrum pool using repeated games , 2015, 2015 IEEE International Conference on Communications (ICC).

[16]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[17]  Ulas C. Kozat,et al.  Stochastic Game for Wireless Network Virtualization , 2013, IEEE/ACM Transactions on Networking.

[18]  Binh Nguyen,et al.  ECHO: A Reliable Distributed Cellular Core Network for Hyper-scale Public Clouds , 2018, MobiCom.

[19]  Patrick M. Pilarski,et al.  Model-Free reinforcement learning with continuous action in practice , 2012, 2012 American Control Conference (ACC).

[20]  Kentaro Ishizu,et al.  Heterogeneous Networks in Shared Spectrum Access Communications , 2017, IEEE Journal on Selected Areas in Communications.

[21]  Kentaro Ishizu,et al.  Resource allocation in shared spectrum access communications for operators with diverse service requirements , 2016, EURASIP J. Adv. Signal Process..

[22]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[23]  Mingyan Liu,et al.  Revenue generation for truthful spectrum auction in dynamic spectrum access , 2009, MobiHoc '09.

[24]  Sangtae Ha,et al.  TUBE: time-dependent pricing for mobile data , 2012, SIGCOMM '12.

[25]  Gustavo de Veciana,et al.  Multi-Tenant Radio Access Network Slicing: Statistical Multiplexing of Spatial Loads , 2016, IEEE/ACM Transactions on Networking.

[26]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[27]  Qian Zhang,et al.  FlexAuc: Serving Dynamic Demands in a Spectrum Trading Market With Flexible Auction , 2014, IEEE Transactions on Wireless Communications.

[28]  Mahesh K. Marina,et al.  Experience Building a Prototype 5G Testbed , 2018, EM-5G@CoNEXT.

[29]  Adam Wierman,et al.  Pricing data center demand response , 2014, SIGMETRICS '14.

[30]  Kentaro Ishizu,et al.  Shared Spectrum Access Communications: A Neutral Host Micro Operator Approach , 2017, IEEE Journal on Selected Areas in Communications.

[31]  Anastasios Kourtis,et al.  A Novel Architectural Concept for Enhanced 5G Network Facilities , 2017 .

[32]  Bikramjit Singh,et al.  Repeated spectrum sharing games in multi-operator heterogeneous networks , 2015, 2015 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN).

[33]  Matti Latva-aho,et al.  Analysis of Spectrum Valuation Approaches: The Viewpoint of Local 5G Networks in Shared Spectrum Bands , 2018, 2018 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN).

[34]  Lin Gao,et al.  An Integrated Contract and Auction Design for Secondary Spectrum Trading , 2013, IEEE Journal on Selected Areas in Communications.

[35]  Yi Wu,et al.  Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments , 2017, NIPS.

[36]  Frank Kelly,et al.  Rate control for communication networks: shadow prices, proportional fairness and stability , 1998, J. Oper. Res. Soc..

[37]  Jordi Pérez-Romero,et al.  Introducing Mobile Edge Computing Capabilities through Distributed 5G Cloud Enabled Small Cells , 2016, Mob. Networks Appl..

[38]  Leandros Tassiulas,et al.  Experimental evaluation of functional splits for 5G cloud-RANs , 2017, 2017 IEEE International Conference on Communications (ICC).

[39]  Faqir Zarrar Yousaf,et al.  Resource Sharing for a 5G Multi-tenant and Multi-service Architecture , 2017 .

[40]  Yuval Tassa,et al.  DeepMind Control Suite , 2018, ArXiv.

[41]  Matti Latva-aho,et al.  Business Models for Local 5G Micro Operators , 2018, 2018 IEEE International Symposium on Dynamic Spectrum Access Networks (DySPAN).

[42]  K. K. Ramakrishnan,et al.  Double Auctions for Dynamic Spectrum Allocation , 2014, IEEE/ACM Transactions on Networking.

[43]  Christian Bonnet,et al.  OpenAirInterface: A Flexible Platform for 5G Research , 2014, CCRV.

[44]  Mahesh K. Marina,et al.  Orion: RAN Slicing for a Flexible and Cost-Effective Multi-Service Mobile Network Architecture , 2017, MobiCom.

[45]  Na Li,et al.  Optimal demand response based on utility maximization in power networks , 2011, 2011 IEEE Power and Energy Society General Meeting.

[46]  Matti Latva-aho,et al.  Micro Operators to Boost Local Service Delivery in 5G , 2017, Wireless Personal Communications.

[47]  Olasunkanmi Durowoju,et al.  LTE indoor small cell capacity and coverage comparison , 2013, 2013 IEEE 24th International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC Workshops).

[48]  Shamik Sengupta,et al.  Designing Auction Mechanisms for Dynamic Spectrum Access , 2008, Mob. Networks Appl..

[49]  Petri Mähönen,et al.  Riding the data tsunami in the cloud: myths and challenges in future wireless access , 2013, IEEE Communications Magazine.

[50]  Marco Gramaglia,et al.  Mobile traffic forecasting for maximizing 5G network slicing resource utilization , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[51]  Pekka Ojanen,et al.  On a new incentive and market based framework for multi-tier shared spectrum access systems , 2014, 2014 IEEE International Symposium on Dynamic Spectrum Access Networks (DYSPAN).

[52]  Lars Thiele,et al.  Coordinated multipoint: Concepts, performance, and field trial results , 2011, IEEE Communications Magazine.

[53]  Steven H. Low,et al.  Optimization flow control—I: basic algorithm and convergence , 1999, TNET.

[54]  Kun Zhu,et al.  Virtualization of 5G Cellular Networks as a Hierarchical Combinatorial Auction , 2015, IEEE Transactions on Mobile Computing.

[55]  Bart De Schutter,et al.  Reinforcement Learning and Dynamic Programming Using Function Approximators , 2010 .

[56]  Petri Ahokangas,et al.  Spectrum sharing using licensed shared access: the concept and its workflow for LTE-advanced networks , 2014, IEEE Wireless Communications.

[57]  Pieter Abbeel,et al.  Benchmarking Deep Reinforcement Learning for Continuous Control , 2016, ICML.

[58]  Guy Lever,et al.  Deterministic Policy Gradient Algorithms , 2014, ICML.

[59]  Antonio Pescapè,et al.  A tool for the generation of realistic network workload for emerging networking scenarios , 2012, Comput. Networks.

[60]  Sergey Levine,et al.  Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[61]  Subhash Suri,et al.  Towards real-time dynamic spectrum auctions , 2008, Comput. Networks.

[62]  Sampath Rangarajan,et al.  NVS: A Substrate for Virtualizing Wireless Resources in Cellular Networks , 2012, IEEE/ACM Transactions on Networking.