TEDGE-Caching: Transformer-based Edge Caching Towards 6G Networks

As a consequence of the COVID-19 pandemic, the demand for telecommunication for remote learning/working and telemedicine has significantly increased. Mobile Edge Caching (MEC) in the 6G networks has been evolved as an efficient solution to meet the phenomenal growth of the global mobile data traffic by bringing multimedia content closer to the users. Although massive connectivity enabled by MEC networks will significantly increase the quality of communications, there are several key challenges ahead. The limited storage of edge nodes, the large size of multimedia content, and the time-variant users’ preferences make it critical to efficiently and dynamically predict the popularity of content to store the most upcoming requested ones before being requested. Recent advancements in Deep Neural Networks (DNNs) have drawn much research attention to predict the content popularity in proactive caching schemes. Existing DNN models in this context, however, suffer from longterm dependencies, computational complexity, and unsuitability for parallel computing. To tackle these challenges, we propose an edge caching framework incorporated with the attention-based Vision Transformer (ViT) neural network, referred to as the Transformer-based Edge (TEDGE) caching, which to the best of our knowledge, is being studied for the first time. Moreover, the TEDGE caching framework requires no data pre-processing and additional contextual information. Simulation results corroborate the effectiveness of the proposed TEDGE caching framework in comparison to its counterparts.

[1]  Yongbo Li,et al.  DeepChunk: Deep Q-Learning for Chunk-Based Caching in Wireless Data Processing Networks , 2019, IEEE Transactions on Cognitive Communications and Networking.

[2]  Chia-Cheng Yen,et al.  Video Popularity Prediction: An Autoencoder Approach With Clustering , 2020, IEEE Access.

[3]  Christophe Diot,et al.  Cache content-selection policies for streaming video services , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[4]  Mihaela van der Schaar,et al.  Popularity-driven content caching , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[5]  Meixia Tao,et al.  Deep Learning for Wireless Coded Caching With Unknown and Time-Variant Content Popularity , 2021, IEEE Transactions on Wireless Communications.

[6]  Senem Velipasalar,et al.  Deep Reinforcement Learning-Based Edge Caching in Wireless Networks , 2020, IEEE Transactions on Cognitive Communications and Networking.

[7]  M. Shamim Hossain,et al.  Mobility-Aware Proactive Edge Caching for Connected Vehicles Using Federated Learning , 2021, IEEE Transactions on Intelligent Transportation Systems.

[8]  Long Shi,et al.  Dynamic Content Update for Wireless Edge Caching via Deep Reinforcement Learning , 2019, IEEE Communications Letters.

[9]  Arash Mohammadi,et al.  Deep Reinforcement Learning for Trustworthy and Time-Varying Connection Scheduling in a Coupled UAV-Based Femtocaching Architecture , 2021, IEEE Access.

[10]  Kuo Chun Tsai,et al.  Mobile social media networks caching with convolutional neural network , 2018, 2018 IEEE Wireless Communications and Networking Conference Workshops (WCNCW).

[11]  Junhao Wen,et al.  PA-Cache: Evolving Learning-Based Popularity- Aware Content Caching in Edge Networks , 2020, IEEE Transactions on Network and Service Management.

[12]  Yuhong Liu,et al.  LSTM for Mobility Based Content Popularity Prediction in Wireless Caching Networks , 2019, 2019 IEEE Globecom Workshops (GC Wkshps).

[13]  Jong Hyuk Park,et al.  DeepCachNet: A Proactive Caching Framework Based on Deep Learning in Cellular Networks , 2019, IEEE Network.

[14]  F. Maxwell Harper,et al.  The MovieLens Datasets: History and Context , 2016, TIIS.

[15]  Suzhi Bi,et al.  Joint Cache Placement and Bandwidth Allocation for FDMA-based Mobile Edge Computing Systems , 2020, ICC 2020 - 2020 IEEE International Conference on Communications (ICC).

[16]  Ying-Yi Hong,et al.  Day-Ahead Solar Irradiation Forecasting Utilizing Gramian Angular Field and Convolutional Long Short-Term Memory , 2020, IEEE Access.

[17]  Tony Q. S. Quek,et al.  Cooperative Caching and Transmission Design in Cluster-Centric Small Cell Networks , 2016, IEEE Transactions on Wireless Communications.

[18]  Choong Seon Hong,et al.  Deep Learning Based Caching for Self-Driving Cars in Multi-Access Edge Computing , 2018, IEEE Transactions on Intelligent Transportation Systems.

[19]  Haitian Pang,et al.  Toward Edge-Assisted Video Content Intelligent Caching With Long Short-Term Memory Learning , 2019, IEEE Access.

[20]  Nei Kato,et al.  HCP: Heterogeneous Computing Platform for Federated Learning Based Collaborative Content Caching Towards 6G Networks , 2022, IEEE Transactions on Emerging Topics in Computing.

[21]  Nam Thoai,et al.  Attention-Based Neural Network: A Novel Approach for Predicting the Popularity of Online Content , 2019, 2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference on Smart City; IEEE 5th International Conference on Data Science and Systems (HPCC/SmartCity/DSS).

[22]  Arash Mohammadi,et al.  Joint Transmission Scheme and Coded Content Placement in Cluster-Centric UAV-Aided Cellular Networks , 2021, IEEE Internet of Things Journal.

[23]  Ning Zhang,et al.  Online Proactive Caching in Mobile Edge Computing Using Bidirectional Deep Recurrent Neural Network , 2019, IEEE Internet of Things Journal.

[24]  Michela Meo,et al.  Caching at the edge in high energy-efficient wireless access networks , 2020, ICC 2020 - 2020 IEEE International Conference on Communications (ICC).

[25]  Hyundong Shin,et al.  Content-Aware Proactive Caching for Backhaul Offloading in Cellular Network , 2018, IEEE Transactions on Wireless Communications.

[26]  Chau Yuen,et al.  Towards Hit-Interruption Tradeoff in Vehicular Edge Caching: Algorithm and Analysis , 2021 .

[27]  Nei Kato,et al.  Edge Cloud Server Deployment With Transmission Power Control Through Machine Learning for 6G Internet of Things , 2021, IEEE Transactions on Emerging Topics in Computing.

[28]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[29]  Xiaofei Wang,et al.  Federated Deep Reinforcement Learning for Internet of Things With Decentralized Cooperative Edge Caching , 2020, IEEE Internet of Things Journal.

[30]  Jamshid Abouei,et al.  Cache Replacement Schemes Based on Adaptive Time Window for Video on Demand Services in Femtocell Networks , 2019, IEEE Transactions on Mobile Computing.

[31]  Arash Mohammadi,et al.  Mobility-Aware Femtocaching Algorithm in D2D Networks Based on Handover , 2020, IEEE Transactions on Vehicular Technology.

[32]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[33]  Geoffrey E. Hinton,et al.  Layer Normalization , 2016, ArXiv.

[34]  Apostolos Avranas,et al.  Spatial Multi-LRU Caching for Wireless Networks with Coverage Overlaps , 2016, SIGMETRICS.