Federated Deep Learning for Immersive Virtual Reality over Wireless Networks

In this paper, the problem of enhancing the virtual reality (VR) experience for wireless users is investigated by minimizing the occurrence of breaks in presence (BIPs) that can detach the users from their virtual world. To measure the BIPs for wireless VR users, a novel model that jointly considers the VR applications, transmission delay, VR video quality, and users’ awareness of the virtual environment is proposed. In the developed model, the base stations (BSs) transmit VR videos to the wireless VR users using directional transmission links so as to increase the data rate of VR users, thus, reducing the number of BIPs for each user. Therefore, the mobility and orientation of VR users must be considered when minimizing BIPs, since the body movements of a VR user may result in blockage of its wireless link. The BIP problem is formulated as an optimization problem which jointly considers the predictions of users’ mobility patterns, orientations, and their BS association. To predict the orientation and mobility patterns of VR users, a distributed learning algorithm based on the machine learning framework of deep echo state networks (ESNs) is proposed. The proposed algorithm uses concept from federated learning to enable multiple BSs to locally train their deep ESNs using their collected data and cooperatively build a learning model to predict the entire users’ mobility patterns and orientations. Using these predictions, the user association policy that minimizes BIPs is derived. Simulation results demonstrate that the developed algorithm reduces the users’ BIPs by up to 16% and 26%, respectively, compared to centralized ESN and deep learning algorithms.