QoE-Aware Power Control for UAV-Aided Media Transmission with Reinforcement Learning

Unmanned aerial vehicles (UAVs) are widely utilized to capture and compress videos of the target area and then transmit the processed videos to the control station (CS) on the ground. The media transmissions in the UAV-aided network face many challenges due to the highly dynamic network topology and limited resources such as bandwidth and energy. This paper introduces a media transmission scheme in the UAV-aided network utilizing reinforcement learning algorithms to efficiently process and transmit the captured video, which is able to improve the quality-of-experience (QoE) and reduce the energy consumption. Exploiting the proposed reinforcement learning algorithm, the UAV dynamically selects the quantization parameter in the source coding process and determines the transmit power without knowing the video transmission model. Simulation results demonstrate that the proposed scheme is capable of achieving a higher video quality and utility with lower energy consumption compared with the state-of-the-art schemes.

[1]  Qingqing Wu,et al.  Joint Trajectory and Communication Design for Multi-UAV Enabled Wireless Networks , 2017, IEEE Transactions on Wireless Communications.

[2]  Wen Gao,et al.  Rate-distortion analysis for H.264/AVC video coding and its application to rate control , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Xiangjian He,et al.  A Joint Framework for QoS and QoE for Video Transmission over Wireless Multimedia Sensor Networks , 2018, IEEE Transactions on Mobile Computing.

[4]  Xiang Cheng,et al.  UAV-Assisted Data Dissemination Scheduling in VANETs , 2018, 2018 IEEE International Conference on Communications (ICC).

[5]  Syed Omer Gilani,et al.  H.264 Encoder Parameter Optimization for Encoded Wireless Multimedia Transmissions , 2018, IEEE Access.

[6]  Bo Cheng,et al.  Adaptive Video Transmission Control System Based on Reinforcement Learning Approach Over Heterogeneous Networks , 2015, IEEE Transactions on Automation Science and Engineering.

[7]  Gary J. Sullivan,et al.  Comparison of the Coding Efficiency of Video Coding Standards—Including High Efficiency Video Coding (HEVC) , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Wenchao Xu,et al.  Air-Ground Integrated Mobile Edge Networks: Architecture, Challenges, and Opportunities , 2018, IEEE Communications Magazine.

[9]  Chau Yuen,et al.  Enabling Adaptive High-Frame-Rate Video Streaming in Mobile Cloud Gaming Applications , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Jun Wu,et al.  Robust Uncoded Video Transmission under Practical Channel Estimation , 2016, 2016 IEEE Global Communications Conference (GLOBECOM).

[11]  Jerry D. Gibson,et al.  Global Motion Assisted Low Complexity Video Encoding for UAV Applications , 2015, IEEE Journal of Selected Topics in Signal Processing.

[12]  Yi Zhou,et al.  Multi-UAV-Aided Networks: Aerial-Ground Cooperative Vehicular Networking Architecture , 2015, IEEE Vehicular Technology Magazine.

[13]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[14]  Jin Chen,et al.  Power Control in UAV-Supported Ultra Dense Networks: Communications, Caching, and Energy Transfer , 2017, IEEE Communications Magazine.

[15]  Arafat J. Al-Dweik,et al.  Low Complexity Power Optimization Algorithm for Multimedia Transmission Over Wireless Networks , 2015, IEEE Journal of Selected Topics in Signal Processing.

[16]  Yao Wang,et al.  Q-STAR: A Perceptual Video Quality Model Considering Impact of Spatial, Temporal, and Amplitude Resolutions , 2012, IEEE Transactions on Image Processing.

[17]  Sam Kwong,et al.  Efficient Motion and Disparity Estimation Optimization for Low Complexity Multiview Video Coding , 2015, IEEE Transactions on Broadcasting.

[18]  Aggelos K. Katsaggelos,et al.  Joint source coding and transmission power management for energy efficient wireless video communications , 2002, IEEE Trans. Circuits Syst. Video Technol..

[19]  Mahbub Hassan,et al.  Implementation and evaluation of adaptive video streaming based on Markov decision process , 2016, 2016 IEEE International Conference on Communications (ICC).

[20]  Zhijun Zhang,et al.  REMT: A Real-Time End-to-End Media Data Transmission Mechanism in UAV-Aided Networks , 2018, IEEE Network.

[21]  King Ngi Ngan,et al.  Free-Energy Principle Inspired Video Quality Metric and Its Use in Video Coding , 2016, IEEE Transactions on Multimedia.

[22]  Xiao Zhang,et al.  Autonomous navigation of UAV in large-scale unknown complex environment with deep reinforcement learning , 2017, 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[23]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.