Constructing Deep Neural Networks with a Priori Knowledge of Wireless Tasks

Deep neural networks (DNNs) have been employed for designing wireless systems in many aspects, say transceiver design, resource optimization, and information prediction. Existing works either use the fully-connected DNN or the DNNs with particular architectures developed in other domains. While generating labels for supervised learning and gathering training samples are time-consuming or cost-prohibitive, how to develop DNNs with wireless priors for reducing training complexity remains open. In this paper, we show that two kinds of permutation invariant properties widely existed in wireless tasks can be harnessed to reduce the number of model parameters and hence the sample and computational complexity for training. We find special architecture of DNNs whose input-output relationships satisfy the properties, called permutation invariant DNN (PINN), and augment the data with the properties. By learning the impact of the scale of a wireless system, the size of the constructed PINNs can flexibly adapt to the input data dimension. We take predictive resource allocation and interference coordination as examples to show how the PINNs can be employed for learning the optimal policy with unsupervised and supervised learning. Simulations results demonstrate a dramatic gain of the proposed PINNs in terms of reducing training complexity.

[1]  Ami Wiesel,et al.  Learning to Detect , 2018, IEEE Transactions on Signal Processing.

[2]  Geoffrey E. Hinton,et al.  Generative models for discovering sparse distributed representations. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[3]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[4]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[5]  Chenyang Yang,et al.  Predictive Resource Allocation with Deep Learning , 2018, 2018 IEEE 88th Vehicular Technology Conference (VTC-Fall).

[6]  Florent Altché,et al.  An LSTM network for highway trajectory prediction , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[7]  M. Hestenes Multiplier and gradient methods , 1969 .

[8]  Ying Li,et al.  Deep Learning Coordinated Beamforming for Highly-Mobile Millimeter Wave Systems , 2018, IEEE Access.

[9]  Jörg Widmer,et al.  Data-Driven Evaluation of Anticipatory Networking in LTE Networks , 2018, IEEE Transactions on Mobile Computing.

[10]  Geoffrey Ye Li,et al.  Model-Driven Deep Learning for Physical Layer Communications , 2018, IEEE Wireless Communications.

[11]  Nikos D. Sidiropoulos,et al.  Learning to optimize: Training deep neural networks for wireless resource management , 2017, 2017 IEEE 18th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[12]  Mérouane Debbah,et al.  Wireless Networks Design in the Era of Deep Learning: Model-Based, AI-Based, or Both? , 2019, IEEE Transactions on Communications.

[13]  Chau Yuen,et al.  Deep Learning for UL/DL Channel Calibration in Generic Massive MIMO Systems , 2019, ICC 2019 - 2019 IEEE International Conference on Communications (ICC).

[14]  Chenyang Yang,et al.  Exploiting Future Radio Resources With End-to-End Prediction by Deep Learning , 2018, IEEE Access.

[15]  Lajos Hanzo,et al.  Optimizing Wireless Systems Using Unsupervised and Reinforced-Unsupervised Deep Learning , 2020, IEEE Network.

[16]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[17]  Chenyang Yang,et al.  Structure of Deep Neural Networks with a Priori Information in Wireless Tasks , 2019, ICC 2020 - 2020 IEEE International Conference on Communications (ICC).

[18]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[19]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[20]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[21]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[22]  Geoffrey Ye Li,et al.  Power of Deep Learning for Channel Estimation and Signal Detection in OFDM Systems , 2017, IEEE Wireless Communications Letters.

[23]  Chenyang Yang,et al.  Learning to Optimize with Unsupervised Learning: Training Deep Neural Networks for URLLC , 2019, 2019 IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC).

[24]  Fei Liang,et al.  Power Control for Interference Management via Ensembling Deep Neural Networks , 2019, 2019 IEEE/CIC International Conference on Communications in China (ICCC).

[25]  Di Su,et al.  User-Centric Downlink Cooperative Transmission With Orthogonal Beamforming Based Limited Feedback , 2015, IEEE Transactions on Communications.

[26]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..