Toward Intelligent Vehicular Networks: A Machine Learning Framework

As wireless networks evolve toward high mobility and providing better support for connected vehicles, a number of new challenges arise due to the resulting high dynamics in vehicular environments and thus motive rethinking of traditional wireless design methodologies. Future intelligent vehicles, which are at the heart of high mobility networks, are increasingly equipped with multiple advanced onboard sensors and keep generating large volumes of data. Machine learning, as an effective approach to artificial intelligence, can provide a rich set of tools to exploit such data for the benefit of the networks. In this paper, we first identify the distinctive characteristics of high mobility vehicular networks and motivate the use of machine learning to address the resulting challenges. After a brief introduction of the major concepts of machine learning, we discuss its applications to learn the dynamics of vehicular networks and make informed decisions to optimize network performance. In particular, we discuss in greater detail the application of reinforcement learning in managing network resources as an alternative to the prevalent optimization approach. Finally, some open issues worth further investigation are highlighted.

[1]  Jianhong Wu,et al.  Data clustering - theory, algorithms, and applications , 2007 .

[2]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[3]  Strother H. Walker,et al.  Estimation of the probability of an event as a function of several independent variables. , 1967, Biometrika.

[4]  Geoffrey Ye Li,et al.  Machine Learning for Vehicular Networks , 2017, ArXiv.

[5]  Xiang Cheng,et al.  Channel Prediction Based Scheduling for Data Dissemination in VANETs , 2017, IEEE Communications Letters.

[6]  Hai Le Vu,et al.  Performance Analysis of the IEEE 802.11 MAC Protocol for DSRC Safety Applications , 2011, IEEE Transactions on Vehicular Technology.

[7]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[8]  Zhi Ding,et al.  Graph-Based Resource Sharing in Vehicular Communication , 2018, IEEE Transactions on Wireless Communications.

[9]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[10]  Nan Zhao,et al.  Integrated Networking, Caching, and Computing for Connected Vehicles: A Deep Reinforcement Learning Approach , 2018, IEEE Transactions on Vehicular Technology.

[11]  Chadi Assi,et al.  Deep reinforcement learning-based scheduling for roadside communication networks , 2017, 2017 15th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt).

[12]  M. Kubát An Introduction to Machine Learning , 2017, Springer International Publishing.

[13]  Geoffrey Ye Li,et al.  Spectrum and Power Allocation for Vehicular Communications With Delayed CSI Feedback , 2017, IEEE Wireless Communications Letters.

[14]  Xiang Cheng,et al.  Mobile Big Data: The Fuel for Data-Driven Wireless , 2017, IEEE Internet of Things Journal.

[15]  Geoffrey Ye Li,et al.  Robust channel estimation for OFDM systems with rapid dispersive fading channels , 1998, IEEE Trans. Commun..

[16]  Erik G. Ström,et al.  Radio Resource Management for D2D-Based V2V Communication , 2016, IEEE Transactions on Vehicular Technology.

[17]  Geoffrey Ye Li,et al.  Machine Learning for Vehicular Networks: Recent Advances and Application Examples , 2018, IEEE Vehicular Technology Magazine.

[18]  Erik G. Ström,et al.  Cluster-Based Radio Resource Management for D2D-Supported Safety-Critical V2X Communications , 2016, IEEE Transactions on Wireless Communications.

[19]  Minglu Li,et al.  A novel vehicular location prediction based on mobility patterns for routing in urban VANET , 2012, EURASIP J. Wirel. Commun. Netw..

[20]  Samuel Pierre,et al.  Centralized and Localized Data Congestion Control Strategy for Vehicular Ad Hoc Networks Using a Machine Learning Clustering Algorithm , 2016, IEEE Transactions on Intelligent Transportation Systems.

[21]  Robert W. Heath,et al.  Inverse Multipath Fingerprinting for Millimeter Wave V2I Beam Alignment , 2017, IEEE Transactions on Vehicular Technology.

[22]  Fei-Yue Wang,et al.  Traffic Flow Prediction With Big Data: A Deep Learning Approach , 2015, IEEE Transactions on Intelligent Transportation Systems.

[23]  Antonio Iera,et al.  LTE for vehicular networking: a survey , 2013, IEEE Communications Magazine.

[24]  Eamonn J. Keogh Nearest Neighbor , 2010, Encyclopedia of Machine Learning.

[25]  Xiang Cheng,et al.  A Novel Centralized TDMA-Based Scheduling Protocol for Vehicular Networks , 2015, IEEE Transactions on Intelligent Transportation Systems.

[26]  Cheng Wang,et al.  User Association for Load Balancing in Vehicular Networks: An Online Reinforcement Learning Approach , 2017, IEEE Transactions on Intelligent Transportation Systems.

[27]  Xiang Cheng,et al.  Interference Graph-Based Resource-Sharing Schemes for Vehicular Networks , 2013, IEEE Transactions on Vehicular Technology.

[28]  Robert W. Heath,et al.  Millimeter-Wave Vehicular Communication to Support Massive Automotive Sensing , 2016, IEEE Communications Magazine.

[29]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[30]  Jian Song,et al.  Time–Frequency Joint Sparse Channel Estimation for MIMO-OFDM Systems , 2015, IEEE Communications Letters.

[31]  Laurent Orseau,et al.  Delusion, Survival, and Intelligent Agents , 2011, AGI.

[32]  Wolfgang Kellerer,et al.  Location dependent resource allocation for mobile device-to-device communications , 2014, 2014 IEEE Wireless Communications and Networking Conference (WCNC).

[33]  Je-Won Kang,et al.  A Novel Intrusion Detection Method Using Deep Neural Network for In-Vehicle Network Security , 2016, 2016 IEEE 83rd Vehicular Technology Conference (VTC Spring).

[34]  Ye Li Pilot-symbol-aided channel estimation for OFDM in wireless systems , 2000, IEEE Trans. Veh. Technol..

[35]  Geoffrey Ye Li,et al.  Power of Deep Learning for Channel Estimation and Signal Detection in OFDM Systems , 2017, IEEE Wireless Communications Letters.

[36]  G.L. Stuber,et al.  Interchannel interference analysis of OFDM in a mobile environment , 1995, 1995 IEEE 45th Vehicular Technology Conference. Countdown to the Wireless Twenty-First Century.

[37]  Jonathan Goldstein,et al.  When Is ''Nearest Neighbor'' Meaningful? , 1999, ICDT.

[38]  David A. Landgrebe,et al.  A survey of decision tree classifier methodology , 1991, IEEE Trans. Syst. Man Cybern..

[39]  Swarun Kumar,et al.  Adaptive communication in multi-robot systems using directionality of signal strength , 2015, Int. J. Robotics Res..

[40]  Nathalie Japkowicz,et al.  Anomaly Detection in Automobile Control Network Data with Long Short-Term Memory Networks , 2016, 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[41]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[42]  T. Lumley,et al.  PRINCIPAL COMPONENT ANALYSIS AND FACTOR ANALYSIS , 2004, Statistical Methods for Biomedical Research.

[43]  D. Basak,et al.  Support Vector Regression , 2008 .

[44]  Geoffrey Ye Li,et al.  Resource Allocation for D2D-Enabled Vehicular Communications , 2017, IEEE Transactions on Communications.

[45]  Xiang Cheng,et al.  D2D for Intelligent Transportation Systems: A Feasibility Study , 2015, IEEE Transactions on Intelligent Transportation Systems.

[46]  Mohsen Guizani,et al.  Reinforcement learning for resource provisioning in the vehicular cloud , 2016, IEEE Wireless Communications.

[47]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[48]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[49]  John Schulman,et al.  Concrete Problems in AI Safety , 2016, ArXiv.

[50]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[51]  John B. Kenney,et al.  Dedicated Short-Range Communications (DSRC) Standards in the United States , 2011, Proceedings of the IEEE.

[52]  Weihua Zhuang,et al.  Stochastic Analysis of a Single-Hop Communication Link in Vehicular Ad Hoc Networks , 2014, IEEE Transactions on Intelligent Transportation Systems.

[53]  Xiang Cheng,et al.  5G-Enabled Cooperative Intelligent Vehicular (5GenCIV) Framework: When Benz Meets Marconi , 2017, IEEE Intelligent Systems.

[54]  Xiang Cheng,et al.  Exploiting Mobile Big Data: Sources, Features, and Applications , 2017, IEEE Network.

[55]  Geoffrey Ye Li,et al.  Vehicular Communications: A Physical Layer Perspective , 2017, IEEE Transactions on Vehicular Technology.

[56]  Walid Saad,et al.  Dynamic Proximity-Aware Resource Allocation in Vehicle-to-Vehicle (V2V) Communications , 2016, 2016 IEEE Globecom Workshops (GC Wkshps).

[57]  Christian Wietfeld,et al.  LTE Connectivity and Vehicular Traffic Prediction Based on Machine Learning Approaches , 2015, 2015 IEEE 82nd Vehicular Technology Conference (VTC2015-Fall).

[58]  Ulrich Kressel,et al.  Probabilistic trajectory prediction with Gaussian mixture models , 2012, 2012 IEEE Intelligent Vehicles Symposium.

[59]  Biing-Hwang Juang,et al.  Recurrent deep neural networks for robust speech recognition , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[60]  Cheng Li,et al.  Fuzzy Q-learning based vertical handoff control for vehicular heterogeneous wireless network , 2014, 2014 IEEE International Conference on Communications (ICC).

[61]  D.M. Mount,et al.  An Efficient k-Means Clustering Algorithm: Analysis and Implementation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[62]  Robert W. Heath,et al.  Millimeter Wave Vehicular Communications: A Survey , 2016, Found. Trends Netw..

[63]  Zhu Han,et al.  Machine Learning Paradigms for Next-Generation Wireless Networks , 2017, IEEE Wireless Communications.

[64]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[65]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[66]  Jerome H. Friedman,et al.  On Bias, Variance, 0/1—Loss, and the Curse-of-Dimensionality , 2004, Data Mining and Knowledge Discovery.

[67]  Yoshua Bengio,et al.  Why Does Unsupervised Pre-training Help Deep Learning? , 2010, AISTATS.

[68]  Zhi Ding,et al.  Graph-Based Radio Resource Management for Vehicular Networks , 2018, 2018 IEEE International Conference on Communications (ICC).

[69]  Ying Li,et al.  Deep Learning Coordinated Beamforming for Highly-Mobile Millimeter Wave Systems , 2018, IEEE Access.

[70]  Victor C. M. Leung,et al.  Delay-Optimal Virtualized Radio Resource Scheduling in Software-Defined Vehicular Networks via Stochastic Learning , 2016, IEEE Transactions on Vehicular Technology.

[71]  Weiqiang Dong On Bias , Variance , 0 / 1-Loss , and the Curse of Dimensionality RK April 13 , 2014 .

[72]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[73]  Shimon Whiteson,et al.  Learning to Communicate with Deep Multi-Agent Reinforcement Learning , 2016, NIPS.

[74]  Jia Yuan Yu,et al.  A Reinforcement Learning Technique for Optimizing Downlink Scheduling in an Energy-Limited Vehicular Network , 2017, IEEE Transactions on Vehicular Technology.

[75]  Geoffrey Ye Li,et al.  Deep Reinforcement Learning for Resource Allocation in V2V Communications , 2017, 2018 IEEE International Conference on Communications (ICC).

[76]  Fredrik Tufvesson,et al.  Delay and Doppler Spreads of Nonstationary Vehicular Channels for Safety-Relevant Scenarios , 2013, IEEE Transactions on Vehicular Technology.

[77]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[78]  Geoffrey Ye Li,et al.  Vehicular Communications: A Network Layer Perspective , 2017, IEEE Transactions on Vehicular Technology.

[79]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[80]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[81]  Lin Yao,et al.  V2X Routing in a VANET Based on the Hidden Markov Model , 2018, IEEE Transactions on Intelligent Transportation Systems.

[82]  Shi Jin,et al.  Channel Estimation for Massive MIMO Using Gaussian-Mixture Bayesian Learning , 2015, IEEE Transactions on Wireless Communications.

[83]  Jason Yosinski,et al.  Deep neural networks are easily fooled: High confidence predictions for unrecognizable images , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[84]  Robert W. Heath,et al.  MmWave Beam Prediction with Situational Awareness: A Machine Learning Approach , 2018, 2018 IEEE 19th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC).

[85]  Yee Whye Teh,et al.  Dirichlet Process , 2017, Encyclopedia of Machine Learning and Data Mining.

[86]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[87]  Robert W. Heath,et al.  An Overview of Signal Processing Techniques for Millimeter Wave MIMO Systems , 2015, IEEE Journal of Selected Topics in Signal Processing.

[88]  Ye Li,et al.  Bounds on the interchannel interference of OFDM in time-varying impairments , 2001, IEEE Trans. Commun..

[89]  Wenchao Xu,et al.  Internet of vehicles in big data era , 2018, IEEE/CAA Journal of Automatica Sinica.

[90]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[91]  Shimon Whiteson,et al.  Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning , 2017, ICML.

[92]  Mate Boban,et al.  Vehicular Communications: Survey and Challenges of Channel and Propagation Models , 2015, IEEE Vehicular Technology Magazine.