Machine learning for network automation: overview, architecture, and applications [Invited Tutorial]

Networks are complex interacting systems involving cloud operations, core and metro transport, and mobile connectivity all the way to video streaming and similar user applications.With localized and highly engineered operational tools, it is typical of these networks to take days to weeks for any changes, upgrades, or service deployments to take effect. Machine learning, a sub-domain of artificial intelligence, is highly suitable for complex system representation. In this tutorial paper, we review several machine learning concepts tailored to the optical networking industry and discuss algorithm choices, data and model management strategies, and integration into existing network control and management tools. We then describe four networking case studies in detail, covering predictive maintenance, virtual network topology management, capacity optimization, and optical spectral analysis.

[1]  Luis Velasco,et al.  Provisioning, Recovery, and In-Operation Planning in Elastic Optical Networks , 2017 .

[2]  Francesco Musumeci,et al.  Machine-Learning-Based Soft-Failure Detection and Identification in Optical Networks , 2018, 2018 Optical Fiber Communications Conference and Exposition (OFC).

[3]  Achim Autenrieth,et al.  The Role of SDN in Application Centric IP and Optical Networks , 2016 .

[4]  Mohit Chamania,et al.  Artificial Intelligence (AI) Methods in Optical Networks: A Comprehensive Survey , 2018, Opt. Switch. Netw..

[5]  Marc Ruiz,et al.  Autonomic disaggregated multilayer networking , 2018, IEEE/OSA Journal of Optical Communications and Networking.

[6]  Achim Autenrieth,et al.  TSDN-Enabled Network Assurance: A Cognitive Fault Detection Architecture , 2017, 2017 European Conference on Optical Communication (ECOC).

[7]  Piero Castoldi,et al.  BER Degradation Detection and Failure Identification in Elastic Optical Networks , 2017, Journal of Lightwave Technology.

[8]  Francesco Musumeci,et al.  A Survey on Application of Machine Learning Techniques in Optical Networks , 2018, ArXiv.

[9]  Victor Lopez,et al.  Virtual network topology adaptability based on data analytics for traffic prediction , 2017, IEEE/OSA Journal of Optical Communications and Networking.

[10]  James Won-Ki Hong,et al.  SNMP-based enterprise IP network topology discovery , 2011, Int. J. Netw. Manag..

[11]  Idelfonso Tafur Monroy,et al.  Application of Machine Learning Techniques for Amplitude and Phase Noise Characterization , 2015, Journal of Lightwave Technology.

[12]  Ricard Vilalta,et al.  Control, Management, and Orchestration of Optical Networks: Evolution, Trends, and Challenges , 2018, Journal of Lightwave Technology.

[13]  S.G. Petridou,et al.  On the Use of Clustering Algorithms for Message Scheduling in WDM Star Networks , 2008, Journal of Lightwave Technology.

[14]  AKHIL GUPTA,et al.  A Survey of 5G Network: Architecture and Emerging Technologies , 2015, IEEE Access.

[15]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[16]  Aleksander Slominski,et al.  A Graph-Based Data Model for API Ecosystem Insights , 2014, 2014 IEEE International Conference on Web Services.

[17]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[18]  Maxim Kuschnerov,et al.  On the Mitigation of Optical Filtering Penalties Originating From ROADM Cascade , 2014, IEEE Photonics Technology Letters.

[19]  Luca Barletta,et al.  QoT estimation for unestablished lighpaths using machine learning , 2017, 2017 Optical Fiber Communications Conference and Exhibition (OFC).

[20]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[21]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[22]  S. J. B. Yoo,et al.  Soft failure localization during commissioning testing and lightpath operation , 2018, IEEE/OSA Journal of Optical Communications and Networking.

[23]  Takeshi Hoshida,et al.  Data-analytics-based Optical Performance Monitoring Technique for Optical Transport Networks , 2018, 2018 Optical Fiber Communications Conference and Exposition (OFC).

[24]  Marco Ruffini,et al.  An Overview on Application of Machine Learning Techniques in Optical Networks , 2018, IEEE Communications Surveys & Tutorials.

[25]  P. Poggiolini.  The GN Model of Non-Linear Propagation in Uncompensated Coherent Optical Systems , 2012, Journal of Lightwave Technology.

[26]  Achim Autenrieth,et al.  Cognitive Assurance Architecture for Optical Network Fault Management , 2018, Journal of Lightwave Technology.

[27]  Luis Miguel Contreras Murillo,et al.  A service-oriented hybrid access network and clouds architecture , 2015, IEEE Communications Magazine.

[28]  Stephen R. Marsland,et al.  Machine Learning - An Algorithmic Perspective , 2009, Chapman and Hall / CRC machine learning and pattern recognition series.

[29]  F. E. Grubbs.  Sample Criteria for Testing Outlying Observations , 1950 .

[30]  Sotiris B. Kotsiantis,et al.  Supervised Machine Learning: A Review of Classification Techniques , 2007, Informatica.

[31]  Achim Autenrieth,et al.  Analytics-Driven Fault Discovery and Diagnosis for Cognitive Root Cause Analysis , 2018, 2018 Optical Fiber Communications Conference and Exposition (OFC).

[32]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[33]  Danish Rafique.  Machine Learning Based Optimal Modulation Format Prediction for Physical Layer Network Planning , 2018, 2018 20th International Conference on Transparent Optical Networks (ICTON).

[34]  Luis Gravano,et al.  k-Shape: Efficient and Accurate Clustering of Time Series , 2015, SIGMOD Conference.

[35]  Keren Bergman,et al.  Dynamic power pre-adjustments with machine learning that mitigate EDFA excursions during defragmentation , 2017, 2017 Optical Fiber Communications Conference and Exhibition (OFC).

[36]  Danish Rafique,et al.  Nonlinear and ROADM induced penalties in 28 Gbaud dynamic optical mesh networks employing electronic signal processing. , 2011, Optics express.

[37]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[38]  Sandrine Vaton,et al.  A tutorial on the EM algorithm for Bayesian networks: Application to self-diagnosis of GPON-FTTH networks , 2016, 2016 International Wireless Communications and Mobile Computing Conference (IWCMC).

[39]  Danish Rafique,et al.  Enabling 64Gbaud coherent optical transceivers , 2017, 2017 Optical Fiber Communications Conference and Exhibition (OFC).

[40]  C. Siva Ram Murthy,et al.  A Reinforcement Learning Framework for Path Selection and Wavelength Selection in Optical Burst Switched Networks , 2007, IEEE Journal on Selected Areas in Communications.