Forecasting loss of signal in optical networks with machine learning

Loss of signal (LOS) represents a significant cost for operators of optical networks. By studying large sets of real-world performance monitoring data collected from six international optical networks, we find that it is possible to forecast LOS events with good precision one to seven days before they occur, albeit at relatively low recall, with supervised machine learning (ML). Our study covers 12 facility types, including 100G lines and ETH10G clients. We show that the precision for a given network improves when training on multiple networks simultaneously relative to training on an individual network. Furthermore, we show that it is possible to forecast LOS from all facility types and all networks with a single model, whereas fine-tuning for a particular facility or network brings only modest improvements. Hence our ML models remain effective for optical networks previously unknown to the model, which makes them usable for commercial applications.

[1]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[2]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[3]  Yingxiong Song,et al.  Transfer learning assisted deep neural network for OSNR estimation. , 2019, Optics express.

[4]  Weisheng Hu,et al.  Application of Machine Learning in Fiber Nonlinearity Modeling and Monitoring for Elastic Optical Networks , 2018, Journal of Lightwave Technology.

[5]  Achim Autenrieth,et al.  Cognitive Assurance Architecture for Optical Network Fault Management , 2018, Journal of Lightwave Technology.

[6]  Jacek Tabor,et al.  Processing of missing data by neural networks , 2018, NeurIPS.

[7]  Francesco Musumeci,et al.  Machine-Learning-Based Soft-Failure Detection and Identification in Optical Networks , 2018, 2018 Optical Fiber Communications Conference and Exposition (OFC).

[8]  G. Moody,et al.  Predicting in-hospital mortality of ICU patients: The PhysioNet/Computing in cardiology challenge 2012 , 2012, 2012 Computing in Cardiology.

[9]  Yan Liu,et al.  Recurrent Neural Networks for Multivariate Time Series with Missing Values , 2016, Scientific Reports.

[10]  Rui Manuel Morais,et al.  On the suitability, requisites, and challenges of machine learning [Invited] , 2020, IEEE/OSA Journal of Optical Communications and Networking.

[11]  Luca Barletta,et al.  Machine-learning method for quality of transmission prediction of unestablished lightpaths , 2018, IEEE/OSA Journal of Optical Communications and Networking.

[12]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[13]  Frank R. Kschischang,et al.  Optical Nonlinear Phase Noise Compensation for $9\times 32$ -Gbaud PolDM-16 QAM Transmission Using a Code-Aided Expectation-Maximization Algorithm , 2015, Journal of Lightwave Technology.

[14]  Massimo Tornatore,et al.  Active vs Transfer Learning Approaches for QoT Estimation with Small Training Datasets , 2020, 2020 Optical Fiber Communications Conference and Exhibition (OFC).

[15]  Wei Cao,et al.  BRITS: Bidirectional Recurrent Imputation for Time Series , 2018, NeurIPS.

[16]  Sachin Shetty,et al.  Transfer learning for detecting unknown network attacks , 2019, EURASIP Journal on Information Security.

[17]  Yongli Zhao,et al.  Dirty-data-based alarm prediction in self-optimizing large-scale optical networks. , 2019, Optics express.

[18]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[19]  Yvan Pointurier,et al.  Associating machine-learning and analytical models for quality of transmission estimation: combining the best of both worlds , 2021, IEEE/OSA Journal of Optical Communications and Networking.

[20]  S. Bigo,et al.  Toward efficient, reliable, and autonomous optical networks: the ORCHESTRA solution [Invited] , 2019, IEEE/OSA Journal of Optical Communications and Networking.

[21]  Yongli Zhao,et al.  SOON: self-optimizing optical networks with machine learning. , 2018, Optics express.

[22]  Ying Zhang,et al.  Multivariate Time Series Imputation with Generative Adversarial Networks , 2018, NeurIPS.

[23]  Bruno Lavigne,et al.  Simple self-optimization of WDM networks based on probabilistic constellation shaping [Invited] , 2020, IEEE/OSA Journal of Optical Communications and Networking.

[24]  David Cote,et al.  Using machine learning in communication networks [Invited] , 2018, IEEE/OSA Journal of Optical Communications and Networking.

[25]  Min Zhang,et al.  Failure prediction using machine learning and time series in optical network. , 2017, Optics express.

[26]  Daniel C. Kilper,et al.  Model transfer of QoT prediction in optical networks based on artificial neural networks , 2019, IEEE/OSA Journal of Optical Communications and Networking.

[27]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[28]  Ying Wang,et al.  Machine-Learning-based Alarm Prediction with GANs-based Self-Optimizing Data Augmentation in Large-Scale Optical Transport Networks , 2020, 2020 International Conference on Computing, Networking and Communications (ICNC).

[29]  Roberto Proietti,et al.  Self-Taught Anomaly Detection With Hybrid Unsupervised/Supervised Machine Learning in Optical Networks , 2019, Journal of Lightwave Technology.

[30]  Xiaoping Zheng,et al.  Routing without Routing Algorithms: An AI-Based Routing Paradigm for Multi-Domain Optical Networks , 2019, 2019 Optical Fiber Communications Conference and Exhibition (OFC).

[31]  M. Boamah.  Analysing Crisis Communication Strategies of Airline Companies in United States: A Case Study of Southwest Airline 2016 Power Outage Crisis , 2019, Studies in Media and Communication.

[32]  Takeshi Hoshida,et al.  Convolutional neural network-based optical performance monitoring for optical transport networks , 2018, IEEE/OSA Journal of Optical Communications and Networking.