QAM classification methods by SVM machine learning for improved optical interconnection

Abstract High-order quadrature amplitude modulation (QAM) formats are very effective for increasing the transmission capacity due to the highly increased spectral efficiency. However, the signal-to-noise-ratio (SNR) hungry and dense constellation of QAM make it very sensitive to nonlinear distortion. The nonlinear decision boundary adaptively generated by machine learning method of support vector machine (SVM) can be effectively used for the classification of the symbols. The different classification methods have different performance in terms of classification complexity. We experimentally investigated five SVM multi-classification methods for machine learning assisted adaptive nonlinear mitigation, including the one versus rest (OvR), the symbol encoding (SE), the binary encoding (BE), the constellation rows and columns (RC), and the in-phase and quadrature components (IQC). The comprehensive results with comparisons are demonstrated, indicating significant nonlinear mitigation with BER reductions. The SVM multi-classifier based on the in-phase and quadrature components is relatively optimal, considering the calculation and storage.