Pareto Optimization of Adaptive Modulation and Coding Set in Nonlinear Fiber-Optic Systems

In recent coherent optical communications, various high-order modulation formats have been used in conjunction with soft-decision forward error correction (FEC) such as capacity-approaching low-density parity-check (LDPC) codes. With a proper selection of modulation order and FEC overhead, we may be able to achieve the highest spectral efficiency (SE) for a given fiber plant configuration. However, it is often not straightforward to select the best pair of modulation format and FEC overhead due to many factors, including fiber nonlinearity, channel spacing, baud rates, communications distance, link budget, and power consumption. In this paper, we introduce a new framework to design adaptive modulation and coding (AMC) sets based on Pareto efficiency in order to optimize multiple objective functions, more specifically, to achieve higher SE, higher nonlinearity tolerance, and lower power consumption at the same time. We compare various modulation formats and variable-rate LDPC codes based on generalized mutual information (GMI) in nonlinear fiber transmissions. In order to account for the penalty of finite-iteration decoding under a constraint of power consumption, we use required GMI as a new metric for AMC design. With our AMC framework, Pareto-efficient pairs of modulation and coding can be identified to achieve both higher SE and higher nonlinearity tolerance constrained on power consumption.