Sphere decoder with box optimisation for faster-than-Nyquist non-orthogonal frequency division multiplexing

In 1975, J. E. Mazo showed the potential faster-than-Nyquist (FTN) gain of the single-carrier binary signal. If the intersymbol interference is eliminated by an optimal detector, FTN single-carrier binary signal can transmit 24.7% more bits than the Nyquist signal without loss of bit error rate performance, which is known as the Mazo limit. In this study, the authors apply sphere decoder (SD) with box optimisation (BO) to reduce inter-carrier interference (ICI) in the FTN non-orthogonal frequency division multiplexing (FTN-NOFDM) system. Compared with the commonly used SD algorithm, SD with BO can achieve the same ICI cancellation performance, and the computational complexity is significantly reduced. When the bandwidth compression factor α is set to 0.802, the transmission rate of FTN-NOFDM is 24.7% faster than the Nyquist rate, and quadrature phase shift keying (QPSK)-modulated FTN-NOFDM has almost the same performance as QPSK-modulated orthogonal frequency division multiplexing (OFDM), which agrees well with Mazo limit. In the simulation, the QPSK-modulated FTN-NOFDM with α = 0.5 outperforms 16-quadrature amplitude modulation (16-QAM)-modulated OFDM by about 1.5 dB, and the 16-QAM-modulated FTN-NOFDM with α = 0.67 and 0.5 outperforms 64and 256-QAM-modulated OFDM by about 1.5 and 2 dB, respectively.