Filter optimization of low out-of-subband emission for universal-filtered multicarrier systems

In this paper, a filter optimization method for suppressing out-of-subband (OOSB) emission in universal-filtered multicarrier (UFMC) systems is proposed. UFMC is an attractive waveform candidate for 5G mobile communication systems because it has both the advantages of OFDM and filter bank multicarrier (FBMC). Typically, UFMC employs transmit filtering on a subband of a particular set of subcarriers so that the OOSB emission can be suppressed. However, the variation of frequency response in the subband caused by filtering could be harmful to the performance of data reception at the receiver. Therefore, this paper presents a filter optimization method that suppresses the OOSB emission with constraints on the subband frequency response. With these constraints the filter design problem can be formulated as an optimization problem with nonconvex constraints. By an appropriate transformation, this problem is relaxed as a linear-constrained convex optimization problem, which can be solved efficiently by using well-known interior-point methods. Simulation results demonstrate that the filter designed by the proposed method has an advantage in bit error probability compared to the filter used in the existing UFMC literature.