FBMC Prototype Filter Design via Convex Optimization

In this paper, we propose a prototype filter design for Filter Bank MultiCarrier (FBMC) systems based on convex optimization, aiming superior spectrum features while maintaining a high symbol reconstruction quality. Initially, the proposed design is written as a non-convex quadratically constrained quadratic programming (QCQP), which is relaxed into a convex QCQP guided by a line search. Through the resulting problem, we design three prototype filters: Type-I, Type-II, and Type-III. In particular, the Type-II filter shows a slightly better performance than the classical Mirabasi-Martin design, whereas Type-I and Type-III filters offer a much more contained spectrum than most of the prototype filters suitable for FBMC applications. Furthermore, numerical results corroborate the effectiveness of the designed filters as the proposed filters offer fast decay and contained spectrum while not jeopardizing symbol reconstruction in practice.