Revisiting Error Analysis in Convolutionally Coded Systems: The Irregular Constellation Case

There has been a rejuvenated interest in the use of non-equally spaced (irregular) constellations after promising performance results were demonstrated compared with the conventional lattice constellations. This paper investigates the use of the irregular constellations in convolutionally coded systems. To enable the use of such irregular constellations in the presence of convolutional encoders, we derive a bit-error-rate (BER) upper bound expression for the downlink of the coded transmit maximum ratio combining systems operating in Nakagami- ${m}$ fading environments. In addition, the analysis is extended to turbo trellis-coded modulation scenarios in which the convolutional encoders are used as the constituent codes. We demonstrate, via simulation, that commonly used performance analysis techniques in the literature fail to provide a valid BER bound in coded cases where quasi-regularity is not satisfied. In contrast, the technique proposed herein does not require the chosen pair of constellation and encoder to be quasi-regular. Furthermore, the system model includes a provision for multiple orthogonal transmission stages with different numbers of transmit antennas. Antenna correlation as well as distributed transmission is also supported by the model. Simulation results demonstrate the accuracy of the derived analytical results for a wide range of system scenarios.