Performance analysis of MIMO DF relay network with the Nth-best user selection scheme in the presence of co-channel interference

Abstract In this paper, we study the performance of multi-user multiple-input multiple-output (MIMO) relay network in the presence of co-channel interference (CCI) as well as additive white Gaussian noise (AWGN). Specifically, the Nth-best user selection scheme is applied to attain multi-user diversity. Both exact and asymptotic closed-form expressions for the outage probability and average symbol error rate (SER) of the considered system are derived, and corroborated by the Monte Carlo simulations. Based on the asymptotic SER expression, the diversity order and array gain are characterized, and the optimum power allocation between the source and the relay is also designed. The results reveal that the diversity order is NR × min {NS, ND(K − N + 1)}, where NS, NR and ND represent the number of antennas at the source, the relay, and each destination, respectively, K is the number of the destinations, and N (1 ≤ N ≤ K) represents the rank of the user chosen. The findings of this paper show that the CCI deteriorates the system performance by reducing the array gain, and optimal power allocation achieves better performance than that of uniform power allocation by offering extra array gain. These findings provide a valuable insight into the practical system design.