Relay Antenna Selection in MIMO Two-Way Relay Networks Over Nakagami- $m$ Fading Channels

In this paper, we propose two relay antenna selection (RAS) schemes in multiple-input-multiple-output (MIMO) two-way relay networks, where an NA-antenna source and an NB-antenna source exchange information via an NR-antenna amplify-and-forward (AF) relay. The proposed RAS schemes maximize the minimum of the two overall end-to-end signal-to-noise ratio (SNR) values. In the first RAS scheme, transmit and receive beamforming (BF) is adopted at the two sources using all antennas. In the second RAS scheme, transmit and receive antenna selection (AS) is adopted at the two sources using one antenna. To conduct a performance comparison of the proposed RAS-BF and RAS-AS schemes, we first derive new expressions for the exact outage probability over independent but not necessarily identically distributed Nakagami-m fading channels. We then derive new closed-form lower and upper bounds on the outage probability to offer an efficient computation. We further derive new compact expressions for the asymptotic outage probability to characterize the performance in the high-SNR regime. Notably, we demonstrate that RAS-BF and RAS-AS achieve the full outage diversity order of min{NANRmA, NBNRmB}, where mA and mB denote the Nakagami-m fading parameters between the NA-antenna source and relay and between the NB-antenna source and relay, respectively. Based on asymptotic results, we determine a concise solution for the optimal power allocation among the sources and the relay to maximize the performance under the assumption that both the sources have the same transmit power. To quantify the effect of spatial correlation at the sources, we derive new closed-form expressions for the asymptotic outage probability over correlated Nakagami-m fading channels. The results indicate that the spatial correlation has no impact on the outage diversity order but significantly deteriorates the outage array gain.