End-to-End Learning-based Two-Way AF Relay Networks with I/Q Imbalance

In this work, we design an end-to-end (E2E) learning-based two-way amplify-and-forward (TWAF) relay network in the presence of I/Q imbalance (IQI) by maximizing the generalized mutual information between the input-output bits. The proposed system employs neural network (NN)-based terminal nodes impacted by the IQI and a TWAF relay node that only retransmits the amplified received signals (no additional processing at the relay nodes). Also, we propose four specific lambda layers at the NN decoders to pre-process the received signals. In particular, we propose to design coded-modulation and decoded-demodulation for the NN encoders and NN decoders of both the terminal nodes jointly, to tackle the interference of simultaneously received signals at the TWAF relay node and remove the deteriorating impacts of IQI at the terminal nodes. The simulation results show that the proposed E2E learning framework outperforms the maximum likelihood detector with no IQI by at least 3 dB.