Effects of the LLL Reduction on the Success Probability of the Babai Point and on the Complexity of Sphere Decoding

A common method to estimate an unknown integer parameter vector in a linear model is to solve an integer least squares (ILS) problem. A typical approach to solving an ILS problem is sphere decoding. To make a sphere decoder faster, the well-known LLL reduction is often used as preprocessing. The Babai point produced by the Babai nearest plane algorithm is a suboptimal solution of the ILS problem. First, we prove that the success probability of the Babai point as a lower bound on the success probability of the ILS estimator is sharper than the lower bound given by Hassibi and Boyd [1]. Then, we show rigorously that applying the LLL reduction algorithm will increase the success probability of the Babai point and give some theoretical and numerical test results. We give examples to show that unlike LLL's column permutation strategy, two often used column permutation strategies SQRD and V-BLAST may decrease the success probability of the Babai point. Finally, we show rigorously that applying the LLL reduction algorithm will also reduce the computational complexity of sphere decoders, which is measured approximately by the number of nodes in the search tree in the literature.

[1]  J.-M. Goethals,et al.  IEEE international symposium on information theory , 1981 .

[2]  A. Korkine,et al.  Sur les formes quadratiques , 1873 .

[3]  R. Connelly In Handbook of Convex Geometry , 1993 .

[4]  László Babai,et al.  On Lovász’ lattice reduction and the nearest lattice point problem , 1986, Comb..

[5]  Qing Han,et al.  Solving Box-Constrained Integer Least Squares Problems , 2008, IEEE Transactions on Wireless Communications.

[6]  P. Xu Voronoi cells, probabilistic bounds, and hypothesis testing in mixed integer linear models , 2006, IEEE Transactions on Information Theory.

[7]  P. Teunissen Success probability of integer GPS ambiguity rounding and bootstrapping , 1998 .

[8]  Claus-Peter Schnorr,et al.  Lattice Basis Reduction: Improved Practical Algorithms and Solving Subset Sum Problems , 1991, FCT.

[9]  Björn E. Ottersten,et al.  On the complexity of sphere decoding in digital communications , 2005, IEEE Transactions on Signal Processing.

[10]  Wai Ho Mow,et al.  Novel Joint Sorting and Reduction Technique for Delay-Constrained LLL-Aided MIMO Detection , 2008, IEEE Signal Processing Letters.

[11]  Alexander Vardy,et al.  Closest point search in lattices , 2002, IEEE Trans. Inf. Theory.

[12]  Daniele Micciancio,et al.  The hardness of the closest vector problem with preprocessing , 2001, IEEE Trans. Inf. Theory.

[13]  László Lovász,et al.  Factoring polynomials with rational coefficients , 1982 .

[14]  Giuseppe Caire,et al.  On maximum-likelihood detection and the search for the closest lattice point , 2003, IEEE Trans. Inf. Theory.

[15]  K. Kammeyer,et al.  Efficient algorithm for decoding layered space-time codes , 2001 .

[16]  Cong Ling,et al.  Effective LLL Reduction for Lattice Decoding , 2007, 2007 IEEE International Symposium on Information Theory.

[17]  Stephen P. Boyd,et al.  Integer parameter estimation in linear models with applications to GPS , 1998, IEEE Trans. Signal Process..

[18]  Damien Stehlé,et al.  An LLL Algorithm with Quadratic Complexity , 2009, SIAM J. Comput..

[19]  Xiao-Wen Chang,et al.  Partial LLL Reduction , 2012, ArXiv.

[20]  Peter Teunissen,et al.  An invariant upperbound for the GNSS bootstrappend ambiguitysuccess-rate , 2003 .

[21]  Peter Teunissen An invariant upper bound for the GNSS bootstrapped ambiguity success-rate , 2003 .

[22]  P. Teunissen An optimality property of the integer least-squares estimator , 1999 .

[23]  Reinaldo A. Valenzuela,et al.  Simplified processing for high spectral efficiency wireless communication employing multi-element arrays , 1999, IEEE J. Sel. Areas Commun..

[24]  Björn E. Ottersten,et al.  The Error Probability of the Fixed-Complexity Sphere Decoder , 2009, IEEE Transactions on Signal Processing.

[25]  Walid Abediseid Efficient Lattice Decoders for the Linear Gaussian Vector Channel: Performance & Complexity Analysis , 2011 .

[26]  Claus-Peter Schnorr,et al.  Lattice basis reduction: Improved practical algorithms and solving subset sum problems , 1991, FCT.

[27]  R. Muirhead Aspects of Multivariate Statistical Theory , 1982, Wiley Series in Probability and Statistics.

[28]  Helmut Bölcskei,et al.  On the Complexity Distribution of Sphere Decoding , 2011, IEEE Transactions on Information Theory.