On the statistical efficiency of LMS algorithms

Summary form only given. This paper describes the statistical efficiency of LMS algorithms. In this work, two gradient descent adaptive algorithms are compared, the LMS algorithm and the LMS/Newton algorithm. LMS is simple and practical and is used in many applications worldwide. LMS/Newton is based on Newton's method and the LMS algorithm. LMS/Newton is optimal in the least squares sense. It maximizes the quality of its adaptive solution while minimizing the use of training data. Many least squares adaptive algorithms have been devised over the years, but no other least squares algorithm can give better performance, on average, than LMS/Newton. Furthermore, LMS algorithm is related to the famous backpropagation algorithm used for training neural networks.