Eccentricity error is the main error source of rotary encoder. In our previous work, a novel polar coordinate encoder has been presented for eccentricity self-detection and error compensation. In this paper, an improved eccentricity self-detection method based on least square algorithm is presented. We first establish the radial displacement equation in terms of rotary angle and eccentricity. Then, overdetermined equations can be constructed by taking sampling points into radial displacement equation, from which the optimal solution of eccentricity can be determined using least squares algorithm. Thus, angle compensation can be performed by submitting the calculated eccentricity into eccentricity error model. The proposed method offers a new strategy for automatic eccentricity self-detection and angle compensation of rotary encoder. Both simulation analysis and experimental test are performed to prove the effectiveness of the proposed method.