Face image captured under uncontrolled illumination conditions is one of the most significant challenges for real-world human face recognition systems. To overcome this problem, we proposed a novel method called adaptive multiple-layer retinex-based color face enhancement (AMRF) to enhance the face images. Firstly, we use an associative filter to decompose a color face image into illumination and reflectance components at multiple layers. Then, the illumination components in each layer are adjusted automatically by multiplying with corresponded illumination compensation coefficients calculated through a referenced Gaussian template. The enhanced color face image is finally obtained by composing the compensated illumination components with the integrated reflectance component based on the Retinex theory. The experiment was performed on four popular color face datasets: LFW, IJB-C, CMU Multi-PIE, and CMU-PIE. Our proposed method makes face images more precise, natural, and smooth. The experiment results show that AMRF’s image quality assessment scores are significantly better than the original and other enhanced methods’ images. Furthermore, AMRF considerably improves the recognition accuracy of deep learning-based face recognition models, such as FaceNet, and ArcFace. Finally, our proposed method also saves computational time comparing the other techniques.