Defects in the optical lens directly affect the scattering properties of the optical lens and decrease the performance of the optical element. Although machine vision instead of manual detection has been widely valued, the feature fusion technique of series operation and edge detection cannot recognize low-contrast and multi-scale targets in the lens. To address these challenges, in this study, an improved YOLOv5-C3CA-SPPF network model is proposed to detect defects on the surface and inside of the lens. The hybrid module combining the coordinate attention and CSPNet (C3) is incorporated into YOLOv5-C3CA for improving the extraction of target feature information and detection accuracy. Furthermore, an SPPF features fusion module is inserted into the neck of the network model to improve the detection accuracy of the network. To enhance the performance of supervised learning algorithms, a dataset containing a total of 3800 images is created, more than 600 images for each type of defect samples. The outcome of the experiment manifests that the mean average precision (mAP) of the YOLOv5-C3CA-SPPF algorithm is 97.1%, and the detection speed FPS is 41 f/s. Contrast to the traditional lens surface defects detection algorithms, YOLOv5-C3CA-SPPF can detect the types of optical lens surface and inside defects more accurately and quickly, the experimental results show that the YOLOv5-C3CA-SPPF model for identifying optical lens defects has good generalizability and robustness, which is favorable for on-line quality automatic detection of optical lens defects and provide an important guarantee for the quality consistency of finished products.