On-site measurements and defect detection are of great importance for precision ground steel rollers due to their large dimension and weight. In addition to dimensional error, form accuracy, surface roughness, and surface/sub-surface cracks, there also exist optical defect requirements for steel roller surfaces, e.g., speckles, chatter marks, or feed traces. Since rollers with optical defects will always duplicate the defect patterns onto the metal sheet or foil during rolling, it is necessary as well as significant to scrutinize the roller surface after grinding. In industrial practice, defects are investigated mainly by experienced engineers through naked-eye inspections along particular directions and under appropriate illumination conditions. This is usually subjective and inconsistent. In this paper, a machine vision system is developed, to add onto the roller grinder, that is capable of acquiring the roller's surface image with high and consistent quality. In addition, to identify defects with fuzzy boundaries, intensity inhomogeneity, and complex background textures, an improved segmentation algorithm is developed based on an active contour without edges model. Furthermore, qualitative and quantitative comparisons of the proposed algorithm with the Chan-Vese model, the local binary fitting model, and the globally signed region pressure force model are carried out. The comparisons prove that the proposed method performs with better accuracy and robustness for fuzzy and inhomogeneous defect segmentation and consumes generally less computational time.