Detecting and removing specularities in facial images

Specularities often confound algorithms designed to solve computer vision tasks such as image segmentation, object detection, and tracking. These tasks usually require color image segmentation to partition an image into regions, where each region corresponds to a particular material. Due to discontinuities resulting from shadows and specularities, a single material is often segmented into several sub-regions. In this paper, a specularity detection and removal technique is proposed that requires no camera calibration or other a priori information regarding the scene. The approach specifically addresses detecting and removing specularities in facial images. The image is first processed by the Luminance Multi-Scale Retinex [B.V. Funt, K. Barnard, M. Brockington, V. Cardei, Luminance-Based Multi-Scale Retinex, AIC'97, Kyoto, Japan, May 1997]. Second, potential specularities are detected and a wavefront is generated outwards from the peak of the specularity to its boundary or until a material boundary has been reached. Upon attaining the specularity boundary, the wavefront contracts inwards while coloring in the specularity until the latter no longer exists. The third step is discussed in a companion paper [M.D. Levine, J. Bhattacharyya, Removing shadows, Pattern Recognition Letters, 26 (2005) 251-265] where a method for detecting and removing shadows has also been introduced. The approach involves training Support Vector Machines to identify shadow boundaries based on their boundary properties. The latter are used to identify shadowed regions in the image and then assign to them the color of non-shadow neighbors of the same material as the shadow. Based on these three steps, we show that more meaningful color image segmentations can be achieved by compensating for illumination using the Illumination Compensation Method proposed in this paper. It is also demonstrated that the accuracy of facial skin detection improves significantly when this illumination compensation approach is used. Finally, we show how illumination compensation can increase the accuracy of face recognition. ition.

[1]  James M. Rehg,et al.  Statistical Color Models with Application to Skin Detection , 2004, International Journal of Computer Vision.

[2]  Steven A. Shafer,et al.  Using color to separate reflection components , 1985 .

[3]  Gudrun Klinker,et al.  A physical approach to color image understanding , 1989, International Journal of Computer Vision.

[4]  Shimon Ullman,et al.  Face Recognition: The Problem of Compensating for Changes in Illumination Direction , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Stan Z. Li,et al.  Local non-negative matrix factorization as a visual representation , 2002, Proceedings 2nd International Conference on Development and Learning. ICDL 2002.

[6]  Erwin Kreyszig,et al.  Advanced Engineering Mathematics, Maple Computer Guide , 2000 .

[7]  Sebastian Lang,et al.  Improving adaptive skin color segmentation by incorporating results from face detection , 2002, Proceedings. 11th IEEE International Workshop on Robot and Human Interactive Communication.

[8]  R. Bajcsy,et al.  Color image segmentation with detection of highlights and local illumination induced by inter-reflections , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[9]  E. Granum,et al.  Skin colour detection under changing lighting conditions , 1999 .

[10]  Fernando Torres Medina,et al.  Automatic Detection of Specular Reflectance in Colour Images Using the MS Diagram , 2003, CAIP.

[11]  Martin D. Levine,et al.  Removing shadows , 2005, Pattern Recognit. Lett..

[12]  Zia-ur Rahman,et al.  Properties and performance of a center/surround retinex , 1997, IEEE Trans. Image Process..

[13]  Sang Wook Lee,et al.  Image Segmentation with Detection of Highlights and Inter-Reflections Using Color , 1989 .

[14]  Rahman Zia.  Properties of a Center/Surround Retinex: Part 1. Signal Processing Design , 1995 .

[15]  Roberto Brunelli,et al.  Face Recognition: Features Versus Templates , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Zia-ur Rahman,et al.  A multiscale retinex for bridging the gap between color images and the human observation of scenes , 1997, IEEE Trans. Image Process..

[17]  Takeo Kanade,et al.  Image Segmentation And Reflection Analysis Through Color , 1988, Defense, Security, and Sensing.

[18]  Sang Wook Lee,et al.  Detection of specularity using colour and multiple views , 1992, Image Vis. Comput..

[19]  Sang Wook Lee,et al.  Color image segmentation and color constancy , 1990, Electronic Imaging.

[20]  Mika Laaksonen,et al.  Skin detection in video under changing illumination conditions , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[21]  Ian Craw,et al.  Tracking and measuring drivers' eyes , 1995, Image Vis. Comput..

[22]  Richard Szeliski,et al.  Extracting layers and analyzing their specular properties using epipolar-plane-image analysis , 2005, Comput. Vis. Image Underst..

[23]  Stephen Lin,et al.  Highlight removal by illumination-constrained inpainting , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[24]  F. W. Kellaway,et al.  Advanced Engineering Mathematics , 1969, The Mathematical Gazette.

[25]  Mehmet Celenk,et al.  A color clustering technique for image segmentation , 1990, Comput. Vis. Graph. Image Process..

[26]  Brian V. Funt,et al.  Luminance-Based Multi-Scale Retinex , 1997 .

[27]  M. Carter.  Computer graphics: Principles and practice , 1997 .

[28]  J. Birgitta Martinkauppi,et al.  Behavior of skin color under varying illumination seen by different cameras at different color spaces , 2001, IS&T/SPIE Electronic Imaging.

[29]  A. W. Glenn,et al.  Properties of a Center/Surround Retinex: Part 2. Surround Design , 1995 .

[30]  Karsten Schlüns,et al.  Photometric Stereo for Non-Lambertian Surfaces Using Color Information , 1993, CAIP.

[31]  Stephen Lin,et al.  Diffuse-Specular Separation and Depth Recovery from Image Sequences , 2002, ECCV.