Understanding of surface reflections in computer vision by color and multiple views

This thesis addresses problems and presents models for the detection and separation of specularities from Lambertian reflections using color and multiple images with different viewing directions. From the models, three algorithms are proposed and experimental results are presented. The first algorithm uses only color information for the separation of diffuse as well as sharp specularities and inter-reflections from Lambertian reflections through image segmentation. A computational model based on the dichromatic model is presented for interpretation of various surface reflections in a spectral space with three orthogonal basis functions. The established model is used for arranging color data for segmentation and separation. Applicable objects and illumination for the algorithm are limited to uniformly colored dielectrics under singly colored scene illumination. Use of multiple views for understanding reflection properties is proposed with the second and the third algorithms called spectral differencing and view sampling, respectively. Both use multiple views in different viewing directions, and are based on the Lambertian consistency that image irradiance from Lambertian reflection does not vary depending on viewing directions while image irradiance from specular reflection does. Spectral differencing is a detection algorithm that detects specularities by color difference between two images without any geometrical correspondence. The object domain for detection is extended to nonuniformly colored dielectrics and metals under multiply colored scene illumination. As a detection algorithm, spectral differencing can be used with the image segmentation algorithm for more stable performance. With densely sampled views in wide angle and with known viewing directions, the view sampling algorithm reconstructs object structure as well as separates specularities from Lambertian reflections. The view sampling algorithm does not require color information, and is applicable to dielectrics and metals. Experimental results agree well with the models and algorithms within the limitations discussed.