Physics-Based Segmentation of Complex Objects Using Multiple Hypotheses of Image Formation

We present a general framework for the segmentation of complex scenes using multiple physical hypotheses of image formation. These hypotheses specify broad classes for the shape, illumination, and material properties of simple image regions. Through analysis, merging, and filtering of hypotheses the framework generates a ranked list of segmentations. We have implemented an algorithm based upon this framework and show example segmentations of scenes containing multicolored piece-wise uniform dielectric objects. By using this new approach we can intelligently segment scenes with objects of greater complexity than previous physics-based algorithms. The results show that by using general physical models we can obtain segmentations that correspond more closely to objects in a scene than segmentations found using only color.