A Phenomenological Approach to Thermal and Visual Sensor Fusion

A new computer vision technique is developed which is based on analyzing different modalities of imaging, simultaneously. Such an approach is termed multisensory computer vision. This talk will describe a system which integrates information from thermal (infra-red) imagery and visual imagery to classify objects in outdoor scenes. Integration is synergistic in that it makes available new information, in this case — estimates of surface heat fluxes, which cannot be obtained by processing thermal and visual imagery separately. The approach establishes a quantitative measure of the imaged object’s relative ability to sink or source heat radiation, and a way of categorizing the object based on this property. Information integration is implemented at different levels of abstraction in the interpretation hierarchy i.e., at the pixel and at the symbolic levels. Heuristic rules are employed in a decision tree classifier to categorize imaged objects as being either vegetation, building, pavement or a vehicle.