An Approach to Fuse Correlation-Based and Gradient-Based Methods for Image-Flow Estimation

Visual motion is commonly extracted from an image sequence in the form of an image-flow field or an image-displacement field. In the past research on estimation of motion-fields, three basic approaches have been suggested: correlation-based approach, gradient-based approach and spatiotemporal energy based approach. Since the underlying measurements used by the three approaches are different, they have different error characteristics. This scenario is representative of the classic multi-sensor problem. Algorithms based on the three basic approaches can be thought of as three different sensors measuring a given quantity, i.e., image-flow, with different error characteristics. The measurements from different sensors can be combined to produce an estimate of image-flow that is optimal, i.e., it minimizes the estimation-error (in a statistical sense). In other words, the three basic approaches can be fused to give an estimate of image-flow that has a higher confidence as compared to the estimate obtained from any one approach alone. We suggest information-fusion as a framework to estimate image-flow. In this framework, multiple sources give their opinion about image-flow in the form an estimate along with a confidence measure. These estimates are then fused on the basis of the corresponding confidence measures to get a robust estimate. We show an implementation of this framework that fuses correlation-based and gradient-based approaches.