Coupled, multi-resolution stereo and motion analysis

We propose a novel approach for fusing multi-resolution stereo and motion (both rigid and non-rigid) analysis in order to complement each other's performance. A hierarchical frame-work is presented to couple motion correspondences and stereo correspondences in order to generate accurate disparity map and motion parameters. One scenario for such system is the analysis of time-varying multi-spectral observations of clouds from meteorological satellites. Our experiments involve such time-varying remote sensing stereo data sets, and the motion is typically non-rigid as the clouds undergo shape changes. Rigid motion matching may still be performed for initial fusion, and gradually raised to non-rigid motion matching as in a coarse-to-fine strategy. Both stereo disparities and motion correspondences are estimated using such multi-resolution coarse-to-fine strategy to a sub-pixel accuracy. Experimental results using time-varying data of visible channel from two satellites in geosynchronous orbit is presented for the Hurricane Frederic.