A scale selection principle for estimating image deformations

A basic functionality of a vision system concerns the ability to compute deformation elds between dierent images of the same physical structure. This article advocates the need for incorporating an explicit mechanism for scale selection in this context, in algorithms for computing descriptors such as optic flow and for performing stereo matching. A basic reason why such a mechanism is essential is the fact that in a coarse-to-ne propagation of disparity or flow information, it is not necessarily the case that the most accurate estimates are obtained at the nest scales. The existence of interfering structures at ne scales may make it impossible to accurately match the image data at ne scales. A systematic methodology for approaching this problem is proposed, by estimating the uncertainty in the computed flow estimate at each scale, and then selecting deformation estimates from the scales that minimize the (suitably normalized) uncertainty over scales. A specic implementation of this idea is presented for a region based dierential flow estimation scheme. It is shown that the integrated scale selection and flow estimation algorithm has the qualitative properties of leading to the selection of coarser scales for larger size image structures and increasing noise level, whereas it leads to the selection of ner scales in the neighbourhood of flow eld discontinuities. The latter property may serve as an indicator when detecting flow eld discontinuities and occlusions. I would like to thank D. Betsis and G. Orban for valuable discussions as well as J. Koenderink and A. van Doorn for providing the torso image in gure 8. This work was partially performed under the Esprit-BRA project InSight and the Esprit-NSF collaboration Diusion. The support from the Swedish Research Council for Engineering Sciences, TFR, is gratefully acknowledged.