The Role of Theory in the Evaluation of Image Motion Algorithms

Undeniably, the numerical evaluation of Computer Vision algorithms is of utmost importance. However, often neglected is the role of theoretical knowledge to interpret the numerical performance of those algorithms. In addition, the lack of theoretical research in Computer Vision has long been recognized. In this contribution, we demonstrate that extended theoretical knowledge of a phenomenon enables one to design algorithms that are better suited for the task at hand and to evaluate the theoretical assumptions of other, similar algorithms. For instance, the problem posed by multiple image motions was poorly understood in the frequency domain yet frequency-based multiple motions algorithms were developed. We present algorithms for computing multiple image motions arising from occlusion and translucency which are capable of extracting the information-content of occlusion boundaries and distinguish between those and additive translucency phenomena. These algorithms are based on recent theoretical results on occlusion in the frequency domain and demonstrate that a complete theoretical understanding of a phenomenon is required in order to design adequate algorithms. We conclude by proposing an evaluation protocol which includes theoretical considerations and their influence on the numerical evaluation of algorithms.