Analysis of cost functions and structured light patterns for modern dynamic programming stereo algorithms

Stereo vision has been an active research field that has produced a myriad of excellent algorithms. Unfortunately most of the algorithms that produce superior results rely on non-linear optimisation techniques that are computationally expensive and therefore not practical for real-time applications such as tele-immersion. Our target application is to reconstruct a 3D model of a user's head in real time at sufficient detail too allow communication between users while enabling motion parallax and therefore realism. This paper will examine dynamic programming (DP) stereo algorithms that operate in the space-time domain. Specifically we will look at the trade-offs in using different cost functions and different structured light patterns. We will compare the recent DP algorithms of Criminisi et al. (2003) and Birchfield et al. (1998). We have extended both to work in the space-time domain, to use different cost functions and to work in conjunction with structured light patterns. The performance of both algorithms will be examined using synthetic images, the impact of different cost functions with regards to each algorithm and the type of structured light pattern will be evaluated as well as the impact of extending the support region into the time domain. These results will be presented and discussed within the context of the target application.