Scale-space with Causal Time Direction

This article presents a theory for multi-scale representation of temporal data. Assuming that a real-time vision system should represent the incoming data at di erent time scales, an additional causality constraint arises compared to traditional scale-space theory|we can only use what has occurred in the past for computing representations at coarser time scales. Based on a previously developed scale-space theory in terms of noncreation of local maxima with increasing scale, a complete classi cation is given of the scale-space kernels that satisfy this property of non-creation of structure and respect the time direction as causal . It is shown that the cases of continuous and discrete time are inherently di erent. For continuous time, there is no non-trivial time-causal semi-group structure. Hence, the time-scale parameter must be discretized, and the only way to construct a linear multi-time-scale representation is by (cascade) convolution with truncated exponential functions having (possibly) di erent time constants. For discrete time, there is a canonical semi-group structure allowing for a continuous temporal scale parameter. It gives rise to a Poisson-type temporal scale-space. In addition, geometric moving average kernels and time-delayed generalized binomial kernels satisfy temporal causality and allow for highly e cient implementations. It is shown that temporal derivatives and derivative approximations can be obtained directly as linear combinations of the temporal channels in the multi-time-scale representation. Hence, to maintain a representation of temporal derivatives at multiple time scales, there is no need for other time bu ers than the temporal channels in the multi-time-scale representation. The framework presented constitutes a useful basis for expressing a large class of algorithms for computer vision, image processing and coding.