Handling transparency in digital video

This thesis is concerned with handling transparency in digital video. We define transparency as a mixture between two different distinctive layers. This thesis has three main contributions. The first contribution is an approach for removing two common types of degradations on Digital Video known as ‘Blotches’ and ‘Line Scratches’. Blotches are impulsive dark and bright spots distributed randomly over an image sequence. Line scratches are temporally consistent dark and bright vertical lines propagating through an image sequence. Current removal techniques model such corruptions as an opaque foreground layer superimposed on the original (background) ‘clean’ layer. This often generates restoration artifacts due to underestimation and/or overestimation of the corruption region. We propose a removal technique which models corruptions as a semi-transparent layer superimposed on the original ‘clean’ layer. Removal is then achieved by estimating corruption opacity and the underlaying original data. We generate a solution using a Bayesian framework and we use novel spatial and temporal priors. Restoration results are compared against ground truth estimates. Ground truth estimates are derived from IR scans of corruptions. Restoration results show that our restoration technique generates more accurate estimation of the corruption borders over previous work. This generates better removal despite texture and motion complexity. The second contribution of this thesis is a technique for automated detection of reflections in image sequences. Regions of reflections are common in video and they are often the result of superimposing a semi-transparent foreground layer over a background layer. This phenomenon causes many image processing techniques to fail as they assume the presence of one layer at each examined site e.g. motion estimation and object recognition. This calls for the need of an automated technique which detects such regions and assigns different treatments to them. However, as reflections can result by mixing any two images, they can come in different forms and colors. This makes their detection a hard problem that was not addressed before. We propose a technique for automated detection of reflections by analyzing feature point trajectories. We examine several spatial and temporal features. This generates a set of weak detectors. A strong detector is generated by combining the weak detectors. We generate a solution using a Machine Learning framework, impose spatial and temporal smoothness on the generated masks and our results show high reflection detection rate with rejection to regions of complicated motion. The third contribution of this thesis is a technique for multiple motion estimation in regions of reflections. As such regions have two layers superimposed over each other, they usually have two motions per pel, one for each layer. Most motion estimators assume the presence of one motion per examined site, an assumption that is violated in reflections. Current multiple motion estimators assume constant motion over at least three frames. As a result they can not handle non-uniform motions as ones arising due to camera shake or motion acceleration. We present an approach for multiple motion estimation based on the observation that the motion for a specific