Video Segmentation and Shot Boundary Detection Using Self-Organizing Maps

We present a video shot boundary detection (SBD) algorithm that spots discontinuities in visual stream by monitoring video frame trajectories on Self-Organizing Maps (SOMs). The SOM mapping compensates for the probability density differences in the feature space, and consequently distances between SOM coordinates are more informative than distances between plain feature vectors. The proposed method compares two sliding best-matching unit windows instead of just measuring distances between two trajectory points, which increases the robustness of the detector. This can be seen as a variant of the adaptive threshold SBD methods. Furthermore, the robustness is increased by using a committee machine of multiple SOM-based detectors. Experimental evaluation made by NIST in the TRECVID evaluation confirms that the SOM-based SBD method works comparatively well in news video segmentation, especially in gradual transition detection.