Detecting shot boundary with sparse coding for video summarization

Abstract Keyframe selection is a common way to summarize video contents. However, delimiting shot boundaries to extract a representative keyframe from each shot is not trivial as most shot boundary techniques are heuristic and sensitive to the types of video transitions. This paper proposes a new shot boundary detection algorithm, that learns a dictionary from the given video using sparse coding and updates atoms in the dictionary, following the philosophy that different shots cannot be reconstructed using the learned dictionary. Technically, our algorithm conducts the learning by simultaneously minimizing the reconstruction loss, restricting the sparsity of the reconstruction matrix, and preserving the structure across patches and frames. Once shot boundaries are determined, one representative keyframe is selected from each shot and then a video summary is constructed by concatenating the representative keyframes through a post process. On two standard video datasets across various genres, i.e., VSUMM and YouTube datasets, our method is shown to be powerful for video summarization with superior performance over several state-of-the-art techniques.