Multi-video synopsis for video representation

The world is covered with millions of cameras with each recording a huge amount of video. It is a time-consuming task to watch these videos, as most of them are of little interest due to the lack of activity. Video representation is thus an important technology to tackle with this issue. However, conventional video representation methods mainly focus on a single video, aiming at reducing the spatiotemporal redundancy as much as possible. In contrast, this paper describes a novel approach to present the dynamics of multiple videos simultaneously, aiming at a less intrusive viewing experience. Given a main video and multiple supplementary videos, the proposed approach automatically constructs a synthesized multi-video synopsis by integrating the supplementary videos into the most suitable spatiotemporal portions within this main video. The problem of finding suitable integration between the main video and supplementary videos is formulated as the maximum a posterior (MAP) problem, in which the desired properties related to a less intrusive viewing experience, i.e., informativeness, consistency, visual naturalness, and stability, are maximized. This problem is solved by using an efficient Viterbi beam search algorithm. Furthermore, an informative blending algorithm that naturalizes the connecting boundary between different videos is proposed. The proposed method has a wide variety of applications such as visual information representation, surveillance video browsing, video summarization, and video advertising. The effectiveness of multi-video synopsis is demonstrated in extensive experiments over different types of videos with different synopsis cases.