Querying Multiple Simultaneous Video Streams with 3D Interest Maps

With proliferation of mobile devices equipped with cameras and video recording applications, it is now common to observe multiple mobile cameras filming the same scene at an event from a diverse set of view angles. These recorded videos provide a rich set of data for someone to re-experience the event at a later time. Not all the videos recorded, however, show a desirable view. Navigating through a large collection of videos to find a video with a better viewing angle can be time consuming. We propose a query-response interface in which users can intuitively switch to another video with an alternate, better, view, by selecting a 2D region within a video as a query. The system would then response with another video that has a better view of the selected region, maximizing the viewpoint entropy. The key to our system is a lightweight 3D scene structure, also termed 3D interest map. A 3D interest map is naturally an extension of saliency maps in the 3D space since most users film what they find interesting from their respective viewpoints. A user study with more than 35 users shows that our video query system achieves a suitable compromise between accuracy and run-time.