The Cameraman Operating My Virtual Camera is Artificial

In this article, we argue that the energy spent in designing autonomous camera control systems is not spent in vain. We present a real-time virtual camera system that can create smooth camera motion. Similar systems are frequently benchmarked with the human operator as the best possible reference; however, we avoid a priori assumptions in our evaluations. Our main question is simply whether we can design algorithms to steer a virtual camera that can compete with the user experience for recordings from an expert operator with several years of experience? In this respect, we present two low-complexity servoing methods that are explored in two user studies. The results from the user studies give a promising answer to the question pursued. Furthermore, all components of the system meet the real-time requirements on commodity hardware. The growing capabilities of both hardware and network in mobile devices give us hope that this system can be deployed to mobile users in the near future. Moreover, the design of the presented system takes into account that services to concurrent users must be supported.

[1]  Wei Xu,et al.  Panoramic video stitching from commodity HDTV cameras , 2013, Multimedia Systems.

[2]  Murat Kunt,et al.  High-Quality Visual Experience , 2010 .

[3]  Peter Eades,et al.  Automatic Camera Path Generation for Graph Navigation in 3D , 2005, APVIS.

[4]  Ming Xu,et al.  Multi-camera video surveillance for real-time analysis and reconstruction of soccer games , 2010, Machine Vision and Applications.

[5]  Ming Xu,et al.  Architecture and algorithms for tracking football players with multiple cameras , 2005 .

[6]  Oliver Grau,et al.  Learning models of camera control for imitation in football matches , 2007 .

[7]  J. Starck,et al.  A Robust Free-Viewpoint Video System for Sport Scenes , 2007, 2007 3DTV Conference.

[8]  Peter Carr,et al.  Hybrid robotic/virtual pan-tilt-zom cameras for autonomous event recording , 2013, ACM Multimedia.

[9]  Sanjit K. Mitra,et al.  Detectability and Annoyance of Synthetic Blocky, Blurry, Noisy, and Ringing Artifacts , 2007, IEEE Transactions on Signal Processing.

[10]  Ragnhild Eg,et al.  Flicker effects in adaptive video streaming to handheld devices , 2011, ACM Multimedia.

[11]  Touradj Ebrahimi,et al.  Paired comparison-based subjective quality assessment of stereoscopic images , 2013, Multimedia Tools and Applications.

[12]  Hannes Fassold,et al.  Real-time Person Tracking in High-resolution Panoramic Video for Automated Broadcast Production , 2011, 2011 Conference for Visual Media Production.

[13]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[14]  J. Elder,et al.  Loosely-Coupled Telepresence Through the Panoramic Image Server , 1998 .

[15]  Pål Halvorsen,et al.  Efficient implementation and processing of a real-time panorama video pipeline with emphasis on color correction , 2013 .

[16]  Yaser Sheikh,et al.  One-man-band: a touch screen interface for producing live multi-camera sports broadcasts , 2013, MM '13.

[17]  Qi Tian,et al.  Trajectory-based ball detection and tracking with applications to semantic analysis of broadcast soccer video , 2003, MULTIMEDIA '03.

[18]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[19]  Marcus A. Magnor,et al.  Virtual video camera: image-based viewpoint navigation through space and time , 2009, SIGGRAPH '09.

[20]  Peter Carr,et al.  Portable Multi-megapixel Camera with Real-Time Recording and Playback , 2009, 2009 Digital Image Computing: Techniques and Applications.

[21]  Christophe De Vleeschouwer,et al.  Personalized production of basketball videos from multi-sensored data under limited display resolution , 2010, Comput. Vis. Image Underst..

[22]  Philippe Bekaert,et al.  Free viewpoint video for soccer using histogram-based validity maps in plane sweeping , 2014, 2014 International Conference on Computer Vision Theory and Applications (VISAPP).

[23]  Francesca De Simone,et al.  Impact of video transcoding artifacts on the subjective quality , 2010, 2010 Second International Workshop on Quality of Multimedia Experience (QoMEX).

[24]  Patrick Olivier,et al.  Virtual Camera Planning: A Survey , 2005, Smart Graphics.

[25]  Peter Eisert,et al.  Creation of High-Resolution Video Panoramas of Sport Events , 2006, Eighth IEEE International Symposium on Multimedia (ISM'06).

[26]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[27]  Bernd Girod,et al.  Video Streaming with Interactive Pan/Tilt/Zoom , 2010 .

[28]  B. S. Manjunath,et al.  Region of interest extraction and virtual camera control based on panoramic video capturing , 2005, IEEE Transactions on Multimedia.

[29]  Carsten Griwodz,et al.  Bagadus: an integrated system for arena sports analytics: a soccer case study , 2013, MMSys.

[30]  Klara Nahrstedt,et al.  Quality of experience in distributed interactive multimedia environments: toward a theoretical framework , 2009, ACM Multimedia.

[31]  Pierre Gurdjos,et al.  Interactive Zoom and Panning from Live Panoramic Video , 2014, NOSSDAV 2014.

[32]  Hironobu Fujiyoshi,et al.  Virtual camerawork for generating lecture video from high resolution images , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[33]  Jitendra Malik,et al.  Modeling and Rendering Architecture from Photographs: A hybrid geometry- and image-based approach , 1996, SIGGRAPH.

[34]  Oliver Grau,et al.  A combined studio production system for 3-D capturing of live action and immersive actor feedback , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[35]  Chng Eng Siong,et al.  Automatic replay generation for soccer video broadcasting , 2004, MULTIMEDIA '04.

[36]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[37]  Chin-Laung Lei,et al.  A crowdsourceable QoE evaluation framework for multimedia content , 2009, ACM Multimedia.

[38]  Yasuo Ariki,et al.  Automatic Production System of Soccer Sports Video by Digital Camera Work Based on Situation Recognition , 2006, Eighth IEEE International Symposium on Multimedia (ISM'06).

[39]  Carsten Griwodz,et al.  Automatic exposure for panoramic systems in uncontrolled lighting conditions: a football stadium case study , 2014, Electronic Imaging.

[40]  Xavier Armangué,et al.  Virtual Camera Synthesis for Soccer Game Replays , 2010, 2010 Conference on Visual Media Production.

[41]  Shenchang Eric Chen,et al.  QuickTime VR: an image-based approach to virtual environment navigation , 1995, SIGGRAPH.