Cinematographic rules applied to a camera network

We present a camera network system consisting of several modules of 2-3 low end cameras attached to one computer. It is not possible for a human to observe all the information coming from such a network simultaneously. Our system is designed to select the best viewpoint for each part of the video sequence, thus automatically creating one real-time video stream that contains the most important data. It acts as a combination of a director and a cameraman. Cinematography developed its own terminology, techniques and rules, how to make a good movie. We illustrate here some of these techniques and how they can be applied to a camera network, to solve the best viewpoint selection problem. Our system consists of only fixed cameras, but the output is not constrained to already existing views. A virtual zoom can be applied to select only a part of the view. We propose a view interpolation algorithm which makes it possible to create new intermediate views from the existing camera images. The combination of all these techniques gathers information from a complete camera network and produces one attractive real-time video stream. The resulting video can typically be used for telepresence applications or as a documentary or instruction video.

[1]  Mubarak Shah,et al.  Human tracking in multiple cameras , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[2]  Claudio S. Pinhanez,et al.  Using approximate models as source of contextual information for vision processing , 1995 .

[3]  Luke S. Zettlemoyer,et al.  Task-sensitive cinematography interfaces for interactive 3D learning environments , 1998, IUI '98.

[4]  Joseph V. Mascelli.  The five C's of cinematography : motion picture filming techniques simplified , 1965 .

[5]  Andrew Blake,et al.  Gaze manipulation for one-to-one teleconferencing , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[6]  David Salesin,et al.  The virtual cinematographer: a paradigm for automatic real-time camera control and directing , 1996, SIGGRAPH.

[7]  Richard Szeliski,et al.  A layered approach to stereo reconstruction , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[8]  Ruigang Yang,et al.  Real‐Time Consensus‐Based Scene Reconstruction Using Commodity Graphics Hardware † , 2003, Comput. Graph. Forum.

[9]  James M. Rehg,et al.  Statistical Color Models with Application to Skin Detection , 2004, International Journal of Computer Vision.

[10]  Mohan M. Trivedi,et al.  Active Camera Networks and Semantic Event Databases for Intelligent Environments , 2002 .

[11]  Ruigang Yang,et al.  Multi-resolution real-time stereo on commodity graphics hardware , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[12]  Luc Van Gool,et al.  Monkeys -- A Software Architecture for ViRoom -- Low-Cost Multicamera System , 2003, ICVS.

[13]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[14]  Jake K. Aggarwal,et al.  Tracking Human Motion in Structured Environments Using a Distributed-Camera System , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Tomáš Svoboda.  Quick Guide to Multi-Camera Self-Calibration , 2003 .

[16]  Tal Hassner,et al.  What Does the Scene Look Like from a Scene Point? , 2002, ECCV.

[17]  Patrick Olivier,et al.  CamPlan: A Camera Planning Agent , 2000 .

[18]  S. M. Steve.  SUSAN - a new approach to low level image processing , 1997 .