Design and implementation of a cluster based smart camera array application framework

With the advance in computing, imaging and rendering technologies, applications taking advantage of multiple view dynamic scene information provided by a camera array have become more and more popular and practical. However, many such systems proposed in previous works are usually project-specific, and hence, there is a need for a general-purpose application framework to enable rapid prototyping and implementation. In this paper, we present a middleware-like framework which provides foundational components and modularized workflow services which are commonly required in different camera array applications. In particular, our framework is targeted to computer cluster based camera array applications, in which multiple computing nodes are network-connected and used to control the cameras and perform distributed computations. Software abstraction of inter-camera message/data communications is provided to enable event-driven workflow and data access location transparency, which in turn enables many computations involving multiple-camera to be parallelizable for better efficiency. In this paper, we demonstrate the effectiveness and performance of our proposed framework by illustrating two practical applications for 3D video and multiple camera based background/foreground segmentation. Our results demonstrate that the proposed framework is very promising and that it satisfies our initial requirements.

[1]  Huang Lee,et al.  Nonstationary Background Removal Via Multiple Camera Collaboration , 2007, 2007 First ACM/IEEE International Conference on Distributed Smart Cameras.

[2]  Leonard McMillan,et al.  A Real-Time Distributed Light Field Camera , 2002, Rendering Techniques.

[3]  Bernhard Rinner,et al.  Improved Agent-Oriented Middleware for Distributed Smart Cameras , 2007, 2007 First ACM/IEEE International Conference on Distributed Smart Cameras.

[4]  Wojciech Matusik,et al.  3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes , 2004, ACM Trans. Graph..

[5]  Marc Levoy,et al.  High-speed videography using a dense camera array , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[6]  Wayne H. Wolf,et al.  Design and implementation of ubiquitous smart cameras , 2006, IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing (SUTC'06).

[7]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[8]  Bob Quinn,et al.  Windows Sockets Network Programming , 1995 .

[9]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Hans-Peter Seidel,et al.  Free-viewpoint video of human actors , 2003, ACM Trans. Graph..

[11]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[12]  Mark A. Horowitz,et al.  Light field video camera , 2000, IS&T/SPIE Electronic Imaging.

[13]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, ACM Trans. Graph..

[14]  Richard T. Vaughan,et al.  The Player/Stage Project: Tools for Multi-Robot and Distributed Sensor Systems , 2003 .

[15]  Cheng Lei,et al.  Region-Tree Based Stereo Using Dynamic Programming Optimization , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[16]  Nobuyuki Yamasaki,et al.  Design and implementation of reconfigurable middleware for sensorized environments , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[17]  Sylvia C. Wong,et al.  Lightweight Agent Framework for Camera Array Applications , 2005, KES.

[18]  Bernhard Rinner,et al.  Distributed embedded smart cameras for surveillance applications , 2006, Computer.

[19]  Yo-Sung Ho,et al.  A Framework for Representation and Processing of Multi-view Video Using the Concept of Layered Depth Image , 2007, J. VLSI Signal Process..

[20]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[21]  Wojciech Matusik,et al.  Natural video matting using camera arrays , 2006, SIGGRAPH '06.

[22]  Tarak Gandhi,et al.  Distributed interactive video arrays for event capture and enhanced situational awareness , 2005, IEEE Intelligent Systems.

[23]  Luc Van Gool,et al.  GPU-Based Foreground-Background Segmentation using an Extended Colinearity Criterion , 2005 .