Real-Time Omnidirectional Imaging System with Interconnected Network of Cameras

The Panoptic camera is an omnidirectional multi-aperture visual system which is realized by mounting multiple imaging sensors on a hemispherical frame. In this chapter, we will present a novel distributed and parallel implementation of the real-time omnidirectional vision reconstruction algorithm of the Panoptic Camera, for camera modules with processing, memory and interconnectivity features. A methodology for the arrangement of camera modules with interconnectivity feature into a target interconnection network topology will be introduced. A unique custom-made multiple-FPGA hardware platform designed for the implementation of an interconnected network of a 49 camera prototype Panoptic system will be explained. A novel way to represent the omnidirectional data obtained from the Panoptic Camera and real-time high dynamic range (HDR) video application which is realized by the system will be presented.

[1]  Christian Früh,et al.  Google Street View: Capturing the World at Street Level , 2010, Computer.

[2]  S. L. HAKIMIt.  AN ALGORITHMIC APPROACH TO NETWORK LOCATION PROBLEMS. , 1979 .

[3]  O. Kariv,et al.  An Algorithmic Approach to Network Location Problems. II: The p-Medians , 1979 .

[4]  Yusuf Leblebici,et al.  Performance Optimization and FPGA Implementation of Real-Time Tone Mapping , 2014, IEEE Transactions on Circuits and Systems II: Express Briefs.

[5]  Laurent Jacques,et al.  The PANOPTIC Camera: A Plenoptic Sensor with Real-Time Omnidirectional Capability , 2013, J. Signal Process. Syst..

[6]  Masayuki Tanimoto,et al.  Multiview Imaging and 3DTV , 2007, IEEE Signal Processing Magazine.

[7]  Christian Weissig,et al.  Ultrahigh-Resolution Panoramic Imaging for Format-Agnostic Video Production , 2013, Proceedings of the IEEE.

[8]  Nan Jiang,et al.  A detailed and flexible cycle-accurate Network-on-Chip simulator , 2013, 2013 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS).

[9]  Yuan Xu,et al.  High-Speed Simultaneous Image Distortion Correction Transformations for a Multicamera Cylindrical Panorama Real-time Video System Using FPGA , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Mustafa Ç. Pınar,et al.  An exact algorithm for the capacitated vertex p-center problem , 2006, Comput. Oper. Res..

[11]  Yusuf Leblebici,et al.  A Real-Time Multiaperture Omnidirectional Visual Sensor Based on an Interconnected Network of Smart Cameras , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH.

[13]  Yusuf Leblebici,et al.  Spherical Panorama Construction Using Multi Sensor Registration Priors and Its Real-Time Hardware , 2013, 2013 IEEE International Symposium on Multimedia.

[14]  Tsuhan Chen,et al.  A Self-Reconfigurable Camera Array , 2004, Rendering Techniques.

[15]  Yusuf Leblebici,et al.  A spherical multi-camera system with real-time omnidirectional video acquisition capability , 2012, IEEE Transactions on Consumer Electronics.

[16]  Mark de Berg,et al.  Computational geometry: algorithms and applications , 1997 .

[17]  Hans-Peter Seidel,et al.  Perceptual evaluation of tone mapping operators with real-world scenes , 2005, IS&T/SPIE Electronic Imaging.

[18]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[19]  Takeo Kanade,et al.  Virtual ized reality: constructing time-varying virtual worlds from real world events , 1997 .

[20]  Panos M. Pardalos,et al.  A Greedy Randomized Adaptive Search Procedure for the Quadratic Assignment Problem , 1993, Quadratic Assignment and Related Problems.

[21]  Takeo Kanade,et al.  Virtualized Reality: Constructing Virtual Worlds from Real Scenes , 1997, IEEE Multim..

[22]  B. Markham,et al.  Summary of Current Radiometric Calibration Coefficients for Landsat MSS, TM, ETM+, and EO-1 ALI Sensors , 2009 .

[23]  Toshiaki Fujii,et al.  Free-Viewpoint TV , 2011, IEEE Signal Processing Magazine.

[24]  Karol Myszkowski,et al.  Adaptive Logarithmic Mapping For Displaying High Contrast Scenes , 2003, Comput. Graph. Forum.

[25]  Yusuf Leblebici,et al.  Image Blending in a High Frame Rate FPGA-based Multi-Camera System , 2013, Journal of Signal Processing Systems.

[26]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, SIGGRAPH 2005.

[27]  Naokazu Yokoya,et al.  Telepresence by Real-Time View-Dependent Image Generation from Omnidirectional Video Streams , 1998, Comput. Vis. Image Underst..

[28]  Yiannis Aloimonos,et al.  Polydioptric camera design and 3D motion estimation , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[29]  Steve Mann,et al.  ON BEING `UNDIGITAL' WITH DIGITAL CAMERAS: EXTENDING DYNAMIC RANGE BY COMBINING DIFFERENTLY EXPOSED PICTURES , 1995 .

[30]  David J. Brady,et al.  Multiscale gigapixel photography , 2012, Nature.

[31]  Naokazu Yokoya,et al.  Panoramic Movie Generation Using an Omnidirectional Multi-camera System for Telepresence , 2003, SCIA.

[32]  Yusuf Leblebici,et al.  Real-Time FPGA Implementation of Linear Blending Vision Reconstruction Algorithm Using a Spherical Light Field Camera , 2012, 2012 IEEE Workshop on Signal Processing Systems.

[33]  O. Kariv,et al.  An Algorithmic Approach to Network Location Problems. I: The p-Centers , 1979 .

[34]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[35]  Viktor Malyarchuk,et al.  Digital cameras with designs inspired by the arthropod eye , 2013, Nature.

[36]  Richard Szeliski,et al.  Image mosaicing for tele-reality applications , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[37]  Y. Leblebici,et al.  Real-time implementation of Gaussian image blending in a spherical light field camera , 2013, 2013 IEEE International Conference on Industrial Technology (ICIT).

[38]  Yusuf Leblebici,et al.  A new omni-directional multi-camera system for high resolution surveillance , 2014, Sensing Technologies + Applications.

[39]  Christian Bloch.  The HDRI Handbook: High Dynamic Range Imaging for Photographers and CG Artists , 2007 .

[40]  Yusuf Leblebici,et al.  Multi-camera platform for panoramic real-time HDR video construction and rendering , 2014, Journal of Real-Time Image Processing.

[41]  Naokazu Yokoya,et al.  Telepresence System with an Omnidirectional HD Camera , 2002 .

[42]  Leonard McMillan,et al.  A Real-Time Distributed Light Field Camera , 2002, Rendering Techniques.

[43]  M. Vrakking,et al.  High dynamic range bio-molecular ion microscopy with the Timepix detector. , 2011, Analytical chemistry.