The eISP low-power and tiny silicon footprint programmable video architecture

CMOS sensors are now more and more frequently integrated into popular consumer products. Images from these sensors thus need to be digitally processed for display purposes. To do so, CMOS sensors are associated with dedicated components that keep power consumption low. However, use of dedicated components limits hardware flexibility and prevents updating of image processing algorithms. This paper describes the eISP, a programmable processing architecture that combines enough computational efficiency for 1080p HD video with silicon area and power characteristics suitable for the next generation of mobile phones (lower than 1 mm2 and 500 mW in TSMC 65 nm).

[1]  Aleksandar Beric,et al.  HiveFlex-Video VSP1: Video Signal Processing Architecture for Video Coding and Post-Processing , 2006, Eighth IEEE International Symposium on Multimedia (ISM'06).

[2]  John C. Russ,et al.  The Image Processing Handbook , 2016, Microscopy and Microanalysis.

[3]  Joaquim Salvi,et al.  Review of CMOS image sensors , 2006, Microelectron. J..

[4]  Jörg Henkel,et al.  Rapid configuration and instruction selection for an ASIP: a case study , 2003, 2003 Design, Automation and Test in Europe Conference and Exhibition.

[5]  N. Kehtarnavaz,et al.  Development and Real-Time Implementation of Auto White Balancing Scoring Algorithm , 2002, Real Time Imaging.

[6]  Lorenzo Verdoscia,et al.  Image processing chain for digital still cameras based on the SIMPil architecture , 2005, 2005 International Conference on Parallel Processing Workshops (ICPPW'05).

[7]  Tao Jianhua,et al.  A Fast Implementation of Adaptive Histogram Equalization , 2006, 2006 8th international Conference on Signal Processing.

[8]  April S. Brown,et al.  SIMPil: an OE integrated SIMD architecture for focal plane processing applications , 1996, Proceedings of Massively Parallel Processing Using Optical Interconnections.

[9]  William J. Dally,et al.  A Programmable 512 GOPS Stream Processor for Signal, Image, and Video Processing , 2007, IEEE Journal of Solid-State Circuits.

[10]  Youngjin Yoo,et al.  CMOS image sensor noise reduction method for image signal processor in digital cameras and camera phones , 2007, Electronic Imaging.

[11]  William J. Dally,et al.  Imagine: Media Processing with Streams , 2001, IEEE Micro.

[12]  Akihiro Tamura,et al.  Adaptive gamma processing of the video cameras for the expansion of the dynamic range , 1995 .

[13]  Shao-Yi Chien,et al.  CRISP: Coarse-Grained Reconfigurable Image Stream Processor for Digital Still Cameras and Camcorders , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Sabine Süsstrunk,et al.  Linear demosaicing inspired by the human visual system , 2005, IEEE Transactions on Image Processing.

[15]  Nicholas Nethercote,et al.  Dynamic Binary Analysis and Instrumentation , 2004 .

[16]  Shorin Kyo,et al.  An integrated memory array processor architecture for embedded image recognition systems , 2005, 32nd International Symposium on Computer Architecture (ISCA'05).

[17]  Kuo-Cheng Liu,et al.  Adaptive Color Filter Array Demosaicking Based on Constant Hue and Local Properties of Luminance , 2007, PSIVT.

[18]  Joost van de Weijer,et al.  Color constancy based on the Grey-edge hypothesis , 2005, IEEE International Conference on Image Processing 2005.

[19]  Tughrul Arslan,et al.  The Reconfigurable Instruction Cell Array , 2008, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[20]  Brian A. Wandell,et al.  How small should pixel size be? , 2000, Electronic Imaging.

[21]  Laurence Meylan,et al.  Digital camera workflow for high dynamic range images using a model of retinal processing , 2008, Electronic Imaging.

[22]  Grant Martin,et al.  Configurable Multi-Processor Platforms for Next Generation Embedded Systems , 2007, 2007 Asia and South Pacific Design Automation Conference.

[23]  S. Tominaga,et al.  Adaptive Filtering for Color Image Sharpening and Denoising , 2007, 14th International Conference of Image Analysis and Processing - Workshops (ICIAPW 2007).

[24]  Ardeshir Goshtasby,et al.  On the Canny edge detector , 2001, Pattern Recognit..

[25]  T. Komuro,et al.  A new algorithm for exposure control based on fuzzy logic for video cameras , 1992 .

[26]  Soo-Won Kim,et al.  Enhanced Autofocus Algorithm Using Robust Focus Measure and Fuzzy Reasoning , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[27]  Laurence Meylan,et al.  High dynamic range image rendering with a retinex-based adaptive filter , 2006, IEEE Transactions on Image Processing.

[28]  Wesley E. Snyder,et al.  Demosaicking methods for Bayer color arrays , 2002, J. Electronic Imaging.

[29]  Jianhua Tao,et al.  A Fast Implementation of Adaptive Histogram Equalization , 2006 .

[30]  Claudio Mucci,et al.  A cycle-accurate ISS for a dynamically reconfigurable processor architecture , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[31]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).