Review of state-of-the-art artificial compound eye imaging systems

The natural compound eye has received much attention in recent years due to its remarkable properties, such as its large field of view (FOV), compact structure, and high sensitivity to moving objects. Many studies have been devoted to mimicking the imaging system of the natural compound eye. The paper gives a review of state-of-the-art artificial compound eye imaging systems. Firstly, we introduce the imaging principle of three types of natural compound eye. Then, we divide current artificial compound eye imaging systems into four categories according to the difference of structural composition. Readers can easily grasp methods to build an artificial compound eye imaging system from the perspective of structural composition. Moreover, we compare the imaging performance of state-of-the-art artificial compound eye imaging systems, which provides a reference for readers to design system parameters of an artificial compound eye imaging system. Next, we present the applications of the artificial compound eye imaging system including imaging with a large FOV, imaging with high resolution, object distance detection, medical imaging, egomotion estimation, and navigation. Finally, an outlook of the artificial compound eye imaging system is highlighted.

[1]  Luke P. Lee,et al.  Inspirations from Biological Optics for Advanced Photonic Systems , 2005, Science.

[2]  Jinglei Du,et al.  Subtle control on hierarchic reflow for the simple and massive fabrication of biomimetic compound eye arrays in polymers for imaging at a large field of view , 2016 .

[3]  Yusuf Leblebici,et al.  A new omni-directional multi-camera system for high resolution surveillance , 2014, Sensing Technologies + Applications.

[4]  Keiichiro Kagawa,et al.  Irregular Lens Arrangement Design to Improve Imaging Performance of Compound-Eye Imaging Systems , 2010 .

[5]  Jun Tanida,et al.  Compact image capturing system based on compound imaging and digital reconstruction , 2001, Optics + Photonics.

[6]  S. Chen,et al.  Hybrid imprinting process to fabricate a multi-layer compound eye for multispectral imaging. , 2017, Optics express.

[7]  Keiichiro Kagawa,et al.  An Active Intraoral Shape Measurement Scheme Using a Compact Compound-Eye Camera with Integrated Pattern Projectors , 2009 .

[8]  Dario Floreano,et al.  A method for ego-motion estimation in micro-hovering platforms flying in very cluttered environments , 2016, Auton. Robots.

[9]  Yusuf Leblebici,et al.  Hemispherical Multiple Camera System for High Resolution Omni-Directional Light Field Imaging , 2013, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[10]  Jens Dunkel,et al.  Fabrication of microoptical freeform arrays on wafer level for imaging applications. , 2015, Optics express.

[11]  Yusuf Leblebici,et al.  Miniaturization of Multi-Camera Systems , 2017 .

[12]  Le Song,et al.  Bionic compound eye for 3D motion detection using an optical freeform surface , 2017 .

[13]  Keyi Wang,et al.  Development of an artificial compound eye system for three-dimensional object detection. , 2014, Applied optics.

[14]  Yongsheng Shi,et al.  Graphene-Based Bioinspired Compound Eyes for Programmable Focusing and Remote Actuation. , 2015, ACS applied materials & interfaces.

[15]  Yusuke Ogura,et al.  Three-Dimensional Information Acquisition Using a Compound Imaging System , 2007 .

[16]  A. Borst,et al.  Optical devices: Seeing the world through an insect's eyes , 2013, Nature.

[17]  Guillem Carles,et al.  Multi-aperture foveated imaging. , 2016, Optics letters.

[18]  Andrew R Harvey,et al.  Super-resolution imaging using a camera array. , 2014, Optics letters.

[19]  Heung-No Lee,et al.  Depth-estimation-enabled compound eyes , 2018 .

[20]  Viktor Malyarchuk,et al.  Digital cameras with designs inspired by the arthropod eye , 2013, Nature.

[21]  Zhao Wu,et al.  Shape-controllable polymeric microlens array duplicated by electrostatic force deformed template , 2013 .

[22]  Albert J. P. Theuwissen,et al.  CMOS image sensors: State-of-the-art , 2008 .

[23]  Toward individually tunable compound eyes with transparent graphene electrode. , 2017, Bioinspiration & biomimetics.

[24]  Lianguan Shen,et al.  Flexible fabrication of biomimetic compound eye array via two-step thermal reflow of simply pre-modeled hierarchic microstructures , 2017 .

[25]  Chong-Min Kyung,et al.  Smart Sensors and Systems , 2015 .

[26]  Andreas Tünnermann,et al.  Ultraslim fixed pattern projectors with inherent homogenization of illumination. , 2012, Applied optics.

[27]  S F Barrett,et al.  A bio-inspired apposition compound eye machine vision sensor system , 2009, Bioinspiration & biomimetics.

[28]  Hanspeter A. Mallot,et al.  Hardware Architecture and Cutting-Edge Assembly Process of a Tiny Curved Compound Eye , 2014, Sensors.

[29]  Yusuke Ogura,et al.  3D information acquisition using a compound imaging system , 2007, SPIE Optical Engineering + Applications.

[30]  R. Smythe,et al.  Vision in the animal world , 1975 .

[31]  Mengchao Ma,et al.  Catadioptric planar compound eye with large field of view. , 2018, Optics express.

[32]  Carl E. Halford,et al.  Design and analysis of apposition compound eye optical sensors , 1995 .

[33]  Yiqing Gao,et al.  Curved micro lens array for bionic compound eye , 2013 .

[34]  Dario Floreano,et al.  A small-scale hyperacute compound eye featuring active eye tremor: application to visual stabilization, target tracking, and short-range odometry. , 2015, Bioinspiration & biomimetics.

[35]  Andreas Tünnermann,et al.  Microoptical telescope compound eye. , 2005, Optics express.

[36]  Young Min Song,et al.  Recent advances in imaging systems and photonic nanostructures inspired by insect eye geometry , 2018 .

[37]  Chunlei Du,et al.  Artificial compound-eye imaging system with a large field of view based on a convex solid substrate , 2010, SPIE/COS Photonics Asia.

[38]  Lei Li,et al.  Microfabrication on a curved surface using 3D microlens array projection , 2009 .

[39]  Young Min Song,et al.  COMPU-EYE: a high resolution computational compound eye. , 2016, Optics express.

[40]  M. Land Visual acuity in insects. , 1997, Annual review of entomology.

[41]  Lei Li,et al.  Design and fabrication of a freeform microlens array for a compact large-field-of-view compound-eye camera. , 2012, Applied optics.

[42]  Changxi Xue,et al.  Design and simulation of a superposition compound eye system based on hybrid diffractive-refractive lenses. , 2017, Applied optics.

[43]  Yusuf Leblebici Design and implementation of real-time multi-sensor vision systems , 2016, 2016 International Great Lakes Symposium on VLSI (GLSVLSI).

[44]  Andreas Tünnermann,et al.  Optical Cluster Eye fabricated on wafer-level. , 2011, Optics express.

[45]  Hongrui Jiang,et al.  Focus-Tunable Microlens Arrays Fabricated on Spherical Surfaces , 2011, Journal of Microelectromechanical Systems.

[46]  Jun Tanida,et al.  Bimodal fingerprint capturing system based on compound-eye imaging module. , 2004, Applied optics.

[47]  A. Tünnermann,et al.  Thin compound-eye camera. , 2005, Applied optics.

[48]  Kuno Kirschfeld,et al.  The Resolution of Lens and Compound Eyes , 1976 .

[49]  Tomás Pajdla,et al.  Structure from motion with wide circular field of view cameras , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Michael F. Land The optics of animal eyes , 1988 .

[51]  Tatsuki Okamoto,et al.  Practical design for compact image scanner with large depth of field by compound eye system. , 2014, Optics express.

[52]  D. Nilsson,et al.  A new type of imaging optics in compound eyes , 1988, Nature.

[53]  Qing Yang,et al.  Direct fabrication of compound-eye microlens array on curved surfaces by a facile femtosecond laser enhanced wet etching process , 2016 .

[54]  J. Zeil,et al.  A new kind of neural superposition eye: the compound eye of male Bibionidae , 1979, Nature.

[55]  David J. Brady,et al.  Multiscale gigapixel photography , 2012, Nature.

[56]  Hongrui Jiang,et al.  Tunable-focus microlens arrays on curved surfaces , 2010 .

[57]  Yusuf Leblebici,et al.  A spherical multi-camera system with real-time omnidirectional video acquisition capability , 2012, IEEE Transactions on Consumer Electronics.

[58]  Hydrogel-actuated tunable-focus microlens arrays mimicking compound eyes , 2009, TRANSDUCERS 2009 - 2009 International Solid-State Sensors, Actuators and Microsystems Conference.

[59]  Kenneth J. Weible,et al.  Miniaturized imaging systems , 2003 .

[60]  Xun Hou,et al.  A simple route to fabricate artificial compound eye structures. , 2012, Optics express.

[61]  Keiichiro Kagawa,et al.  A three‐dimensional multifunctional compound‐eye endoscopic system with extended depth of field , 2012 .

[62]  Peter Dannberg,et al.  Chirped arrays of refractive ellipsoidal microlenses for aberration correction under oblique incidence. , 2005, Optics express.

[63]  F. G. Varela The vertebrate and the (insect) compound eye in evolutionary perspective. , 1971, Vision Research.

[64]  H. Yu,et al.  Fabrication and investigation of the bionic curved visual microlens array films , 2017 .

[65]  Man Zhang,et al.  Design and fabrication of a multifocal bionic compound eye for imaging , 2018, Bioinspiration & biomimetics.

[66]  M E Gehm,et al.  Characterization of the AWARE 10 two-gigapixel wide-field-of-view visible imager. , 2014, Applied optics.

[67]  Andreas Tünnermann,et al.  Artificial neural superposition eye. , 2007, Optics express.

[68]  Tim Dobbert Matchmoving: The Invisible Art of Camera Tracking , 2005 .

[69]  D. Brady,et al.  Characterization of the AWARE 40 wide-field-of-view visible imager , 2015 .

[70]  Keyi Wang,et al.  Design and fabrication of tunable aspherical lens for spherical compound eye , 2010, International Symposium on Advanced Optical Manufacturing and Testing Technologies (AOMATT).

[71]  Cameron H. G. Wright,et al.  Musca domestica inspired machine vision sensor with hyperacuity , 2008, Bioinspiration & biomimetics.

[72]  Peter Schreiber,et al.  Theoretical analysis of an artificial superposition compound eye for application in ultra flat digital image acquisition devices , 2004, SPIE Optical Systems Design.

[73]  Luke P. Lee,et al.  Bioinspired Fabrication of High‐Quality 3D Artificial Compound Eyes by Voxel‐Modulation Femtosecond Laser Writing for Distortion‐Free Wide‐Field‐of‐View Imaging , 2014 .

[74]  Shree K. Nayar,et al.  PiCam , 2013, ACM Trans. Graph..

[75]  Keyi Wang,et al.  Automatic geometric calibration and three-dimensional detecting with an artificial compound eye , 2017 .

[76]  G. Su,et al.  Fabrication of polydimethylsiloxane microlens array on spherical surface using multi-replication process , 2013 .

[77]  Lei Li,et al.  Development of a 3D artificial compound eye. , 2010, Optics express.

[78]  M C Hutley,et al.  Imaging properties of the Gabor superlens , 1999 .

[79]  Nobert Kaiser Optical Systems Design , 2015 .

[80]  D. Nilsson,et al.  Vision Optics and EvolutionNature's engineering has produced astonishing diversity in eye design , 1989 .

[81]  Keiichiro Kagawa,et al.  Application of a compound imaging system to odontotherapy , 2015 .

[82]  M Sieler,et al.  Microoptical array projectors for free-form screen applications. , 2013, Optics express.

[83]  Christian Früh,et al.  Google Street View: Capturing the World at Street Level , 2010, Computer.

[84]  E. Warrant,et al.  Visual sensitivity in the crepuscular owl butterfly Caligo memnon and the diurnal blue morpho Morpho peleides: a clue to explain the evolution of nocturnal apposition eyes? , 2008, Journal of Experimental Biology.

[85]  Qun Hao,et al.  Reducing defocus aberration of a compound and human hybrid eye using liquid lens. , 2018, Applied optics.

[86]  Heung Cho Ko,et al.  A hemispherical electronic eye camera based on compressible silicon optoelectronics , 2008, Nature.

[87]  Hamid Reza Fallah,et al.  MTF of compound eye. , 2010, Optics express.

[88]  Kang Wei,et al.  Insect-Human Hybrid Eye (IHHE): an adaptive optofluidic lens combining the structural characteristics of insect and human eyes. , 2014, Lab on a chip.

[89]  Kuno Kirschfeld,et al.  The Absolute Sensitivity of Lens and Compound Eyes , 1974, Zeitschrift fur Naturforschung. Section C, Biosciences.

[90]  Ming Zhu,et al.  Artificial compound eye: a survey of the state-of-the-art , 2016, Artificial Intelligence Review.

[91]  J. Tanida,et al.  Thin Observation Module by Bound Optics (TOMBO): Concept and Experimental Verification. , 2001, Applied optics.

[92]  Keiichiro Kagawa,et al.  A compact shape-measurement module based on a thin compound-eye camera with multiwavelength diffractive pattern projection for intraoral diagnosis , 2009, Optical Engineering + Applications.

[93]  Luke P. Lee,et al.  Biologically Inspired Artificial Compound Eyes , 2006, Science.

[94]  Lu Yu,et al.  Optical system design of artificial compound eye based on field stitching , 2017 .

[95]  Keyi Wang,et al.  Photometric Calibration and Image Stitching for a Large Field of View Multi-Camera System , 2016, Sensors.

[96]  Laurent Jacques,et al.  The PANOPTIC Camera: A Plenoptic Sensor with Real-Time Omnidirectional Capability , 2013, J. Signal Process. Syst..

[97]  Jun Tanida,et al.  Color imaging with an integrated compound imaging system. , 2003, Optics express.

[98]  Xiaobin Cui,et al.  Fabrication of microlens arrays with varied focal lengths on curved surfaces using an electrostatic deformed template , 2014 .

[99]  Julien Serres,et al.  A quasi-panoramic bio-inspired eye for flying parallel to walls , 2017, 2017 IEEE SENSORS.

[100]  A. Borst Drosophila's View on Insect Vision , 2009, Current Biology.

[101]  Hamid Reza Fallah,et al.  Design and simulation of a high-resolution superposition compound eye , 2007 .

[102]  Andreas Tünnermann,et al.  The Gabor superlens as an alternative wafer-level camera approach inspired by superposition compound eyes of nocturnal insects. , 2009, Optics express.

[103]  Andreas Tünnermann,et al.  Ultra-compact imaging system based on multi-aperture architecture , 2011, MOEMS-MEMS.

[104]  Jacques Duparré,et al.  Spherical artificial compound eye captures real images , 2007, SPIE MOEMS-MEMS.

[105]  Yongcai Guo,et al.  Enhancing the imaging quality and fabrication efficiency of bionic compound eyes using a sandwich structure , 2018 .

[106]  Eric J. Warrant,et al.  Retinal and optical adaptations for nocturnal vision in the halictid bee Megalopta genalis , 2004, Cell and Tissue Research.

[107]  Qing Yang,et al.  Fabrication of bioinspired omnidirectional and gapless microlens array for wide field-of-view detections , 2012 .

[108]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, SIGGRAPH 2005.

[109]  Frank Wippermann,et al.  Micro-optical artificial compound eyes. , 2006 .

[110]  Frank Wippermann,et al.  Microoptical artificial compound eyes: from design to experimental verification of two different concepts , 2005, SPIE Optical Systems Design.

[111]  Jun Tanida,et al.  Compact compound-eye projector using superresolved projection. , 2011, Optics letters.

[112]  Andreas Brückner,et al.  Advanced artificial compound-eye imaging systems , 2008, SPIE MOEMS-MEMS.

[113]  Peter Schreiber,et al.  Micro-optically fabricated artificial apposition compound eye , 2004, IS&T/SPIE Electronic Imaging.

[114]  Yi Zhao,et al.  A BIOINSPIRED 3D ARTIFICIAL COMPOUND EYE WITH FOCUS-TUNABLE SINGLE LENSES , 2011 .

[115]  Hanspeter A. Mallot,et al.  An artificial elementary eye with optic flow detection and compositional properties , 2015, Journal of The Royal Society Interface.

[116]  Peter Dannberg,et al.  Artificial compound eye zoom camera , 2008, Bioinspiration & biomimetics.

[117]  Franck Ruffier,et al.  Flying over uneven moving terrain based on optic-flow cues without any need for reference frames or accelerometers , 2015, Bioinspiration & biomimetics.

[118]  Yanjun Li,et al.  Robot vision navigation based on a fly's compound eyes , 2007, Other Conferences.

[119]  Single step fabrication of microlens arrays with hybrid HfO2-SiO2 sol-gel glass on conventional lens surface. , 2005, Optics express.

[120]  Peter Schreiber,et al.  Artificial compound eyes: different concepts and their application for ultraflat image acquisition sensors , 2004, SPIE MOEMS-MEMS.

[121]  Qun Hao,et al.  Compound eye and retina-like combination sensor with a large field of view based on a space-variant curved micro lens array. , 2017, Applied optics.

[122]  Weixing Yu,et al.  SCECam: a spherical compound eye camera for fast location and recognition of objects at a large field of view , 2017 .

[123]  Andreas Brückner,et al.  Latest developments in micro-optical artificial compound eyes: a promising approach for next generation ultracompact machine vision , 2007, Electronic Imaging.

[124]  Andreas Tünnermann,et al.  Laser lithographic fabrication and characterization of a spherical artificial compound eye. , 2007, Optics express.

[125]  H. Yu,et al.  Fabrication and characterization of artificial miniaturized insect compound eyes for imaging. , 2015, Bioinspiration & biomimetics.

[126]  Allan W. Snyder,et al.  Spatial information capacity of compound eyes , 2004, Journal of comparative physiology.

[127]  Dario Floreano,et al.  Miniature artificial compound eyes for optic-flow-based robotic navigation , 2014, 2014 13th Workshop on Information Optics (WIO).

[128]  Dario Floreano,et al.  Miniature curved artificial compound eyes , 2013, Proceedings of the National Academy of Sciences.

[129]  M. Land Compound eye structure: Matching eye to environment , 1999 .

[130]  Hanspeter A. Mallot,et al.  Odometry for Ground Moving Agents by Optic Flow Recorded with Optical Mouse Chips , 2014, Sensors.

[131]  Luke P. Lee,et al.  Artificial ommatidia by self-aligned microlenses and waveguides. , 2005, Optics letters.

[132]  Andreas Tünnermann,et al.  Thin wafer-level camera lenses inspired by insect compound eyes. , 2010, Optics express.

[133]  Yusuf Leblebici,et al.  An Insect Eye Inspired Miniaturized Multi-Camera System for Endoscopic Imaging , 2017, IEEE Transactions on Biomedical Circuits and Systems.

[134]  Adnan Acan,et al.  A multiagent, dynamic rank-driven multi-deme architecture for real-valued multiobjective optimization , 2016, Artificial Intelligence Review.

[135]  Man Zhang,et al.  Structural design and image processing of a spherical artificial compound eye , 2015 .

[136]  Chunlei Du,et al.  An artificial compound eye system for large field imaging , 2012, Photonics Asia.

[137]  Richard Hornsey,et al.  Calibration techniques for object tracking using a compound eye image sensor , 2004, SPIE Security + Defence.

[138]  N. Franceschini,et al.  From insect vision to robot vision , 1992 .

[139]  Ju Sung Park,et al.  Artificially Engineered Compound Eye Sensing Systems , 2017 .