An Insect Eye Inspired Miniaturized Multi-Camera System for Endoscopic Imaging

In this work, we present a miniaturized high definition vision system inspired by insect eyes, with a distributed illumination method, which can work in dark environments for proximity imaging applications such as endoscopy. Our approach is based on modeling biological systems with off-the-shelf miniaturized cameras combined with digital circuit design for real time image processing. We built a 5 mm radius hemispherical compound eye, imaging a <inline-formula><tex-math notation="LaTeX">$180^{\circ}\times 180^{\circ}$</tex-math></inline-formula> degrees field of view while providing more than 1.1 megapixels (emulated ommatidias) as real-time video with an inter-ommatidial angle <inline-formula><tex-math notation="LaTeX">$\Delta\phi=0.5^{\circ}$</tex-math></inline-formula> at 18 mm radial distance. We made an FPGA implementation of the image processing system which is capable of generating 25 fps video with 1080 <inline-formula><tex-math notation="LaTeX">$\times$</tex-math></inline-formula> 1080 pixel resolution at a 120 MHz processing clock frequency. When compared to similar size insect eye mimicking systems in literature, the system proposed in this paper features 1000 <inline-formula><tex-math notation="LaTeX">$\times$</tex-math></inline-formula> resolution increase. To the best of our knowledge, this is the first time that a compound eye with built-in illumination idea is reported. We are offering our miniaturized imaging system for endoscopic applications like colonoscopy or laparoscopic surgery where there is a need for large field of view high definition imagery. For that purpose we tested our system inside a human colon model. We also present the resulting images and videos from the human colon model in this paper.

[1]  I. Gralnek,et al.  Emerging technological advancements in colonoscopy: Third Eye® Retroscope® and Third Eye® PanoramicTM, Fuse® Full Spectrum Endoscopy® colonoscopy platform, Extra‐Wide‐Angle‐View colonoscope, and NaviAidTM G‐EYETM balloon colonoscope , 2015, Digestive endoscopy : official journal of the Japan Gastroenterological Endoscopy Society.

[2]  J. Palka,et al.  Diffraction and Visual Acuity of Insects , 1965, Science.

[3]  Laurent Jacques,et al.  The PANOPTIC Camera: A Plenoptic Sensor with Real-Time Omnidirectional Capability , 2013, J. Signal Process. Syst..

[4]  A. Jemal,et al.  Global cancer statistics, 2012 , 2015, CA: a cancer journal for clinicians.

[5]  Viktor Malyarchuk,et al.  Digital cameras with designs inspired by the arthropod eye , 2013, Nature.

[6]  Qiyin Fang,et al.  Development of a catadioptric endoscope objective with forward and side views. , 2011, Journal of biomedical optics.

[7]  Dario Floreano,et al.  Miniature curved artificial compound eyes , 2013, Proceedings of the National Academy of Sciences.

[8]  Peter D Siersema,et al.  Comparison of standard forward-viewing mode versus ultrawide-viewing mode of a novel colonoscopy platform: a prospective, multicenter study in the detection of simulated polyps in an in vitro colon model (with video). , 2013, Gastrointestinal endoscopy.

[9]  Yusuf Leblebici,et al.  A Real-Time Multiaperture Omnidirectional Visual Sensor Based on an Interconnected Network of Smart Cameras , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Jung-Hwan Oh,et al.  Automatic real-time detection of endoscopic procedures using temporal features , 2012, Comput. Methods Programs Biomed..

[11]  Hongrui Jiang,et al.  Flexible Miniaturized Camera Array Inspired by Natural Visual Systems , 2013, Journal of Microelectromechanical Systems.

[12]  Yasushi Yagi,et al.  Omnidirectional Vision Attachment for Medical Endoscopes , 2008 .

[13]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[14]  Fernando Vilariño,et al.  Impact of image preprocessing methods on polyp localization in colonoscopy frames , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[15]  Olivier Romain,et al.  Prototype of Video Endoscopic Capsule With 3-D Imaging Capabilities , 2010, IEEE Transactions on Biomedical Circuits and Systems.

[16]  Ching-Hwa Cheng,et al.  A panoramic endoscope design and implementation for Minimally Invasive Surgery , 2014, 2014 IEEE International Symposium on Circuits and Systems (ISCAS).

[17]  Thomas D. Wang,et al.  Future and advances in endoscopy , 2011, Journal of biophotonics.

[18]  Fernando Vilariño,et al.  Towards automatic polyp detection with a polyp appearance model , 2012, Pattern Recognit..

[19]  Luke P. Lee,et al.  Inspirations from Biological Optics for Advanced Photonic Systems , 2005, Science.

[20]  Yusuf Leblebici,et al.  A new omni-directional multi-camera system for high resolution surveillance , 2014, Sensing Technologies + Applications.

[21]  Luke P. Lee,et al.  Biologically Inspired Artificial Compound Eyes , 2006, Science.

[22]  Simon Thibault,et al.  360° endoscopy using panomorph lens technology , 2010, BiOS.

[23]  Ralf Kiesslich,et al.  A novel balloon colonoscope detects significantly more simulated polyps than a standard colonoscope in a colon model. , 2014, Gastrointestinal endoscopy.

[24]  Yusuf Leblebici,et al.  Image Blending in a High Frame Rate FPGA-based Multi-Camera System , 2013, Journal of Signal Processing Systems.

[25]  Huilong Duan,et al.  Global and Local Panoramic Views for Gastroscopy: An Assisted Method of Gastroscopic Lesion Surveillance , 2015, IEEE Transactions on Biomedical Engineering.

[26]  Franz Aurenhammer,et al.  Voronoi diagrams—a survey of a fundamental geometric data structure , 1991, CSUR.

[27]  Zhihua Wang,et al.  A new system design of the multi-view Micro-Ball endoscopy system , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[28]  Fernando Vilariño,et al.  Texture-Based Polyp Detection in Colonoscopy , 2009, Bildverarbeitung für die Medizin.

[29]  Yusuf Leblebici,et al.  Spherical Panorama Construction Using Multi Sensor Registration Priors and Its Real-Time Hardware , 2013, 2013 IEEE International Symposium on Multimedia.

[30]  Shinji Tanaka,et al.  A novel extra-wide-angle-view colonoscope: a simulated pilot study using anatomic colorectal models. , 2013, Gastrointestinal endoscopy.

[31]  Moshe Rubin,et al.  Mo1517 Successful Deployment and Use of Third Eye Panoramic™ a Novel Side Viewing Video CAP Fitted on a Standard Colonoscope , 2014 .

[32]  Zamir Halpern,et al.  Su1221 Aer-O-Scope™, a Self-Propelled Pneumatic Colonoscope, Is Superior to Conventional Colonoscopy in Polyp Detection , 2014 .

[33]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[34]  M. Land Visual acuity in insects. , 1997, Annual review of entomology.

[35]  T E Sherk,et al.  Development of the compound eyes of dragonflies (Odonata). III. Adult compound eyes. , 1977, The Journal of experimental zoology.