Three-dimensional Imaging, Visualization and Recognition in Low Light Environments

There has been significant research in developing novel image sensors to capture a scene in low illumination environments. Using conventional passive imaging sensors that operate in the visible spectrum, camera read-noise becomes higher than the signal information from the scene resulting in read-noise dominant images. These images suffer from poor signal-to-noise ratio making scene visualization unobtainable. Thus, imaging in photon-limited environments has been viewed as a nuisance. This dissertation will consist of three separate parts that discuss utilizing photon-counting principals for secure information storage or imaging in photon-limited environments. First, an optical security system for information storage is introduced that uses secure three-dimensional optical phase codes whose unique optical signature can be authenticated using the random-forest classifier. Next, a system for securing three-dimensional integral imaging displays using a quick-response encoded elemental image array will be discussed that combines photoncounting with traditional optical encryption methods for secure information storage. The Adam Markman, Doctor of Philosophy University of Connecticut, 2018 second part of the dissertation will discuss integrated circuit authentication in photonlimited environments. More specifically, a photon-counting model will be applied to an image of an integrated circuit (IC) captured using an x-ray to demonstrate that IC authentication can be achieved when few photons are available in the scene. In the third part of this dissertation, three-dimensional (3D) imaging and object recognition in low light environments using real experimental data will be discussed. Afterwards, an object recognition framework for 3D reconstructed images obtained from elemental images taken in low light conditions using Convolutional Neural Networks will be introduced.

[1]  G. Unnikrishnan,et al.  Optical encryption by double-random phase encoding in the fractional Fourier domain. , 2000, Optics letters.

[2]  L Hesselink,et al.  Encrypted holographic data storage based on orthogonal-phase-code multiplexing. , 1995, Applied optics.

[3]  Applicability of an EM-CCD for spatially resolved TIR-ICS. , 2010, Optics express.

[4]  Considerations for minimizing radiation doses to components during X-ray inspection , 2005 .

[5]  Bahram Javidi,et al.  Three-dimensional photon counting integral imaging reconstruction using penalized maximum likelihood expectation maximization. , 2011, Optics express.

[6]  M. Ma,et al.  X-Ray Inspection-Induced Latent Damage in DRAM , 2006, 2006 IEEE International Reliability Physics Symposium Proceedings.

[7]  B. Javidi,et al.  Encrypted optical storage with wavelength-key and random phase codes. , 1999, Applied optics.

[8]  Rasit Turan,et al.  Effect of particle properties and light polarization on the plasmonic resonances in metallic nanoparticles. , 2010, Optics express.

[9]  K. Taguchi,et al.  Vision 20/20: Single photon counting x-ray detectors in medical imaging. , 2013, Medical physics.

[10]  A. Szalay,et al.  Imaging of light emission from the expression of luciferases in living cells and organisms: a review. , 2002, Luminescence : the journal of biological and chemical luminescence.

[11]  F. Okano,et al.  Gradient-index lens-array method based on real-time integral photography for three-dimensional images. , 1998, Applied optics.

[12]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[13]  J. C. Dainty,et al.  I The Statistics of Speckle Patterns , 1977 .

[14]  Bahram Javidi,et al.  Advances in optical security systems , 2014 .

[15]  B. Javidi,et al.  Photon-Counting Security Tagging and Verification Using Optically Encoded QR Codes , 2014, IEEE Photonics Journal.

[16]  Bahram Javidi,et al.  Three-dimensional identification of biological microorganism using integral imaging. , 2006, Optics express.

[17]  B Javidi,et al.  Optimum filter for detection of a target in nonoverlapping scene noise. , 1994, Applied optics.

[18]  Myungjin Cho,et al.  Information authentication using photon-counting double-random-phase encrypted images. , 2011, Optics letters.

[19]  Myungjin Cho,et al.  Three-dimensional photon counting double-random-phase encryption. , 2013, Optics letters.

[20]  Peng Zhang,et al.  Known-plaintext attack on optical encryption based on double random phase keys. , 2006, Optics letters.

[21]  Ranjan Bose,et al.  Information theory, coding and cryptography , 2003 .

[22]  Bahram Javidi,et al.  Histogram Approaches for Lossy Compression of Digital Holograms of Three-Dimensional Objects , 2007, IEEE Transactions on Image Processing.

[23]  Noam Levin,et al.  A global analysis of factors controlling VIIRS nighttime light levels from densely populated areas , 2017 .

[24]  F. Okano,et al.  Analysis of resolution limitation of integral photography , 1998 .

[25]  E. Roessl,et al.  K-edge imaging in x-ray computed tomography using multi-bin photon counting detectors , 2007, Physics in medicine and biology.

[26]  B Javidi,et al.  Free-depths reconstruction with synthetic impulse response in integral imaging. , 2015, Optics express.

[27]  Jonathan T. Barron,et al.  Burst photography for high dynamic range and low-light imaging on mobile cameras , 2016, ACM Trans. Graph..

[28]  Bahram Javidi,et al.  Polarimetric 3D integral imaging in photon-starved conditions. , 2015, Optics express.

[29]  Yiorgos Makris,et al.  Parametric counterfeit IC detection via Support Vector Machines , 2012, 2012 IEEE International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems (DFT).

[30]  M McCormick,et al.  Discussion of the optics of a new 3-D imaging system. , 1988, Applied optics.

[31]  Bahram Javidi,et al.  Security authentication with a three-dimensional optical phase code using random forest classifier. , 2016, Journal of the Optical Society of America. A, Optics, image science, and vision.

[32]  Ralph Gross,et al.  An Image Preprocessing Algorithm for Illumination Invariant Face Recognition , 2003, AVBPA.

[33]  Byoungho Lee,et al.  Depth‐enhanced integral‐imaging 3D display using different optical path lengths by polarization devices or mirror barrier array , 2004 .

[34]  B Javidi,et al.  3D integral imaging display by smart pseudoscopic-to-orthoscopic conversion (SPOC). , 2010, Optics express.

[35]  G M Morris,et al.  Image correlation at low light levels: a computer simulation. , 1984, Applied optics.

[36]  David A. Huffman,et al.  A method for the construction of minimum-redundancy codes , 1952, Proceedings of the IRE.

[37]  Junichi Nakamura,et al.  Image Sensors and Signal Processing for Digital Still Cameras , 2005 .

[38]  A. Cheng,et al.  Single-molecule detection and spectroscopy in point-like geometries ( a ) Point-like excitation and detection , 2012 .

[39]  Bahram Javidi,et al.  Resistance of the double random phase encryption against various attacks. , 2007, Optics express.

[40]  Bahram Javidi,et al.  Optical pattern recognition for validation and security verification , 1994 .

[41]  Bahram Javidi,et al.  Polarization encoding for optical security systems , 2000 .

[42]  Filiberto Pla,et al.  Human gesture recognition using three-dimensional integral imaging. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[43]  B. Javidi Nonlinear joint power spectrum based optical correlation. , 1989, Applied optics.

[44]  Kazuya Nakano,et al.  Evaluations of phase-only double random phase encoding based on key-space analysis. , 2013, Applied optics.

[45]  David J. Kriegman,et al.  What Is the Set of Images of an Object Under All Possible Illumination Conditions? , 1998, International Journal of Computer Vision.

[46]  Truong Q. Nguyen,et al.  An Augmented Lagrangian Method for Total Variation Video Restoration , 2011, IEEE Transactions on Image Processing.

[47]  B Javidi,et al.  Securing information by use of digital holography. , 2000, Optics letters.

[48]  Kouichi Nitta,et al.  Optical authentication method using a three-dimensional phase object with various wavelength readouts. , 2008, Applied optics.

[49]  Paul A. Viola,et al.  Detecting Pedestrians Using Patterns of Motion and Appearance , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[50]  Makoto Okui,et al.  Real-Time Integral Imaging Based on Extremely High Resolution Video System , 2006, Proceedings of the IEEE.

[51]  Tomoyuki Mishina 3D television system based on Integral Photography , 2010, 28th Picture Coding Symposium.

[52]  Bahram Javidi,et al.  Security authentication using phase-encoded nanoparticle structures and polarized light. , 2015, Optics letters.

[53]  B. Javidi,et al.  Integral imaging with improved depth of field by use of amplitude-modulated microlens arrays. , 2004, Applied optics.

[54]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[55]  Tobias M. P. Hartwich,et al.  Video-rate nanoscopy using sCMOS camera- specific single-molecule localization algorithms , 2013 .

[56]  Kehar Singh,et al.  Optical image encryption using a jigsaw transform for silhouette removal in interference-based methods and decryption with a single spatial light modulator. , 2011, Applied optics.

[57]  Bruce J. Schachter Automatic Target Recognition , 2016 .

[58]  F Dubois Automatic spatial frequency selection algorithm for pattern recognition by correlation. , 1993, Applied optics.

[59]  Bahram Javidi,et al.  Improved resolution 3D object sensing and recognition using time multiplexed computational integral imaging. , 2003, Optics express.

[60]  Kenjiro Watanabe,et al.  Novel Encryption Method Using Multi Reference Patterns in Coaxial Holographic Data Storage , 2007 .

[61]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Dhirendra Kumar,et al.  Photon counting imaging and phase mask multiplexing for multiple images authentication and digital hologram security , 2015 .

[63]  D. Edwards Data Mining: Concepts, Models, Methods, and Algorithms , 2003 .

[64]  Bahram Javidi,et al.  Optical security verification by synthesizing thin films with unique polarimetric signatures. , 2015, Optics letters.

[65]  Naveen K Nishchal,et al.  An optical encryption and authentication scheme using asymmetric keys. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[66]  B. Javidi,et al.  Three-dimensional synthetic aperture integral imaging. , 2002, Optics letters.

[67]  Hiroyuki Suzuki,et al.  Known plaintext attack on double random phase encoding using fingerprint as key and a method for avoiding the attack. , 2010, Optics express.

[68]  David J. Sheskin,et al.  Handbook of Parametric and Nonparametric Statistical Procedures , 1997 .

[69]  Berk Sunar,et al.  Trojan Detection using IC Fingerprinting , 2007, 2007 IEEE Symposium on Security and Privacy (SP '07).

[70]  Christian Eggeling,et al.  Breaking the diffraction barrier in fluorescence microscopy at low light intensities by using reversibly photoswitchable proteins. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[71]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[72]  Peng Guo,et al.  An adaptive enhancement algorithm for low-illumination image based on hue reserving , 2011, Proceedings of 2011 Cross Strait Quad-Regional Radio Science and Wireless Technology Conference.

[73]  Roberto Torroba,et al.  Optical encryption and QR codes: secure and noise-free information retrieval. , 2013, Optics express.

[74]  P. Réfrégier,et al.  Bhattacharyya distance as a contrast parameter for statistical processing of noisy optical images. , 2004, Journal of the Optical Society of America. A, Optics, image science, and vision.

[75]  Kouichi Nitta,et al.  Secure data storage by three-dimensional absorbers in highly scattering volume medium , 2008 .

[76]  Arturo Carnicer,et al.  Vulnerability to chosen-cyphertext attacks of optical encryption schemes based on double random phase keys. , 2005, Optics letters.

[77]  Bahram Javidi,et al.  Photon-counting double-random-phase encoding for secure image verification and retrieval , 2012 .

[78]  Bahram Javidi,et al.  Three-dimensional volumetric object reconstruction using computational integral imaging. , 2004, Optics express.

[79]  Bahram Javidi,et al.  Distortion-tolerant 3D recognition of underwater objects using neural networks. , 2010, Journal of the Optical Society of America. A, Optics, image science, and vision.

[80]  Tom Dowling,et al.  Introducing secure modes of operation for optical encryption. , 2008, Journal of the Optical Society of America. A, Optics, image science, and vision.

[81]  Konstantins Jefimovs,et al.  Linear and nonlinear optical responses influenced by broken symmetry in an array of gold nanoparticles. , 2004, Optics express.

[82]  Filiberto Pla,et al.  Real-time automated counterfeit integrated circuit detection using x-ray microscopy , 2015 .

[83]  Bahram Javidi,et al.  Advances in three-dimensional integral imaging: sensing, display, and applications [Invited]. , 2013, Applied optics.

[84]  Klaus Suhling,et al.  Photon arrival timing with sub-camera exposure time resolution in wide-field time-resolved photon counting imaging. , 2010, Optics express.

[85]  Myungjin Cho,et al.  Three-Dimensional Visualization of Objects in Turbid Water Using Integral Imaging , 2010, Journal of Display Technology.

[86]  Bahram Javidi,et al.  Influence of a perturbation in a double phase-encoding system , 1998 .

[87]  Li-Chen Fu,et al.  Pedestrian detection system in low illumination conditions through Fusion of image and range data , 2014, 17th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[88]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[89]  Chris Roman,et al.  Observations of in situ deep-sea marine bioluminescence with a high-speed, high-resolution sCMOS camera , 2016 .

[90]  Bahram Javidi,et al.  Three-dimensional integral imaging and object detection using long-wave infrared imaging. , 2017, Applied optics.

[91]  Bahram Javidi,et al.  Augmented reality three-dimensional object visualization and recognition with axially distributed sensing. , 2016, Optics letters.

[92]  Luc Van Gool,et al.  Efficient Non-Maximum Suppression , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[93]  J. Berger Statistical Decision Theory and Bayesian Analysis , 1988 .

[94]  B. Javidi,et al.  Three-dimensional object visualization and detection in low light illumination using integral imaging. , 2017, Optics letters.

[95]  B Javidi,et al.  Optical image encryption based on input plane and Fourier plane random encoding. , 1995, Optics letters.

[96]  Bahram Javidi,et al.  Fully phase encrypted image processor , 1999 .

[97]  Myungjin Cho,et al.  Three-Dimensional Photon Counting Axially Distributed Image Sensing , 2013, Journal of Display Technology.

[98]  Bahram Javidi,et al.  Photon counting passive 3D image sensing for automatic target recognition. , 2005, Optics express.

[99]  Bahram Javidi,et al.  Full-phase photon-counting double-random-phase encryption. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[100]  Makoto Okui,et al.  Integral Three-Dimensional Television Using a 33-Megapixel Imaging System , 2010, Journal of Display Technology.

[101]  Xu Liu,et al.  VCode—Pervasive Data Transfer Using Video Barcode , 2008, IEEE Transactions on Multimedia.

[102]  Kouichi Nitta,et al.  Assessment of weak light condition in parallel four-step phase-shifting digital holography. , 2013, Applied optics.

[103]  A. Stern,et al.  Experiments With Three-Dimensional Integral Imaging Under Low Light Levels , 2012, IEEE Photonics Journal.

[104]  Bahram Javidi,et al.  3D imaging with axially distributed sensing. , 2009, Optics letters.

[105]  Bahram Javidi,et al.  Nonlinear matched filter based optical correlation. , 1989, Applied optics.

[106]  B. Javidi,et al.  Three-dimensional visualization of objects in scattering medium by use of computational integral imaging. , 2008, Optics express.

[107]  Bahram Javidi,et al.  Three-dimensional recognition of photon-starved events using computational integral imaging and statistical sampling. , 2009, Optics letters.

[108]  Bahram Javidi,et al.  Three-dimensional integral imaging displays using a quick-response encoded elemental image array , 2014 .

[109]  Masahiro Yamaguchi,et al.  Calculation for computer generated hologram using ray-sampling plane. , 2011, Optics express.

[110]  B. Javidi,et al.  Encrypting three-dimensional information with digital holography. , 2000, Applied optics.

[111]  Bahram Javidi,et al.  3D object watermarking by a 3D hidden object. , 2003, Optics express.

[112]  D. Vanderstraeten,et al.  Three-dimensional x-ray laminography as a tool for detection and characterization of BGA package defects , 2002 .

[113]  Philippe Réfrégier,et al.  Maximum-likelihood estimation of an astronomical image from a sequence at low photon levels , 1998 .

[114]  B Javidi,et al.  Enhanced viewing-angle integral imaging by multiple-axis telecentric relay system. , 2007, Optics express.

[115]  Vandana,et al.  Survey of Nearest Neighbor Techniques , 2010, ArXiv.

[116]  Bahram Javidi,et al.  Three-dimensional distortion-tolerant object recognition using photon-counting integral imaging. , 2007, Optics express.

[117]  Bahram Javidi,et al.  Augmented Reality 3D Displays With Micro Integral Imaging , 2015, Journal of Display Technology.

[118]  Abhijit Mahalanobis,et al.  Target-adaptive polarimetric synthetic aperture radar target discrimination using maximum average correlation height filters. , 2006, Applied optics.

[119]  Dhirendra Kumar,et al.  Photon counting imaging and polarized light encoding for secure image verification and hologam watermarking , 2014 .

[120]  O Matoba,et al.  Secure optical memory system with polarization encryption. , 2001, Applied optics.

[121]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[122]  George Michael Morris Scene matching using photon-limited images , 1984 .

[123]  Mark Mohammad Tehranipoor,et al.  Counterfeit Integrated Circuits: A Rising Threat in the Global Semiconductor Supply Chain , 2014, Proceedings of the IEEE.

[124]  Bahram Javidi,et al.  3D photon counting integral imaging with unknown sensor positions. , 2012, Journal of the Optical Society of America. A, Optics, image science, and vision.

[125]  Bahram Javidi,et al.  Three dimensional object recognition with photon counting imagery in the presence of noise. , 2010, Optics express.

[126]  Jie Li,et al.  At-speed delay characterization for IC authentication and Trojan Horse detection , 2008, 2008 IEEE International Workshop on Hardware-Oriented Security and Trust.

[127]  Bahram Javidi,et al.  Integrated circuit authentication using photon-limited x-ray microscopy. , 2016, Optics letters.

[128]  Jingjuan Zhang,et al.  Double random-phase encoding in the Fresnel domain. , 2004, Optics letters.

[129]  B. Hadwen,et al.  The noise performance of electron multiplying charge-coupled devices , 2003 .

[130]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[131]  Boris Skoric,et al.  Quantum-secure authentication of a physical unclonable key , 2014, CLEO 2015.

[132]  Masahiro Yamaguchi,et al.  3D touchable holographic light-field display. , 2016, Applied optics.

[133]  J Arai,et al.  Real-time pickup method for a three-dimensional image based on integral photography. , 1997, Applied optics.

[134]  et al,et al.  The European Photon Imaging Camera on XMM-Newton: The MOS cameras : The MOS cameras , 2000, astro-ph/0011498.

[135]  Bahram Javidi,et al.  Three dimensional visualization by photon counting computational Integral Imaging. , 2008, Optics express.

[136]  Sung-Wook Min,et al.  Wide-viewing-angle integral three-dimensional imaging system by curving a screen and a lens array. , 2005, Applied optics.

[137]  B Javidi,et al.  Encrypted optical memory system using three-dimensional keys in the Fresnel domain. , 1999, Optics letters.

[138]  Myungjin Cho,et al.  Three-dimensional photon counting integral imaging using Bayesian estimation. , 2010, Optics letters.