Learned rotationally symmetric diffractive achromat for full-spectrum computational imaging

Diffractive achromats (DAs) promise ultra-thin and light-weight form factors for full-color computational imaging systems. However, designing DAs with the optimal optical transfer function (OTF) distribution suitable for image reconstruction algorithms has been a difficult challenge. Emerging end-to-end optimization paradigms of diffractive optics and processing algorithms have achieved impressive results, but these approaches require immense computational resources and solve non-convex inverse problems with millions of parameters. Here, we propose a learned rotational symmetric DA design using a concentric ring decomposition that reduces the computational complexity and memory requirements by one order of magnitude compared with conventional end-to-end optimization procedures, which simplifies the optimization significantly. With this approach, we realize the joint learning of a DA with an aperture size of 8 mm and an image recovery neural network, i.e., Res-Unet, in an end-to-end manner across the full visible spectrum (429–699 nm). The peak signal-to-noise ratio of the recovered images of our learned DA is 1.3 dB higher than that of DAs designed by conventional sequential approaches. This is because the learned DA exhibits higher amplitudes of the OTF at high frequencies over the full spectrum. We fabricate the learned DA using imprinting lithography. Experiments show that it resolves both fine details and color fidelity of diverse real-world scenes under natural illumination. The proposed design paradigm paves the way for incorporating DAs for thinner, lighter, and more compact full-spectrum imaging systems.

[1]  Berardi Sensale-Rodriguez,et al.  A Computational Design Framework for Efficient, Fabrication Error-Tolerant, Planar THz Diffractive Optical Elements , 2019, Scientific Reports.

[2]  A. Ozcan,et al.  On the use of deep learning for computational imaging , 2019, Optica.

[3]  Yifan Peng,et al.  Focal sweep imaging with multi-focal diffractive optics , 2018, 2018 IEEE International Conference on Computational Photography (ICCP).

[4]  Brian Kulis,et al.  W-Net: A Deep Model for Fully Unsupervised Image Segmentation , 2017, ArXiv.

[5]  Wolfgang Heidrich,et al.  High-quality computational imaging through simple lenses , 2013, TOGS.

[6]  Wolfgang Heidrich,et al.  Encoded diffractive optics for full-spectrum computational imaging , 2016, Scientific Reports.

[7]  Wolfgang Heidrich,et al.  Computational imaging using lightweight diffractive-refractive optics. , 2015, Optics express.

[8]  Ashok Veeraraghavan,et al.  PhaseCam3D — Learning Phase Masks for Passive Single View Depth Estimation , 2019, 2019 IEEE International Conference on Computational Photography (ICCP).

[9]  Aswin C. Sankaranarayanan,et al.  FlatCam: Replacing Lenses with Masks and Computation , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[10]  Laura Waller,et al.  DiffuserCam: Lensless Single-exposure 3D Imaging , 2017, ArXiv.

[11]  David G. Stork,et al.  Lensless Ultra-Miniature Imagers Using Odd-Symmetry Spiral Phase Gratings , 2013 .

[12]  Shree K. Nayar,et al.  Generalized Assorted Pixel Camera: Postcapture Control of Resolution, Dynamic Range, and Spectrum , 2010, IEEE Transactions on Image Processing.

[13]  Wangmeng Zuo,et al.  Learning Deep CNN Denoiser Prior for Image Restoration , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Yifan Peng,et al.  Deep Optics for Single-Shot High-Dynamic-Range Imaging , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Tae Hyun Kim,et al.  Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[17]  Arka Majumdar,et al.  Varifocal zoom imaging with large area focal length adjustable metalenses , 2018, Optica.

[18]  Stephen P. Boyd,et al.  End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging , 2018, ACM Trans. Graph..

[19]  Raja Giryes,et al.  Depth Estimation From a Single Image Using Deep Learned Phase Coded Mask , 2018, IEEE Transactions on Computational Imaging.

[20]  Yifan Peng,et al.  The diffractive achromat full spectrum computational imaging with diffractive optics , 2016, ACM Trans. Graph..

[21]  Xiong Dun,et al.  Compact snapshot hyperspectral imaging with diffracted rotation , 2019, ACM Trans. Graph..

[22]  Michael S. Brown,et al.  Training-Based Spectral Reconstruction from a Single RGB Image , 2014, ECCV.

[23]  Shree K. Nayar,et al.  PiCam , 2013, ACM Trans. Graph..

[24]  Raja Giryes,et al.  Learned phase coded aperture for the benefit of depth of field extension. , 2018, Optics express.

[25]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[26]  B. Shen,et al.  Broadband imaging with one planar diffractive lens , 2017, Scientific Reports.

[27]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Adam S. Backer,et al.  Optimal point spread function design for 3D imaging. , 2014, Physical review letters.

[29]  Fred A. Kruse,et al.  The Spectral Image Processing System (SIPS) - Interactive visualization and analysis of imaging spectrometer data , 1993 .

[30]  Rajesh Menon,et al.  Large-area, high-numerical-aperture multi-level diffractive lens via inverse design , 2020, 2001.06523.

[31]  Arka Majumdar,et al.  Metasurface optics for full-color computational imaging , 2018, Science Advances.

[32]  Joseph E. Ford,et al.  Gigapixel Imaging with the AWARE Multiscale Camera , 2012 .

[33]  Gordon Wetzstein,et al.  Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification , 2018, Scientific Reports.

[34]  E. Samei,et al.  A method for measuring the presampled MTF of digital radiographic systems using an edge test device. , 1998, Medical physics.

[35]  Ayan Chakrabarti,et al.  Statistics of real-world hyperspectral images , 2011, CVPR 2011.

[36]  Rajesh Menon,et al.  Extreme-depth-of-focus imaging with a flat lens , 2020 .

[37]  G. Whitesides,et al.  Soft Lithography. , 1998, Angewandte Chemie.

[38]  Rajesh Menon,et al.  Imaging with flat optics: metalenses or diffractive lenses? , 2019, Optica.

[39]  Yifan Peng,et al.  Learned large field-of-view imaging with thin-plate optics , 2019, ACM Trans. Graph..

[40]  Rajesh Menon,et al.  Chromatic-aberration-corrected diffractive lenses for ultra-broadband focusing , 2016, Scientific Reports.

[41]  Laura Waller,et al.  Learned reconstructions for practical mask-based lensless imaging , 2019, Optics express.

[42]  Shuai Li,et al.  Lensless computational imaging through deep learning , 2017, ArXiv.

[43]  Gordon Wetzstein,et al.  Deep Optics for Monocular Depth Estimation and 3D Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).