NeuWS: Neural wavefront shaping for guidestar-free imaging through static and dynamic scattering media

Diffraction-limited optical imaging through scattering media has the potential to transform many applications such as airborne and space-based imaging (through the atmosphere), bioimaging (through skin and human tissue), and fiber-based imaging (through fiber bundles). Existing wavefront shaping methods can image through scattering media and other obscurants by optically correcting wavefront aberrations using high-resolution spatial light modulators—but these methods generally require (i) guidestars, (ii) controlled illumination, (iii) point scanning, and/or (iv) statics scenes and aberrations. We propose neural wavefront shaping (NeuWS), a scanning-free wavefront shaping technique that integrates maximum likelihood estimation, measurement modulation, and neural signal representations to reconstruct diffraction-limited images through strong static and dynamic scattering media without guidestars, sparse targets, controlled illumination, nor specialized image sensors. We experimentally demonstrate guidestar-free, wide field-of-view, high-resolution, diffraction-limited imaging of extended, nonsparse, and static/dynamic scenes captured through static/dynamic aberrations.

[1]  Brandon Yushan Feng,et al.  VIINTER: View Interpolation with Implicit Neural Representations of Images , 2022, SIGGRAPH Asia.

[2]  Jiamin Wu,et al.  An integrated imaging sensor for aberration-corrected 3D photography , 2022, Nature.

[3]  L. Waller,et al.  Dynamic Structured Illumination Microscopy with a Neural Space-time Model , 2022, 2022 IEEE International Conference on Computational Photography (ICCP).

[4]  Technion,et al.  Fluorescent wavefront shaping using incoherent iterative phase conjugation , 2022, Optica.

[5]  Aswin C. Sankaranarayanan,et al.  Enhancing Speckle Statistics for Imaging Inside Scattering Media , 2022, Optica.

[6]  Danilo Jimenez Rezende,et al.  From data to functa: Your data point is a function and you can treat it like one , 2022, ICML.

[7]  Shalini De Mello,et al.  Efficient Geometry-aware 3D Generative Adversarial Networks , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Timothy D. Weber,et al.  Roadmap on wavefront shaping and deep imaging in complex media , 2021, Journal of Physics: Photonics.

[9]  Yu Sun,et al.  Recovery of continuous 3D refractive index maps from discrete intensity-only measurements using neural fields , 2021, Nature Machine Intelligence.

[10]  Kazuhiro Kurokawa,et al.  Adaptive optics for high-resolution imaging , 2021, Nature Reviews Methods Primers.

[11]  Brandon Yushan Feng,et al.  SIGNET: Efficient Neural Representation for Light Fields , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[12]  Matthias T. Banet,et al.  3D multi-plane sharpness metric maximization with variable corrective phase screens. , 2021, Applied optics.

[13]  Aydogan Ozcan,et al.  Computational imaging without a computer: seeing through random diffusers at the speed of light , 2021, eLight.

[14]  Marina Alterman,et al.  Imaging with Local Speckle Intensity Correlations: Theory and Practice , 2021, ACM Trans. Graph..

[15]  Maarten H. P. Kole,et al.  Robust adaptive optics for localization microscopy deep in complex tissue , 2021, Nature Communications.

[16]  Qionghai Dai,et al.  Iterative tomography with digital adaptive optics permits hour-long intravital observation of 3D subcellular dynamics at millisecond scale , 2021, Cell.

[17]  A. Jesacher,et al.  Fast holographic scattering compensation for deep tissue biological imaging , 2021, Nature Communications.

[18]  Ellen D. Zhong,et al.  CryoDRGN: Reconstruction of heterogeneous cryo-EM structures using neural networks , 2021, Nature Methods.

[19]  Julien Lozi,et al.  Predictive control for adaptive optics using neural networks , 2021, Journal of Astronomical Telescopes, Instruments, and Systems.

[20]  Tomer Yeminy,et al.  Guidestar-free image-guided wavefront shaping , 2020, Science Advances.

[21]  Gordon Wetzstein,et al.  Implicit Neural Representations with Periodic Activation Functions , 2020, NeurIPS.

[22]  Oliver Cossairt,et al.  WISHED: Wavefront imaging sensor with high resolution and depth ranging , 2020, 2020 IEEE International Conference on Computational Photography (ICCP).

[23]  Pratul P. Srinivasan,et al.  NeRF , 2020, ECCV.

[24]  Sungsam Kang,et al.  Deep optical imaging within complex scattering media , 2020, Nature Reviews Physics.

[25]  M. J. Booth,et al.  Wavefront‐sensorless adaptive optics with a laser‐free spinning disk confocal microscope , 2020, bioRxiv.

[26]  Gerwin Osnabrugge,et al.  Model-based wavefront shaping microscopy. , 2020, Optics letters.

[27]  Laura Waller,et al.  Deep phase decoder: self-calibrating phase microscopy with an untrained deep neural network , 2020, Optica.

[28]  Michael Unser,et al.  Time-Dependent Deep Image Prior for Dynamic MRI , 2019, IEEE Transactions on Medical Imaging.

[29]  Y. Silberberg,et al.  Light focusing through scattering media via linear fluorescence variance maximization, and its application for fluorescence imaging. , 2019, Optics express.

[30]  Jonathan Dong,et al.  Noninvasive light focusing in scattering media using speckle variance optimization , 2019, Optica.

[31]  Manoj Kumar Sharma,et al.  WISH: wavefront imaging sensor with high resolution , 2019, Light: Science & Applications.

[32]  Richard A. Newcombe,et al.  DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Abbie T. Watnik,et al.  Wavefront Sensing in Deep Turbulence , 2018, Optics and Photonics News.

[34]  Ori Katz,et al.  Noninvasive focusing through scattering layers using speckle correlations. , 2018, Optics letters.

[35]  Lei Tian,et al.  Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media , 2018, Optica.

[36]  Ulugbek Kamilov,et al.  Efficient and accurate inversion of multiple scattering with deep learning , 2018, Optics express.

[37]  Hao Li,et al.  Visualizing the Loss Landscape of Neural Nets , 2017, NeurIPS.

[38]  Jin Hyoung Park,et al.  High-resolution adaptive optical imaging within thick scattering media using closed-loop accumulation of single scattering , 2017, Nature Communications.

[39]  Tae Joong Eom,et al.  In vivo study of optical speckle decorrelation time across depths in the mouse brain. , 2017, Biomedical optics express.

[40]  Robert J Zawadzki,et al.  Review of adaptive optics OCT (AO-OCT): principles and applications for retinal imaging [Invited]. , 2017, Biomedical optics express.

[41]  Ioannis N. Papadopoulos,et al.  Scattering compensation by focus scanning holographic aberration probing (F-SHARP) , 2016, Nature Photonics.

[42]  R. Raskar,et al.  All Photons Imaging Through Volumetric Scattering , 2016, Scientific Reports.

[43]  Changhuei Yang,et al.  Guidestar-assisted wavefront-shaping methods for focusing light into biological tissue , 2015, Nature Photonics.

[44]  Michael Unser,et al.  Learning approach to optical tomography , 2015, 1502.01914.

[45]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[46]  Puxiang Lai,et al.  Photoacoustically guided wavefront shaping for enhanced optical focusing in scattering media , 2014, Nature Photonics.

[47]  Martin J. Booth,et al.  Adaptive optical microscopy: the ongoing quest for a perfect image , 2014, Light: Science & Applications.

[48]  M. Fink,et al.  Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations , 2014, Nature Photonics.

[49]  Xiaodong Li,et al.  Phase Retrieval from Coded Diffraction Patterns , 2013, 1310.3240.

[50]  Jacopo Bertolotti,et al.  Non-invasive imaging through opaque scattering layers , 2012, Nature.

[51]  Ivo M Vellekoop,et al.  Digital optical phase conjugation of fluorescence in turbid tissue. , 2012, Applied physics letters.

[52]  G. Lerosey,et al.  Controlling waves in space and time for imaging and focusing in complex media , 2012, Nature Photonics.

[53]  D. Conkey,et al.  Genetic algorithm optimization for focusing through turbid media in noisy environments. , 2012, Optics express.

[54]  Lihong V. Wang,et al.  Time-reversed ultrasonically encoded optical focusing into scattering media , 2010, Nature photonics.

[55]  Demetri Psaltis,et al.  Digital phase conjugation of second harmonic radiation emitted by nanoparticles in turbid media. , 2010, Optics express.

[56]  Tony Wilson,et al.  Image-based adaptive optics for two-photon microscopy. , 2009, Optics letters.

[57]  Abbie E. Tippie,et al.  Phase-error correction for multiple planes using a sharpness metric , 2009 .

[58]  A. Mosk,et al.  Focusing coherent light through opaque strongly scattering media. , 2007, Optics letters.

[59]  R. Muller,et al.  Real-time correction of atmospherically degraded telescope images through image sharpening , 1974 .

[60]  Christopher A. Metzler,et al.  Solving Inverse Problems using Self-Supervised Deep Neural Nets , 2021, OSA Imaging and Applied Optics Congress 2021 (3D, COSI, DH, ISA, pcAOP).

[61]  Ang,et al.  Ultra-high resolution coded wavefront sensor , 2017 .