Broadband nonlinear modulation of incoherent light using a transparent optoelectronic neuron array

Nonlinear optical processing of ambient natural light is highly desired in computational imaging and sensing applications. A strong optical nonlinear response that can work under weak broadband incoherent light is essential for this purpose. Here we introduce an optoelectronic nonlinear filter array that can address this emerging need. By merging 2D transparent phototransistors (TPTs) with liquid crystal (LC) modulators, we create an optoelectronic neuron array that allows self-amplitude modulation of spatially incoherent light, achieving a large nonlinear contrast over a broad spectrum at orders-of-magnitude lower intensity than what is achievable in most optical nonlinear materials. For a proof-of-concept demonstration, we fabricated a 10,000-pixel array of optoelectronic neurons, each serving as a nonlinear filter, and experimentally demonstrated an intelligent imaging system that uses the nonlinear response to instantly reduce input glares while retaining the weaker-intensity objects within the field of view of a cellphone camera. This intelligent glare-reduction capability is important for various imaging applications, including autonomous driving, machine vision, and security cameras. Beyond imaging and sensing, this optoelectronic neuron array, with its rapid nonlinear modulation for processing incoherent broadband light, might also find applications in optical computing, where nonlinear activation functions that can work under ambient light conditions are highly sought.

[1]  J. Redwing,et al.  Active pixel sensor matrix based on monolayer MoS2 phototransistor array , 2022, Nature Materials.

[2]  P. Zhou,et al.  A 619-pixel machine vision enhancement chip based on two-dimensional semiconductors , 2022, Science advances.

[3]  Devin J. Dean,et al.  Femtojoule femtosecond all-optical switching in lithium niobate nanophotonics , 2022, Nature Photonics.

[4]  Jian Liu,et al.  Single-layer spatial analog meta-processor for imaging processing , 2022, Nature Communications.

[5]  J. Abe,et al.  Recent advances in low-power-threshold nonlinear photochromic materials. , 2022, Chemical Society reviews.

[6]  X. Duan,et al.  Highly stretchable van der Waals thin films for adaptable and breathable electronic membranes , 2022, Science.

[7]  Xuewu Xu,et al.  High resolution multispectral spatial light modulators based on tunable Fabry-Perot nanocavities , 2021, Light, science & applications.

[8]  Joshua Robertson,et al.  Photonic neuromorphic computing using vertical cavity semiconductor lasers , 2021, ArXiv.

[9]  D. Brunner,et al.  Two-color optically addressed spatial light modulator as a generic spatiotemporal system. , 2021, Chaos.

[10]  Sui Yang,et al.  Direct electrical modulation of second-order optical susceptibility via phase transitions , 2021, Nature Electronics.

[11]  X. Duan,et al.  Two-dimensional van der Waals thin film transistors as active matrix for spatially resolved pressure sensing , 2021, Nano Research.

[12]  Jinlan Wang,et al.  Epitaxial growth of wafer-scale molybdenum disulfide semiconductor single crystals on sapphire , 2021, Nature Nanotechnology.

[13]  A. Shishido,et al.  A Deformable Low-Threshold Optical Limiter with Oligothiophene-Doped Liquid Crystals. , 2021, ACS applied materials & interfaces.

[14]  Z. Zhu,et al.  Optical nonlinearity and non-reciprocal transmission of graphene integrated metasurface , 2021 .

[15]  D. Psaltis,et al.  Scalable optical learning operator , 2020, Nature Computational Science.

[16]  Jeffrey A. Fessler,et al.  Neural network based 3D tracking with a graphene transparent focal stack imaging system , 2020, Nature Communications.

[17]  G. Kumar,et al.  Efficient second-harmonic generation of a high-energy, femtosecond laser pulse in a lithium triborate crystal. , 2020, Optics letters.

[18]  P. Michael,et al.  A conversion guide: solar irradiance and lux illuminance , 2020 .

[19]  Chaoran Huang,et al.  Reconfigurable all-optical nonlinear activation functions for neuromorphic photonics. , 2020, Optics letters.

[20]  T. S. Rasmussen,et al.  All-optical non-linear activation function for neuromorphic photonic computing using semiconductor Fano lasers. , 2020, Optics letters.

[21]  J. Dionne,et al.  Dynamic focusing with high-quality-factor metalenses. , 2020, Nano letters.

[22]  Tao Yan,et al.  In situ optical backpropagation training of diffractive optical neural networks , 2020 .

[23]  Dmitry K. Polyushkin,et al.  Ultrafast machine vision with 2D material neural network image sensors , 2020, Nature.

[24]  Il Yong Chun,et al.  Ranging and light field imaging with transparent photodetectors , 2020 .

[25]  Tairan Liu,et al.  Pathological crystal imaging with single‐shot computational polarized light microscopy , 2019, Journal of biophotonics.

[26]  Yu Huang,et al.  Van der Waals thin-film electronics , 2019, Nature Electronics.

[27]  Takhee Lee,et al.  Intrinsic Optoelectronic Characteristics of MoS2 Phototransistors via a Fully Transparent van der Waals Heterostructure. , 2019, ACS nano.

[28]  Feng Xu,et al.  Fourier-space Diffractive Deep Neural Network. , 2019, Physical review letters.

[29]  Kwanghyun Lee,et al.  Fast Automatic Exposure Adjustment Method for Iris Recognition System , 2019, 2019 11th International Conference on Electronics, Computers and Artificial Intelligence (ECAI).

[30]  Y. Kong,et al.  The Photorefractive Response of Zn and Mo Codoped LiNbO3 in the Visible Region , 2019, Crystals.

[31]  Yue Jiang,et al.  All-optical neural network with nonlinear activation functions , 2019, Optica.

[32]  Shanhui Fan,et al.  Training of Photonic Neural Networks through In Situ Backpropagation , 2018, 2019 Conference on Lasers and Electro-Optics (CLEO).

[33]  V. Chigrinov,et al.  Ferroelectric Liquid Crystals: Physics and Applications , 2016, Crystals.

[34]  Iu,et al.  All-optical neural network with nonlinear activation functions: supplementary material , 2019 .

[35]  Mario Miscuglio,et al.  All-optical nonlinear activation function for photonic neural networks [Invited] , 2018, Optical Materials Express.

[36]  Yu Huang,et al.  Solution-processable 2D semiconductors for high-performance large-area electronics , 2018, Nature.

[37]  Jin-Woo Oh,et al.  Recent progress in organic photorefractive materials , 2018 .

[38]  Yi Luo,et al.  All-optical machine learning using diffractive deep neural networks , 2018, Science.

[39]  Mohammad M Kabir,et al.  Nonlinear Focal Modulation Microscopy. , 2017, Physical review letters.

[40]  Mohammad M Kabir,et al.  Saturated absorption competition microscopy , 2017, 1701.06358.

[41]  S. Goossens,et al.  Broadband image sensor array based on graphene–CMOS integration , 2017, Nature Photonics.

[42]  Dirk Englund,et al.  Deep learning with coherent nanophotonic circuits , 2017, 2017 Fifth Berkeley Symposium on Energy Efficient Electronic Systems & Steep Transistors Workshop (E3S).

[43]  D. Muller,et al.  Large-scale chemical assembly of atomically thin transistors and circuits. , 2016, Nature nanotechnology.

[44]  Todd C. Monson,et al.  Sub-Millisecond Response Time in a Photorefractive Composite Operating under CW Conditions , 2016, Scientific Reports.

[45]  Hai Zhu,et al.  Optomechanical enhancement of doubly resonant 2D optical nonlinearity , 2016, 2016 Conference on Lasers and Electro-Optics (CLEO).

[46]  Yvonne Freeh,et al.  Optical Imaging And Spectroscopy , 2016 .

[47]  C. Chao,et al.  The formation of supramolecular liquid-crystal gels for enhancing the electro-optical properties of twisted nematic liquid crystals , 2015 .

[48]  S. Lau,et al.  Tuning nonlinear optical absorption properties of WS₂ nanosheets. , 2015, Nanoscale.

[49]  H. Schmitzer,et al.  Real-time contrast-enhanced holographic imaging using phase coherent photorefractive quantum wells. , 2015, Optics express.

[50]  F. Xia,et al.  Two-dimensional material nanophotonics , 2014, Nature Photonics.

[51]  Chihaya Adachi,et al.  Large reverse saturable absorption under weak continuous incoherent light. , 2014, Nature materials.

[52]  Neil Collings,et al.  Liquid crystal high-resolution optically addressed spatial light modulator using a nanodimensional chalcogenide photosensor. , 2014, Optics letters.

[53]  Qing Hua Wang,et al.  Electronics and optoelectronics of two-dimensional transition metal dichalcogenides. , 2012, Nature nanotechnology.

[54]  Albert J. P. Theuwissen,et al.  Computational imaging , 2012, 2012 IEEE International Solid-State Circuits Conference.

[55]  Torsten Hegmann,et al.  Nanoparticles in liquid crystals and liquid crystalline nanoparticles. , 2012, Topics in current chemistry.

[56]  Pierre Ambs,et al.  Optical Computing: A 60-Year Adventure , 2010 .

[57]  D. Basko,et al.  Graphene mode-locked ultrafast laser. , 2009, ACS nano.

[58]  Chih-Yu Chao,et al.  Effect of Insulating Nanoparticles Doping on Electro-Optical Characteristics in Nematic Liquid Crystal Cells , 2009 .

[59]  A. Sartori,et al.  A linear-logarithmic CMOS pixel for high dynamic range behavior with fixed-pattern-noise correction and tunable responsivity , 2008, 2008 IEEE Sensors.

[60]  Shunsuke Kobayashi,et al.  Fast Switching of Frequency Modulation Twisted Nematic Liquid Crystal Display Fabricated by Doping Nanoparticles and Its Mechanism , 2004, SPIE OPTO.

[61]  Germano Montemezzani,et al.  Enhanced photorefractive properties of Te-doped Sn2P2S6 , 2003 .

[62]  Grégory Gadret,et al.  Enhanced photorefractive properties of Bi-doped Sn 2 P 2 S 6 , 2003 .

[63]  L. Gonzo,et al.  Novel CMOS image sensor with a 132-dB dynamic range , 2002, IEEE J. Solid State Circuits.

[64]  V. Lampret,et al.  Luminous flux and luminous efficacy of black-body radiation: An analytical approximation , 2002 .

[65]  M. Wood,et al.  Extremely nonlinear photosensitive liquid crystals for image sensing and sensor protection. , 1999, Optics express.

[66]  Martin Schadt,et al.  LIQUID CRYSTAL MATERIALS AND LIQUID CRYSTAL DISPLAYS , 1997 .

[67]  Afshin Partovi,et al.  CARRIER TRANSPORT IN A PHOTOREFRACTIVE MULTIPLE QUANTUM WELL DEVICE , 1996 .

[68]  M. Schadt Liquid Crystals in Information Technology , 1993 .

[69]  Afshin Partovi,et al.  Cr‐doped GaAs/AlGaAs semi‐insulating multiple quantum well photorefractive devices , 1993 .

[70]  Stephen Ducharme,et al.  Altering the photorefractive properties of BaTiO 3 by reduction and oxidation at 650°C , 1986 .

[71]  N. Clark,et al.  Submicrosecond bistable electro‐optic switching in liquid crystals , 1980 .

[72]  Leonard J. Porcello,et al.  Optical data processing and filtering systems , 1960, IRE Trans. Inf. Theory.