Inverse design of grating couplers using the policy gradient method from reinforcement learning

Abstract We present a proof-of-concept technique for the inverse design of electromagnetic devices motivated by the policy gradient method in reinforcement learning, named PHORCED (PHotonic Optimization using REINFORCE Criteria for Enhanced Design). This technique uses a probabilistic generative neural network interfaced with an electromagnetic solver to assist in the design of photonic devices, such as grating couplers. We show that PHORCED obtains better performing grating coupler designs than local gradient-based inverse design via the adjoint method, while potentially providing faster convergence over competing state-of-the-art generative methods. As a further example of the benefits of this method, we implement transfer learning with PHORCED, demonstrating that a neural network trained to optimize 8° grating couplers can then be re-trained on grating couplers with alternate scattering angles while requiring >10× fewer simulations than control cases.

[1]  William P. Wardley,et al.  Machine Learning-Based Diffractive Image Analysis with Subwavelength Resolution , 2021, ACS Photonics.

[2]  Devesh K. Jha,et al.  Deep Neural Networks for Inverse Design of Nanophotonic Devices , 2021, Journal of Lightwave Technology.

[3]  Dries Vercruysse,et al.  Analytical level set fabrication constraints for inverse design , 2019, Scientific Reports.

[4]  Steven G. Johnson,et al.  Active learning of deep surrogates for PDEs: application to metasurface design , 2020, npj Computational Materials.

[5]  Ken-ichi Kawarabayashi,et al.  Coherent Ising Machine - Optical Neural Network Operating at the Quantum Limit - , 2018, 2018 Conference on Lasers and Electro-Optics Pacific Rim (CLEO-PR).

[6]  Alexander Y. Piggott,et al.  Inverse design and demonstration of a compact and broadband on-chip wavelength demultiplexer , 2015, Nature Photonics.

[7]  Ming C. Wu,et al.  Hierarchical Design and Optimization of Silicon Photonics , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[8]  Ian A. D. Williamson,et al.  Inverse design of photonic crystals through automatic differentiation , 2020, ACS Photonics.

[9]  Shanhui Fan,et al.  Wave physics as an analog recurrent neural network , 2019, Science Advances.

[10]  C. Wright,et al.  Photonics for artificial intelligence and neuromorphic computing , 2020, ArXiv.

[11]  Xiaoqun Cao,et al.  Solving Partial Differential Equations Using Deep Learning and Physical Constraints , 2020, Applied Sciences.

[12]  George Panotopoulos,et al.  Detachable 1x8 single mode optical interface for DWDM microring silicon photonic transceivers , 2020, OPTO.

[13]  Dries Vercruysse,et al.  Fully-automated optimization of grating couplers. , 2017, Optics express.

[14]  Nan Zhang,et al.  Very sharp adiabatic bends based on an inverse design. , 2018, Optics letters.

[15]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[16]  Jiaqi Jiang,et al.  Multiobjective and categorical global optimization of photonic structures based on ResNet generative neural networks , 2020, ArXiv.

[17]  E. Yablonovitch,et al.  Broadband mirrors with >99% reflectivity for thermophotovoltaic power conversion , 2021, Defense + Commercial Sensing.

[18]  Raymond G. Beausoleil,et al.  Adjoint Optimization of Efficient CMOS-Compatible Si-SiN Vertical Grating Couplers for DWDM Applications , 2020, Journal of Lightwave Technology.

[19]  Ole Sigmund,et al.  Topology optimized mode multiplexing in silicon-on-insulator photonic wire waveguides. , 2016, Optics express.

[20]  Ken-ichi Kawarabayashi,et al.  A coherent Ising machine for 2000-node optimization problems , 2016, Science.

[21]  Carsten Rockstuhl,et al.  Inverse Design of Nanophotonic Devices with Structural Integrity , 2020 .

[22]  Jesse Lu,et al.  Objective-first design of high-efficiency, small-footprint couplers between arbitrary nanophotonic waveguide modes. , 2012, Optics express.

[23]  Xi Chen,et al.  Evolution Strategies as a Scalable Alternative to Reinforcement Learning , 2017, ArXiv.

[24]  Shlomo Ruschin,et al.  S-matrix absolute optimization method for a perfect vertical waveguide grating coupler. , 2019, Optics express.

[25]  Flavian Vasile,et al.  Neural Generative Models for Global Optimization with Gradients , 2018, ArXiv.

[26]  Juerg Leuthold,et al.  Perpendicular Grating Coupler Based on a Blazed Antiback-Reflection Structure , 2017, Journal of Lightwave Technology.

[27]  H. Ghraieb,et al.  Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows , 2020, Physical Review Fluids.

[28]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[29]  Wei Ma,et al.  Deep learning for the design of photonic structures , 2020, Nature Photonics.

[30]  D. Vercruysse,et al.  Inverse-Designed Photonic Crystal Circuits for Optical Beam Steering , 2021, ACS Photonics.

[31]  Jelena Vucković,et al.  Inverse design in nanophotonics , 2018, Nature Photonics.

[32]  Alec M. Hammond,et al.  Designing integrated photonic devices using artificial neural networks. , 2018, Optics express.

[33]  Eli Yablonovitch,et al.  Adjoint shape optimization applied to electromagnetic design. , 2013, Optics express.

[34]  Siegfried Janz,et al.  Design of fully apodized and perfectly vertical surface grating couplers using machine learning optimization , 2021, OPTO.

[35]  Dirk Englund,et al.  Deep learning with coherent nanophotonic circuits , 2017, 2017 Fifth Berkeley Symposium on Energy Efficient Electronic Systems & Steep Transistors Workshop (E3S).

[36]  A. Boes,et al.  11 TOPS photonic convolutional accelerator for optical neural networks , 2021, Nature.

[37]  Ole Sigmund,et al.  Time domain topology optimization of 3D nanophotonic devices , 2014 .

[38]  Jonathan A. Fan,et al.  Global optimization of dielectric metasurfaces using a physics-driven neural network , 2019, Nano letters.

[39]  L. Dal Negro,et al.  Physics-informed neural networks for inverse problems in nano-optics and metamaterials. , 2019, Optics express.

[40]  S. Fan,et al.  Inverse Design of Lightweight Broadband Reflector for Relativistic Lightsail Propulsion , 2020, ACS Photonics.

[41]  Ravi Hegde,et al.  Sample-efficient deep learning for accelerating photonic inverse design , 2021 .

[42]  Zin Lin,et al.  Inverse design of compact multimode cavity couplers. , 2018, Optics express.

[43]  Steven G. Johnson,et al.  Physics-informed neural networks with hard constraints for inverse design , 2021, SIAM J. Sci. Comput..

[44]  Raymond Beausoleil,et al.  Adjoint-method-inspired grating couplers for CWDM O-band applications. , 2020, Optics express.

[45]  D. Melati,et al.  Design of Compact and Efficient Silicon Photonic Micro Antennas With Perfectly Vertical Emission , 2020, IEEE Journal of Selected Topics in Quantum Electronics.

[46]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[47]  Gregory R. Steinbrecher,et al.  Quantum transport simulations in a programmable nanophotonic processor , 2015, Nature Photonics.

[48]  Ming C. Wu,et al.  Inverse design optimization for efficient coupling of an electrically injected optical antenna-LED to a single-mode waveguide. , 2019, Optics express.

[49]  D. Vercruysse,et al.  Inverse-Designed Photonic Crystal Devices for Optical Beam Steering , 2021 .

[50]  Yu Li,et al.  Parameter extraction and inverse design of semiconductor lasers based on the deep learning and particle swarm optimization method. , 2020, Optics express.

[51]  Steven G. Johnson,et al.  End-to-end nanophotonic inverse design for imaging and polarimetry , 2020 .

[52]  Eli Yablonovitch,et al.  Inverse design of near unity efficiency perfectly vertical grating couplers. , 2017, Optics express.

[53]  Shanhui Fan,et al.  Adjoint Method and Inverse Design for Nonlinear Nanophotonic Devices , 2018, ACS Photonics.

[54]  Lei Ying,et al.  Nanophotonic media for artificial neural inference , 2018, Photonics Research.

[55]  Jonathan A. Fan,et al.  Simulator-based training of generative neural networks for the inverse design of metasurfaces , 2019, Nanophotonics.

[56]  Olivier Fercoq,et al.  Improving Evolutionary Strategies with Generative Neural Networks , 2019, ArXiv.

[57]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[58]  Eli Yablonovitch,et al.  Leveraging continuous material averaging for inverse electromagnetic design. , 2017, Optics express.

[59]  Ronald J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[60]  Trevon Badloe,et al.  Deep learning enabled inverse design in nanophotonics , 2020, Nanophotonics.

[61]  Steven G. Johnson,et al.  Inverse Designed Metalenses with Extended Depth of Focus , 2020, 2020 Conference on Lasers and Electro-Optics (CLEO).

[62]  Winnie N. Ye,et al.  An Open-Source Artificial Neural Network Model for Polarization-Insensitive Silicon-on-Insulator Subwavelength Grating Couplers , 2019, IEEE Journal of Selected Topics in Quantum Electronics.

[63]  Li Jing,et al.  Migrating Knowledge between Physical Scenarios based on Artificial Neural Networks , 2018, ACS Photonics.

[64]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Tutorial , 2016, ArXiv.

[65]  Régis Duvigneau,et al.  Global optimization of metasurface designs using statistical learning methods , 2019, Scientific Reports.

[66]  Logan Su,et al.  Data-driven acceleration of photonic simulations , 2019, Scientific Reports.

[67]  Jiaqi Jiang,et al.  Deep neural networks for the evaluation and design of photonic devices , 2020, Nature Reviews Materials.

[68]  D. Melati,et al.  Mapping the global design space of integrated photonic components using machine learning pattern recognition , 2018 .

[69]  Ole Sigmund,et al.  Topology optimization for nano‐photonics , 2011 .

[70]  Ravi S. Hegde,et al.  Deep learning: a new tool for photonic nanostructure design , 2020, Nanoscale advances.

[71]  Toshiaki Koike-Akino,et al.  Deep Neural Network Inverse Design of Integrated Photonic Power Splitters , 2019, Scientific Reports.