Automated multi-layer optical design via deep reinforcement learning

Optical multi-layer thin films are widely used in optical and energy applications requiring photonic designs. Engineers often design such structures based on their physical intuition. However, solely relying on human experts can be time-consuming and may lead to sub-optimal designs, especially when the design space is large. In this work, we frame the multi-layer optical design task as a sequence generation problem. A deep sequence generation network is proposed for efficiently generating optical layer sequences. We train the deep sequence generation network with proximal policy optimization to generate multi-layer structures with desired properties. The proposed method is applied to two energy applications. Our algorithm successfully discovered high-performance designs, outperforming structures designed by human experts in task 1, and a state-of-the-art memetic algorithm in task 2.

[1]  A. Tikhonravov,et al.  Application of the needle optimization technique to the design of optical coatings. , 1996, Applied optics.

[2]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[3]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[4]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[5]  P. Peumans,et al.  Broadband optical absorption enhancement through coherent light trapping in thin-film photovoltaic cells. , 2008, Optics express.

[6]  E. Schubert,et al.  Design of multilayer antireflection coatings made from co-sputtered and low-refractive-index materials by genetic algorithm. , 2008, Optics express.

[7]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[8]  Rabi Rabady,et al.  Global optimal design of optical multilayer thin-film filters using particle swarm optimization , 2014 .

[9]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[10]  Marc Abou Anoma,et al.  Passive radiative cooling below ambient air temperature under direct sunlight , 2014, Nature.

[11]  Navdeep Jaitly,et al.  Pointer Networks , 2015, NIPS.

[12]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[13]  Jianfeng Gao,et al.  Deep Reinforcement Learning for Dialogue Generation , 2016, EMNLP.

[14]  L. Guo,et al.  Compact Multilayer Film Structures for Ultrabroadband, Omnidirectional, and Efficient Absorption , 2016 .

[15]  Jing Zhou,et al.  Efficient Thermal–Light Interconversions Based on Optical Topological Transition in the Metal‐Dielectric Multilayered Metamaterials , 2016, Advanced materials.

[16]  Steven J. Byrnes,et al.  Multilayer optical calculations , 2016, 1603.02720.

[17]  M. Soljačić,et al.  Tailoring high-temperature radiation and the resurrection of the incandescent source. , 2016, Nature nanotechnology.

[18]  Sergey Levine,et al.  High-Dimensional Continuous Control Using Generalized Advantage Estimation , 2015, ICLR.

[19]  Shanhui Fan,et al.  A Comprehensive Photonic Approach for Solar Cell Cooling , 2017 .

[20]  S. Fan,et al.  Optimization of Multilayer Optical Films with a Memetic Algorithm and Mixed Integer Programming , 2017 .

[21]  Samy Bengio,et al.  Device Placement Optimization with Reinforcement Learning , 2017, ICML.

[22]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[23]  Samy Bengio,et al.  Neural Combinatorial Optimization with Reinforcement Learning , 2016, ICLR.

[24]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[25]  Le Song,et al.  2 Common Formulation for Greedy Algorithms on Graphs , 2018 .

[26]  Yongmin Liu,et al.  Deep-Learning-Enabled On-Demand Design of Chiral Metamaterials. , 2018, ACS nano.

[27]  S. Fan,et al.  Photonic thermal management of coloured objects , 2018, Nature Communications.

[28]  Olexandr Isayev,et al.  Deep reinforcement learning for de novo drug design , 2017, Science Advances.

[29]  Kyu-Tae Lee,et al.  A Generative Model for Inverse Design of Metamaterials , 2018, Nano letters.

[30]  S. Fan,et al.  Nanophotonic control of thermal radiation for energy applications [Invited]. , 2018, Optics express.

[31]  W. Cai,et al.  A Generative Model for Inverse Design of Metamaterials , 2018, Nano letters.

[32]  Jason Hickey,et al.  Data-driven metasurface discovery , 2018, ACS nano.

[33]  Wojciech M. Czarnecki,et al.  Grandmaster level in StarCraft II using multi-agent reinforcement learning , 2019, Nature.

[34]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[35]  Zongfu Yu,et al.  Training Deep Neural Networks for the Inverse Design of Nanophotonic Structures , 2017, 2019 Conference on Lasers and Electro-Optics (CLEO).

[36]  J. Rho,et al.  Double-deep Q-learning to increase the efficiency of metasurface holograms , 2019, Scientific Reports.

[37]  Trevon Badloe,et al.  Optimisation of colour generation from dielectric nanostructures using reinforcement learning. , 2019, Optics express.

[38]  Xinyun Chen,et al.  Learning to Perform Local Rewriting for Combinatorial Optimization , 2018, NeurIPS.

[39]  Hao Lu,et al.  A Learning-based Iterative Method for Solving Vehicle Routing Problems , 2020, ICLR.

[40]  Quoc V. Le,et al.  Chip Placement with Deep Reinforcement Learning , 2020, ArXiv.

[41]  David Dohan,et al.  Model-based reinforcement learning for biological sequence design , 2020, ICLR.

[42]  Corey T. Matyas,et al.  Optimized Multilayer Structures With Ultrabroadband Near-Perfect Absorption , 2020, IEEE Photonics Journal.