Efficient Navigation of Active Particles in an Unseen Environment via Deep Reinforcement Learning

Equipping active particles with intelligence such that they can efficiently navigate in an unknown complex environment is essential for emerging applications like precision surgery and targeted drug delivery. Here we develop a deep reinforcement learning algorithm that can train active particles to navigate in environments with random obstacles. Through numerical experiments, we show that the trained particle agent learns to make navigation decision regarding both obstacle avoidance and travel time minimization, relying only on local pixel-level sensory inputs but not on pre-knowledge of the entire environment. In unseen complex obstacle environments, the trained particle agent can navigate nearly optimally in arbitrarily long distance nearly optimally at a fixed computational cost. This study illustrates the potentials of employing artificial intelligence to bridge the gap between active particle engineering and emerging real-world applications.

[1]  T. Mallouk,et al.  Powering nanorobots. , 2009, Scientific American.

[2]  Joseph Wang,et al.  Micro/nanorobots for biomedicine: Delivery, surgery, sensing, and detoxification , 2017, Science Robotics.

[3]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[4]  Aldo A. Faisal,et al.  The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care , 2018, Nature Medicine.

[5]  Joseph Wang,et al.  Rocket Science at the Nanoscale. , 2016, ACS nano.

[6]  Samuel Sánchez,et al.  Chemically powered micro- and nanomotors. , 2015, Angewandte Chemie.

[7]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[8]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[9]  Syn Schmitt,et al.  External control strategies for self-propelled particles: Optimizing navigational efficiency in the presence of limited resources. , 2016, Physical review. E.

[10]  Michael A Bevan,et al.  Optimal Feedback Controlled Assembly of Perfect Crystals. , 2016, ACS nano.

[11]  Michael A Bevan,et al.  Interfacial colloidal rod dynamics: Coefficients, simulations, and analysis. , 2017, The Journal of chemical physics.

[12]  H. Löwen,et al.  Dynamics of a Brownian circle swimmer. , 2008, Physical review. E, Statistical, nonlinear, and soft matter physics.

[13]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[14]  Xi-Qiao Feng,et al.  Collective oscillation in dense suspension of self-propelled chiral rods. , 2019, Soft matter.

[15]  Richard N. Zare,et al.  Optimizing Chemical Reactions with Deep Reinforcement Learning , 2017, ACS central science.

[16]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[17]  Giorgio Volpe,et al.  The topography of the environment alters the optimal search strategy for active particles , 2017, Proceedings of the National Academy of Sciences.

[18]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[19]  Olexandr Isayev,et al.  Deep reinforcement learning for de novo drug design , 2017, Science Advances.

[20]  Petros Koumoutsakos,et al.  Efficient collective swimming by harnessing vortices through deep reinforcement learning , 2018, Proceedings of the National Academy of Sciences.

[21]  Samuel Sanchez,et al.  Self-Propelled Micromotors for Cleaning Polluted Water , 2013, ACS nano.

[22]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[23]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[24]  Frank Cichos,et al.  Harnessing thermal fluctuations for purposeful activities: the manipulation of single micro-swimmers by adaptive photon nudging , 2013 .

[25]  Ramin Golestanian,et al.  Self-motile colloidal particles: from directed propulsion to random walk. , 2007, Physical review letters.

[26]  Yuguang Yang,et al.  Optimal Navigation of Self-Propelled Colloids. , 2018, ACS nano.

[27]  Benno Liebchen,et al.  Optimal Control Strategies for Active Particle Navigation , 2019 .