Amortized Synthesis of Constrained Configurations Using a Differentiable Surrogate

In design, fabrication, and control problems, we are often faced with the task of synthesis, in which we must generate an object or configuration that satisfies a set of constraints while maximizing one or more objective functions. The synthesis problem is typically characterized by a physical process in which many different realizations may achieve the goal. This many-to-one map presents challenges to the supervised learning of feed-forward synthesis, as the set of viable designs may have a complex structure. In addition, the non-differentiable nature of many physical simulations prevents direct optimization. We address both of these problems with a two-stage neural network architecture that we may consider to be an autoencoder. We first learn the decoder: a differentiable surrogate that approximates the many-to-one physical realization process. We then learn the encoder, which maps from goal to design, while using the fixed decoder to evaluate the quality of the realization. We evaluate the approach on two case studies: extruder path planning in additive manufacturing and constrained soft robot inverse kinematics. We compare our approach to direct optimization of design using the learned surrogate, and to supervised learning of the synthesis problem. We find that our approach produces higher quality solutions than supervised learning, while being competitive in quality with direct optimization, at a greatly reduced computational cost.

[1]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[2]  Sujin Bureerat,et al.  Four-bar linkage path generation through self-adaptive population size teaching-learning based optimization , 2017, Knowl. Based Syst..

[3]  Jiajun Wu,et al.  MarrNet: 3D Shape Reconstruction via 2.5D Sketches , 2017, NIPS.

[4]  Regina Barzilay,et al.  Multi-Objective Molecule Generation using Interpretable Substructures , 2020, ICML.

[5]  Wojciech Matusik,et al.  Computational design of mechanical characters , 2013, ACM Trans. Graph..

[6]  Hadas Kress-Gazit,et al.  Automated Synthesis of Modular Manipulators' Structure and Control for Continuous Tasks around Obstacles , 2020, Robotics: Science and Systems.

[7]  Regina Barzilay,et al.  Deep learning identifies synergistic drug combinations for treating COVID-19 , 2021, Proceedings of the National Academy of Sciences.

[8]  Zhong-Hua Han,et al.  Surrogate-Based Optimization , 2012, Engineering Design Optimization.

[9]  Antonio Moreno Ortiz,et al.  An evolutionary algorithm for path synthesis of mechanisms , 2011 .

[10]  Leifur Leifsson,et al.  Surrogate-Based Methods , 2011, Computational Optimization, Methods and Algorithms.

[11]  Pradeep K. Khosla,et al.  A formulation for task based design of robot manipulators , 1993, Proceedings of 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS '93).

[12]  Xiaoming Wang,et al.  A level set method for structural topology optimization , 2003 .

[13]  Andy J. Keane,et al.  Recent advances in surrogate-based optimization , 2009 .

[14]  Jiajun Wu,et al.  Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[15]  Adam Roberts,et al.  Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Models , 2017, ICLR.

[16]  Bin Li,et al.  Applications of machine learning in drug discovery and development , 2019, Nature Reviews Drug Discovery.

[17]  Michael Carbin,et al.  DiffTune: Optimizing CPU Simulator Parameters with Learned Differentiable Surrogates , 2020, 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO).

[18]  Ryan P. Adams,et al.  Elliptical slice sampling , 2009, AISTATS.

[19]  Alex Hawkins-Hooker,et al.  Generating functional protein variants with variational autoencoders , 2020, bioRxiv.

[20]  Hajime Igarashi,et al.  Topology Optimization Accelerated by Deep Learning , 2019, IEEE Transactions on Magnetics.

[21]  Zhibo Pang,et al.  CoboSkin: Soft Robot Skin With Variable Stiffness for Safer Human–Robot Collaboration , 2021, IEEE Transactions on Industrial Electronics.

[22]  J. A. Cabrera,et al.  Optimal synthesis of mechanisms with genetic algorithms , 2002 .

[23]  Jennifer Listgarten,et al.  Design by adaptive sampling , 2018, ArXiv.

[24]  Jordan B. Pollack,et al.  Evolution of generative design systems for modular physical robots , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[25]  Jiajun Wu,et al.  Learning to Infer and Execute 3D Shape Programs , 2019, ICLR.

[26]  Liu Yang,et al.  Surrogate Losses in Passive and Active Learning , 2012, Electronic Journal of Statistics.

[27]  Rohaizan Ramlan,et al.  An Overview on 3D Printing Technology: Technological, Materials, and Applications , 2019, Procedia Manufacturing.

[28]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[29]  Andrey Ustyuzhanin,et al.  Black-Box Optimization with Local Generative Surrogates , 2020, NeurIPS.

[30]  Chuang Gan,et al.  Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding , 2018, NeurIPS.

[31]  D. Subramanian,et al.  Kinematic synthesis with configuration spaces , 1995 .

[32]  Ryan P. Adams,et al.  Discrete Object Generation with Reversible Inductive Construction , 2019, NeurIPS.

[33]  G. Swaminathan Robot Motion Planning , 2006 .

[34]  Hao Su,et al.  A Point Set Generation Network for 3D Object Reconstruction from a Single Image , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Hadas Kress-Gazit,et al.  Synthesizing Modular Manipulators For Tasks With Time, Obstacle, And Torque Constraints , 2021, ArXiv.

[36]  Sehoon Ha,et al.  Computational Design of Robotic Devices From High-Level Motion Specifications , 2018, IEEE Transactions on Robotics.

[37]  Seid Koric,et al.  Deep learning for topology optimization of 2D metamaterials , 2020 .

[38]  Joel W. Burdick,et al.  Determining task optimal modular robot assembly configurations , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[39]  Shiqiang Zhu,et al.  Self-powered soft robot in the Mariana Trench , 2021, Nature.

[40]  Suleman Asif,et al.  Modelling and path planning for additive manufacturing of continuous fiber composites , 2018 .

[41]  Sergey Levine,et al.  Conservative Objective Models for Effective Offline Model-Based Optimization , 2021, ICML.

[42]  Slawomir Koziel,et al.  Surrogate-Based Modeling and Optimization , 2013 .

[43]  John Bares,et al.  Automated Task-Based Synthesis and Optimization of Field Robots , 1999 .

[44]  O. Sigmund,et al.  Topology optimization approaches , 2013, Structural and Multidisciplinary Optimization.

[45]  Brendan J. Frey,et al.  Generating and designing DNA with deep generative models , 2017, ArXiv.

[46]  M. Shariat Panahi,et al.  Optimal design of four-bar mechanisms using a hybrid multi-objective GA with adaptive local search , 2011 .

[47]  Emilio Frazzoli,et al.  Sampling-based algorithms for optimal motion planning , 2011, Int. J. Robotics Res..

[48]  Roderic A. Grupen,et al.  Robust Reinforcement Learning in Motion Planning , 1993, NIPS.

[49]  Satyandra K. Gupta,et al.  Trajectory Planning for Conformal 3D Printing Using Non-Planar Layers , 2018 .

[50]  Huangchao Yu,et al.  A survey of design methods for material extrusion polymer 3D printing , 2020, Virtual and Physical Prototyping.

[51]  Emma J. Chory,et al.  A Deep Learning Approach to Antibiotic Discovery , 2020, Cell.

[52]  Shane Farritor,et al.  A systems-level modular design approach to field robotics , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[53]  Gattigorla Nagendar,et al.  Neuro-IoU: Learning a Surrogate Loss for Semantic Segmentation , 2018, BMVC.

[54]  Nikos D. Lagaros,et al.  Accelerated topology optimization by means of deep learning , 2020, Structural and Multidisciplinary Optimization.

[55]  Joel Nothman,et al.  SciPy 1.0-Fundamental Algorithms for Scientific Computing in Python , 2019, ArXiv.

[56]  Ekta Singla,et al.  An Optimal Architectural Design for Unconventional Modular Reconfigurable Manipulation System , 2020 .

[57]  Clara Fannjiang,et al.  Autofocused oracles for model-based design , 2020, NeurIPS.

[58]  Olympia Roeva,et al.  Real-World Applications of Genetic Algorithms , 2012 .

[59]  M. Heinkenschloss,et al.  Large-Scale PDE-Constrained Optimization: An Introduction , 2003 .

[60]  Ron Alterovitz,et al.  Asymptotically Optimal Design of Piecewise Cylindrical Robots using Motion Planning , 2017, Robotics: Science and Systems.

[61]  Yonggyun Yu,et al.  Deep learning for topology optimization design , 2018, ArXiv.

[62]  Yurii Nesterov,et al.  Lectures on Convex Optimization , 2018 .

[63]  Sergey Levine,et al.  Offline Model-Based Optimization via Normalized Maximum Likelihood Estimation , 2021, ICLR.

[64]  Thomas Blaschke,et al.  The rise of deep learning in drug discovery. , 2018, Drug discovery today.

[65]  Mohammad Biglarbegian,et al.  A memetic algorithm approach for solving the task-based configuration optimization problem in serial modular and reconfigurable robots , 2014, Robotica.

[66]  Hong Xiao,et al.  An Efficient and Adaptable Path Planning Algorithm for Automated Fiber Placement Based on Meshing and Multi Guidelines , 2020, Materials.

[67]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[68]  Alán Aspuru-Guzik,et al.  Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules , 2016, ACS central science.

[69]  Jiri Matas,et al.  Learning Surrogates via Deep Embedding , 2020, ECCV.

[70]  A. McCallum,et al.  Materials Synthesis Insights from Scientific Literature via Text Extraction and Machine Learning , 2017 .

[71]  Masashi Sugiyama,et al.  Calibrated Surrogate Losses for Adversarially Robust Classification , 2020, COLT.

[72]  T. Nanayakkara,et al.  Soft Robotics Technologies to Address Shortcomings in Today ’ s Minimally Invasive Surgery : The STIFF-FLOP Approach , 2014 .

[73]  V. Bhuvaneswari,et al.  Deep learning for material synthesis and manufacturing systems: A review , 2021, Materials Today: Proceedings.

[74]  D. Rus,et al.  Design, fabrication and control of soft robots , 2015, Nature.

[75]  Jia Deng,et al.  A Unified Framework of Surrogate Loss by Refactoring and Interpolation , 2020, ECCV.

[76]  Lars Schmidt-Thieme,et al.  Learning Surrogate Losses , 2019, ArXiv.

[77]  James Zou,et al.  Feedback GAN for DNA optimizes protein functions , 2019, Nature Machine Intelligence.

[78]  Sarosh Patel,et al.  Task based synthesis of serial manipulators , 2015, Journal of advanced research.

[79]  Milan Sonka,et al.  Robust Deep AUC Maximization: A New Surrogate Loss and Empirical Studies on Medical Image Classification , 2020, ArXiv.

[80]  Robert C. Bolles,et al.  Parametric Correspondence and Chamfer Matching: Two New Techniques for Image Matching , 1977, IJCAI.

[81]  Atharv Bhosekar,et al.  Advances in surrogate based modeling, feasibility analysis, and optimization: A review , 2018, Comput. Chem. Eng..

[82]  Hadas Kress-Gazit,et al.  Task-Based Design of Ad-hoc Modular Manipulators , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[83]  Baining Guo,et al.  Motion-guided mechanical toy modeling , 2012, ACM Trans. Graph..

[84]  Olga Kononova,et al.  Semi-supervised machine-learning classification of materials synthesis procedures , 2019, npj Computational Materials.

[85]  Chi K. Tse,et al.  Trajectory planning for 3D printing: A revisit to traveling salesman problem , 2016, 2016 2nd International Conference on Control, Automation and Robotics (ICCAR).

[86]  Dmitry Vetrov,et al.  Entangled Conditional Adversarial Autoencoder for de Novo Drug Discovery. , 2018, Molecular pharmaceutics.

[87]  Howie Choset,et al.  Task-Specific Manipulator Design and Trajectory Synthesis , 2019, IEEE Robotics and Automation Letters.

[88]  Ersin Yumer,et al.  FAME: 3D Shape Generation via Functionality-Aware Model Evolution , 2022, IEEE Transactions on Visualization and Computer Graphics.

[89]  Jennifer Listgarten,et al.  Conditioning by adaptive sampling for robust design , 2019, ICML.

[90]  E. J. van Henten,et al.  Optimal manipulator design for a cucumber harvesting robot , 2009 .

[91]  Wan Kyun Chung,et al.  Task based design of modular robot manipulator using efficient genetic algorithm , 1997, Proceedings of International Conference on Robotics and Automation.

[92]  Ryan P. Adams,et al.  Amortized Finite Element Analysis for Fast PDE-Constrained Optimization , 2020, ICML.

[93]  Myle Ott,et al.  Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences , 2019, Proceedings of the National Academy of Sciences.

[94]  Connor W. Coley,et al.  Machine Learning in Computer-Aided Synthesis Planning. , 2018, Accounts of chemical research.

[95]  Sehoon Ha,et al.  Task-based limb optimization for legged robots , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[96]  Jonathan P. How,et al.  Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[97]  M. Bendsøe,et al.  Topology Optimization: "Theory, Methods, And Applications" , 2011 .