Learning Time-Series Data of Industrial Design Optimization using Recurrent Neural Networks

In automotive digital development, 3D shape morphing techniques are used to create new designs in order to match design targets, such as aerodynamic or stylistic requirements. Control-point based shape morphing alters existing geometries either through human user interactions or through computational optimization algorithms that optimize for product performance targets. Shape morphing is typically continuous and results in potentially large data sets of time-series recordings of control point movements. In the present paper, we utilize recurrent neural networks to model such time-series recordings in order to predict future design steps based on the history of currently performed design modifications. To build a data set sufficiently large for the training of neural networks, we use target shape matching optimization as digital analogy for a human user interactive shape modification and to build data sets of control point movements in an automated fashion. Experiments show the potential of recurrent neural networks to successfully learn time-series data representing design changes and to perform single-and multi-step prediction of potential next design steps. We thus demonstrate the feasibility of recurrent neural networks for learning successful design sequences in order to predict promising next design steps in future design tasks.

[1]  Thomas W. Sederberg,et al.  Free-form deformation of solid geometric models , 1986, SIGGRAPH.

[2]  Thomas Bäck,et al.  A Survey of Evolution Strategies , 1991, ICGA.

[3]  Eduardo Sontag,et al.  Turing computability with neural nets , 1991 .

[4]  Thomas Bäck,et al.  An Overview of Evolutionary Algorithms for Parameter Optimization , 1993, Evolutionary Computation.

[5]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[6]  Timothy W. Simpson,et al.  On the Use of Statistics in Design and the Implications for Deterministic Computer Experiments , 1997 .

[7]  Jürgen Schmidhuber,et al.  Learning to forget: continual prediction with LSTM , 1999 .

[8]  J. Samareh Survey of Shape Parameterization Techniques for High-Fidelity Multidisciplinary Shape Optimization , 2001 .

[9]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[10]  David G. Ullman,et al.  Toward the ideal mechanical engineering design support system , 2002 .

[11]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[12]  G. Gary Wang,et al.  Review of Metamodeling Techniques in Support of Engineering Design Optimization , 2007, DAC 2006.

[13]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[14]  Xin Yao,et al.  Target shape design optimization by evolving splines , 2007, 2007 IEEE Congress on Evolutionary Computation.

[15]  Bernhard Sendhoff,et al.  Representing the Change - Free Form Deformation for Evolutionary Design Optimization , 2008, Evolutionary Computation in Practice.

[16]  Stefan Menzel,et al.  A METAMODEL-DRIVEN INTERACTIVE FRAMEWORK FOR A DESIGNER ASSISTANCE SYSTEM , 2010 .

[17]  Yong Jae Lee,et al.  ShadowDraw: real-time user guidance for freehand drawing , 2011, SIGGRAPH 2011.

[18]  Yong Jae Lee,et al.  ShadowDraw: real-time user guidance for freehand drawing , 2011, ACM Trans. Graph..

[19]  Geoffrey E. Hinton,et al.  Generating Text with Recurrent Neural Networks , 2011, ICML.

[20]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[22]  Ali Osman Pektas,et al.  ANN hybrid model versus ARIMA and ARIMAX models of runoff coefficient , 2013 .

[23]  Nitish Srivastava,et al.  Unsupervised Learning of Video Representations using LSTMs , 2015, ICML.

[24]  Zachary Chase Lipton A Critical Review of Recurrent Neural Networks for Sequence Learning , 2015, ArXiv.

[25]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[26]  Steve B. Jiang,et al.  Nonlinear Systems Identification Using Deep Dynamic Neural Networks , 2016, ArXiv.

[27]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[28]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[29]  J. Yosinski,et al.  Time-series Extreme Event Forecasting with Neural Networks at Uber , 2017 .

[30]  Douglas Eck,et al.  A Neural Representation of Sketch Drawings , 2017, ICLR.

[31]  David C. Brown,et al.  Design creativity , 2018, Artificial Intelligence for Engineering Design, Analysis and Manufacturing.

[32]  Jeffrey Heer,et al.  Agency plus automation: Designing artificial intelligence into interactive systems , 2019, Proceedings of the National Academy of Sciences.