A metamodeling approach towards virtual production intelligence

Decision making for competitive production in high-wage countries is getting strongly influenced by computer simulations in order to cope with the fast changing global market demands. These simulations offer the possibility to find better designs within a short time, predict and optimize properties that cannot be measured directly from experiments, and explore new information and parameter ranges in known processes that are not easily accessible experimentally. Virtual prototyping is being successfully introduced in production where new products are virtually designed on the computer before building expensive machine components. This led to an explosion of tools that are being unified into a new notion in production, the virtual production system. This virtual system is based on the unification of real (experiments) and virtual (simulations) worlds, bringing together the scientific knowledge and technical know-how about the process. The outcome is presented by operative tools, namely process maps and visual design apps targeted to get more skilled developers and operators. The primary achievement of this thesis is associated with the generation of process maps and visual design apps that are capable to explore the design space spanned by solutions of complex simulation models with the aid of a fast and frugal model, known as a metamodel. Metamodeling follows Benjamin Franklin’s rule for decision making formulated in 1772 [46], where he defined "Prudential Algebra", to be one of the major ingredients that can be applied in order to obtain one algebraic Pareto-value labelling alternative decisions. Metamodeling techniques define a procedure to analyze and represent complex physical systems using easy to use, fast mathematical designs to create cheap numeric surrogates that describe cause-effect relationships between setting parameters as inputs and criteria like product quality variables as outputs. This dissertation focuses on two main research questions and three achievements. The first research question focuses on how to efficiently generate and validate the approximation quality of a metamodel representing a complex simulation model. The question is investigated by different sampling, interpolation, and validation techniques. The outcome is a novel iterative method that generates a high quality metamodel for outputs with continuous as well as piecewise responsesthe first achievement. It basically combines the Sequential Approximate Optimization (SAO) procedure and the Radial Basis Function network (RBFN). The second research question focuses on how to improve a complex model by replacing the simulation model by a metamodel. This question is investigated by interfacing the RBFN metamodel with several global sensitivity analysis and visualization techniques. The outcome is a "Design-Cockpit" used for Virtual Production Intelligencethe second achievement. Its main advantage is exploring information from complex multidimensional computer simulation models for optimizing the process and improving the know-how as well as the model structure with respect to the relevant properties. Finally, the concept of process maps and visual designs are applied to five different laser manufacturing applications with different parameter domain spaces ranging from two to seven dimensions of parameter space. The results show a great efficiency of process map and a visual design in supporting decision makingthe third achievement. To the author’s best knowledge, no fast interactive process map is conducted in any publication of laser manufacturing process before. Although the process maps and underlying process apps are applied to laser applications, the metamodeling techniques can be applied to almost any economical, ecological, or technical process, where the process itself is described by a black box model giving scalar data.

[1]  E. Stinstra The meta-model approach for simulation-based design optimization , 2006 .

[2]  Hasan Kurtaran,et al.  Crashworthiness design optimization using successive response surface approximations , 2002 .

[3]  Kevin Jiang Introduction , 2013, Nature Medicine.

[4]  Saltelli Andrea,et al.  Global Sensitivity Analysis: The Primer , 2008 .

[5]  Urs Eppelt,et al.  Laser ablation mechanism of transparent dielectrics with picosecond laser pulses , 2012, Laser Damage.

[6]  Art B. Owen,et al.  9 Computer experiments , 1996, Design and analysis of experiments.

[7]  A. Saltelli,et al.  A quantitative model-independent method for global sensitivity analysis of model output , 1999 .

[8]  Hirotaka Nakayama,et al.  Sequential Approximate Multiobjective Optimization Using Computational Intelligence , 2009, Vector Optimization.

[9]  Urs Eppelt,et al.  Advanced Metamodeling Techniques Applied to Multidimensional Applications with Piecewise Responses , 2015, MOD.

[10]  M. Sasena,et al.  Exploration of Metamodeling Sampling Criteria for Constrained Global Optimization , 2002 .

[11]  Urs Eppelt,et al.  Picosecond laser ablation of transparent materials , 2013, Photonics West - Lasers and Applications in Science and Engineering.

[12]  C. D. Boor,et al.  On multivariate polynomial interpolation , 1990 .

[13]  Jack P. C. Kleijnen,et al.  Application-driven sequential designs for simulation experiments: Kriging metamodelling , 2004, J. Oper. Res. Soc..

[14]  Stefano Soatto,et al.  Quick Shift and Kernel Methods for Mode Seeking , 2008, ECCV.

[15]  Stefano Tarantola,et al.  Sensitivity Analysis as an Ingredient of Modeling , 2000 .

[16]  Gerhard Venter,et al.  Review of optimization techniques , 2010 .

[17]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[18]  André I. Khuri,et al.  Response surface methodology , 2010 .

[19]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[20]  Dick den Hertog,et al.  Order of Nonlinearity as a Complexity Measure for Models Generated by Symbolic Regression via Pareto Genetic Programming , 2009, IEEE Transactions on Evolutionary Computation.

[21]  Christian H. Bischof,et al.  The virtual production simulation platform: from collaborative distributed simulation to integrated visual analysis , 2011, Prod. Eng..

[22]  Samuel Gerber,et al.  Data Analysis with the Morse-Smale Complex: The msr Package for R , 2012 .

[23]  Max D. Morris,et al.  Factorial sampling plans for preliminary computational experiments , 1991 .

[24]  Urs Eppelt,et al.  An integrated approach for the knowledge discovery in computer simulation models with a multi-dimensional parameter space , 2016 .

[25]  Helge Toutenburg Versuchsplanung und Modellwahl , 1994 .

[26]  Bernard Grossman,et al.  Noisy Aerodynamic Response and Smooth Approximations in HSCT Design , 1994 .

[27]  Torsten Hermanns,et al.  Analysis and optimal control for free melt flow boundaries in laser cutting with distributed radiation , 2015 .

[28]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[29]  Urs Eppelt,et al.  Sensitivity Analysis of Laser Cutting Based on Metamodeling Approach , 2015 .

[30]  Helwig Hauser,et al.  Visualization and Visual Analysis of Multifaceted Scientific Data: A Survey , 2013, IEEE Transactions on Visualization and Computer Graphics.

[31]  Jack P. C. Kleijnen,et al.  A User's Guide to the Brave New World of Designing Simulation Experiments. State-of-the-Art Review , 2005 .

[32]  Luis Mateus Rocha,et al.  Singular value decomposition and principal component analysis , 2003 .

[33]  Daniel Schilberg Architektur eines Datenintegrators zur durchgängigen Kopplung von verteilten numerischen Simulationen , 2010 .

[34]  Songqing Shan,et al.  Introducing Rough Set for Design Space Exploration and Optimization , 2003, DAC 2003.

[35]  W. Hoeffding A Class of Statistics with Asymptotically Normal Distribution , 1948 .

[36]  Rudolf Reinhard,et al.  How Virtual Production Intelligence Can Improve Laser-Cutting Planning Processes , 2016 .

[37]  David Levin,et al.  The approximation power of moving least-squares , 1998, Math. Comput..

[38]  Peter Nyhuis,et al.  Fundamentals of Production Logistics , 2009 .

[39]  Bernard Yannou,et al.  Metamodeling of Combined Discrete/Continuous Responses , 2001 .

[40]  Mansooreh Mollaghasemi,et al.  The development of a methodology for the use of neural networks and simulation modeling in system design , 1999, WSC '99.

[41]  Alfred Inselberg,et al.  Parallel coordinates: a tool for visualizing multi-dimensional geometry , 1990, Proceedings of the First IEEE Conference on Visualization: Visualization `90.

[42]  Markus Nießen,et al.  Optimization of partial differential equations for minimizing the roughness of laser cutting surfaces , 2010 .

[43]  Herbert A. Simon,et al.  Applications of machine learning and rule induction , 1995, CACM.

[44]  T. J. Mitchell,et al.  Exploratory designs for computational experiments , 1995 .

[45]  Daniel M. Dunlavy,et al.  Formulations for Surrogate-Based Optimization with Data Fit, Multifidelity, and Reduced-Order Models , 2006 .

[46]  R. Kuehl Design of Experiments: Statistical Principles of Research Design and Analysis , 1999 .

[47]  Patrick J. Roache,et al.  Verification and Validation in Computational Science and Engineering , 1998 .

[48]  E. Buckingham On Physically Similar Systems; Illustrations of the Use of Dimensional Equations , 1914 .

[49]  Russell R. Barton,et al.  A review on design, modeling and applications of computer experiments , 2006 .

[50]  Rudolf Reinhard,et al.  The Contribution of Virtual Production Intelligence to Laser Cutting Planning Processes , 2014 .

[51]  Miloš Madić,et al.  EXPERIMENTAL INVESTIGATIONS OF CO2 LASER CUT QUALITY: A REVIEW , 2011 .

[52]  D. L. Glizmanenko,et al.  Gas Welding and Cutting , 1982 .

[53]  Jesse Freeman,et al.  in Morse theory, , 1999 .

[54]  K. Nomizu Affine Differential Geometry , 1994 .

[55]  Urs Eppelt,et al.  Simulation of Laser Cutting , 2009 .

[56]  L. Schmit,et al.  Some Approximation Concepts for Structural Synthesis , 1974 .

[57]  Edward R. Tufte,et al.  Envisioning Information , 1990 .

[58]  Timothy W. Simpson,et al.  Analysis of support vector regression for approximation of complex engineering analyses , 2003, DAC 2003.

[59]  Thomas M. Cover,et al.  Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition , 1965, IEEE Trans. Electron. Comput..

[60]  Laurent Cordier,et al.  Proper Orthogonal Decomposition: an overview , 2008 .

[61]  Thomas J. Santner,et al.  Design and analysis of computer experiments , 1998 .

[62]  PJ Pieter Heres,et al.  Robust and Efficient Krylov Subspace Methods for Model Order Reduction , 2005 .

[63]  Valerio Pascucci,et al.  Visual Exploration of High Dimensional Scalar Functions , 2010, IEEE Transactions on Visualization and Computer Graphics.

[64]  Raphael T. Haftka,et al.  Performance Estimate and Simultaneous Application of Multiple Surrogates , 2006 .

[65]  Caren Marzban,et al.  Variance-Based Sensitivity Analysis: An Illustration on the Lorenz'63 Model , 2013 .

[66]  A. R. Zinsmeister,et al.  Statistics for Experimenters: An Introduction to Design, Data Analysis, and Model Building, by G. E. P. Box, W. G. Hunter, and J. S. Hunter , 1981 .

[67]  Martin Meckesheimer,et al.  A FRAMEWORK FOR METAMODEL-BASED DESIGN: SUBSYSTEM METAMODEL ASSESSMENT AND IMPLEMENTATION ISSUES , 2001 .

[68]  D G Krige,et al.  A statistical approach to some mine valuation and allied problems on the Witwatersrand , 2015 .

[69]  Timothy W. Simpson,et al.  Metamodels for Computer-based Engineering Design: Survey and recommendations , 2001, Engineering with Computers.

[70]  Ronald L. Iman,et al.  A FORTRAN-77 PROGRAM AND USER'S GUIDE FOR THE GENERATION OF LATIN HYPERCUBE AND RANDOM SAMPLES FOR USE WITH COMPUTER MODELS , 1984 .

[71]  S. Rippa,et al.  Numerical Procedures for Surface Fitting of Scattered Data by Radial Functions , 1986 .

[72]  Jeong‐Soo Park Optimal Latin-hypercube designs for computer experiments , 1994 .

[73]  Jack P. C. Kleijnen,et al.  A methodology for the fitting and validation of metamodels in simulation , 2000 .

[74]  Wei Chen,et al.  ROBUST CONCEPT EXPLORATION OF PROPULSION SYSTEMS WITH ENHANCED MODEL APPROXIMATION CAPABILITIES , 2000 .

[75]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[76]  Yoav Benjamini,et al.  Opening the Box of a Boxplot , 1988 .

[77]  Wolfgang Schulz,et al.  Meta-Modelling Techniques Towards Virtual Production Intelligence , 2015 .

[78]  John R. King,et al.  Asymptotic solutions to the Stefan problem with a constant heat source at the moving boundary , 2000, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[79]  Timothy G. Trucano,et al.  Validation Methodology in Computational Fluid Dynamics , 2000 .

[80]  Samuel Eliot Morison,et al.  The papers of Benjamin Franklin , 1960 .

[81]  Jerome Sacks,et al.  Designs for Computer Experiments , 1989 .

[82]  Adel Younis,et al.  Space exploration and region elimination global optimization algorithms for multidisciplinary design optimization , 2010 .

[83]  Cristina H. Amon,et al.  On Adaptive Sampling for Single and Multi-Response Bayesian Surrogate Models , 2006, DAC 2006.

[84]  Michael D. McKay,et al.  Evaluating Prediction Uncertainty , 1995 .

[85]  Sonia Victoria Vivas Prieto Genomic and clinical data correlation for relapse prediction in cancer samples , 2014 .

[86]  T. Simpson,et al.  Comparative studies of metamodeling techniques under multiple modeling criteria , 2000 .

[87]  Wolfgang Schulz,et al.  A free boundary problem related to laser beam fusion cutting: ODE approximation , 1997 .

[88]  J. Dennis,et al.  MANAGING APPROXIMATION MODELS IN OPTIMIZATION , 2007 .

[89]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[90]  Byeongdo Kim,et al.  Comparison study on the accuracy of metamodeling technique for non-convex functions , 2009 .

[91]  Y. Lam,et al.  A hybrid of mode-pursuing sampling method and genetic algorithm for minimization of injection molding warpage , 2010 .

[92]  Cécile Piret,et al.  Analytical and numerical advances in radial basis functions , 2007 .

[93]  Holger Wendland,et al.  Scattered Data Approximation: Conditionally positive definite functions , 2004 .

[94]  Jack P. C. Kleijnen,et al.  Measurement scales and resolution IV designs : A note (Version 3) , 1990 .

[95]  Mathematisch-Naturwissenschaftlichen Fakultat,et al.  Approaches to analyse and interpret biological profile data , 2006 .

[96]  José G. Ramírez,et al.  Understanding Design of Experiments: A Primer for Technologists , 1999, Technometrics.

[97]  Urs Eppelt,et al.  Review on laser drilling I. Fundamentals, modeling, and simulation , 2013 .

[98]  Farrokh Mistree,et al.  Statistical Approximations for Multidisciplinary Design Optimization: The Problem of Size , 1999 .

[99]  Nielen Stander,et al.  Comparing three error criteria for selecting radial basis function network topology , 2009 .

[100]  M. E. Johnson,et al.  Minimax and maximin distance designs , 1990 .

[101]  L. Nilsson,et al.  On polynomial response surfaces and Kriging for use in structural optimization of crashworthiness , 2005 .

[102]  Hirotaka Nakayama,et al.  Simulation-Based Optimization Using Computational Intelligence , 2002 .

[103]  Chao Jiang,et al.  An efficient multi-objective optimization method for black-box functions using sequential approximate technique , 2012, Appl. Soft Comput..

[104]  Gary James Jason,et al.  The Logic of Scientific Discovery , 1988 .

[105]  Vinod Yadava,et al.  Laser beam machining—A review , 2008 .

[106]  R. Haftka,et al.  Multiple surrogates: how cross-validation errors can help us to obtain the best predictor , 2009 .

[107]  J. Mark Introduction to radial basis function networks , 1996 .

[108]  C. F. Jeff Wu,et al.  Experiments , 2021, Wiley Series in Probability and Statistics.

[109]  H. Fang,et al.  Global response approximation with radial basis functions , 2006 .

[110]  Simon Haykin,et al.  Neural networks , 1994 .

[111]  K. Shuler,et al.  Nonlinear sensitivity analysis of multiparameter model systems , 1977 .

[112]  Gene H. Golub,et al.  Calculating the singular values and pseudo-inverse of a matrix , 2007, Milestones in Matrix Computation.

[113]  Noel A Cressie,et al.  Statistics for Spatial Data. , 1992 .

[114]  A. Kaplan,et al.  Laser–oxygen cutting of mild steel: the thermodynamics of the oxidation reaction , 2009 .

[115]  R. Franke Smooth Interpolation of Scattered Data by Local Thin Plate Splines , 1982 .

[116]  Masao Arakawa,et al.  Sequential approximate multi-objective optimization using radial basis function network , 2013 .

[117]  J. Friedman Multivariate adaptive regression splines , 1990 .

[118]  P. Wakeley,et al.  Synthesis , 2013, The Role of Animals in Emerging Viral Diseases.

[119]  Timothy W. Simpson,et al.  Sampling Strategies for Computer Experiments: Design and Analysis , 2001 .

[120]  M. Morse Relations between the critical points of a real function of $n$ independent variables , 1925 .

[121]  Farrokh Mistree,et al.  Kriging Models for Global Approximation in Simulation-Based Multidisciplinary Design Optimization , 2001 .

[122]  Martin S. Fridson,et al.  Trends , 1948, Bankmagazin.

[123]  Cristina H. Amon,et al.  An engineering design methodology with multistage Bayesian surrogates and optimal sampling , 1996 .

[124]  Peter L. Brooks,et al.  Visualizing data , 1997 .

[125]  D. Shahsavani,et al.  Variance-based sensitivity analysis of model outputs using surrogate models , 2011, Environ. Model. Softw..

[126]  D. Dennis,et al.  A statistical method for global optimization , 1992, [Proceedings] 1992 IEEE International Conference on Systems, Man, and Cybernetics.

[127]  Y. Yeun,et al.  Managing approximation models in multiobjective optimization , 2002 .

[128]  L. Mlodinow,et al.  THE GRAND DESIGN , 2010 .

[129]  Genzi Li,et al.  Online and Offline Approximations for Population based Multi-objective Optimization , 2007 .

[130]  Urs Eppelt,et al.  Numerical analysis of laser ablation and damage in glass with multiple picosecond laser pulses. , 2013, Optics express.

[131]  Pierre Goovaerts,et al.  Adaptive Experimental Design Applied to Ergonomics Testing Procedure , 2002, DAC 2002.

[132]  Nasir Bilal Design optimization of the suction manifold of a reciprocating compressor using uncertainty and sensitivity analysis , 2011 .

[133]  Derek J. Pike,et al.  Empirical Model‐building and Response Surfaces. , 1988 .

[134]  Marcus Redhe,et al.  An investigation of structural optimization in crashworthiness design using a stochastic approach , 2004 .

[135]  U. Diwekar Batch Distillation: Simulation, Optimal Design, and Control, Second Edition , 2011 .

[136]  Dirk Gorissen,et al.  Grid-enabled adaptive surrogate modeling for computer aided engineering , 2010 .

[137]  Ann-Britt Ryberg Metamodel-based design optimization: a multidisciplinary approach for automotive structures , 2013 .

[138]  Jay D. Martin,et al.  USE OF ADAPTIVE METAMODELING FOR DESIGN OPTIMIZATION , 2002 .

[139]  Florian Jurecka,et al.  Robust Design Optimization Based on Metamodeling Techniques , 2007 .

[140]  Georges M. Fadel,et al.  Approximating Pareto curves using the hyper-ellipse , 1998 .

[141]  Genichi Taguchi,et al.  Introduction to quality engineering.... , 2014 .

[142]  Min Xie,et al.  A systematic comparison of metamodeling techniques for simulation optimization in Decision Support Systems , 2010, Appl. Soft Comput..

[143]  C Tong,et al.  Toward a more robust variance-based global sensitivity analysis of model outputs , 2007 .

[144]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[145]  Andrea Saltelli,et al.  An effective screening design for sensitivity analysis of large models , 2007, Environ. Model. Softw..

[146]  G. Gary Wang,et al.  Review of Metamodeling Techniques in Support of Engineering Design Optimization , 2007, DAC 2006.

[147]  Manolis Papadrakakis,et al.  Structural optimization using evolution strategies and neural networks , 1998 .

[148]  Urs Eppelt,et al.  Diagnostic and simulation of ps-laser glass cutting , 2012 .

[149]  Richard J. Beckman,et al.  A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code , 2000, Technometrics.

[150]  Randal J. Barnes,et al.  Infill sampling criteria to locate extremes , 1995 .

[151]  Thomas W. Lucas,et al.  Exploring the world of agent-based simulations: simple models, complex analyses , 2002, Proceedings of the Winter Simulation Conference.

[152]  Christian H. Bischof,et al.  Towards a Flexible and Distributed Simulation Platform , 2008, ICCSA.

[153]  Jack P. C. Kleijnen Design and Analysis of Simulation Experiments , 2007 .

[154]  Urmila M. Diwekar,et al.  An efficient sampling technique for off-line quality control , 1997 .

[155]  Benjamin Racine,et al.  CFD-Based Method for Simulation of Marine-Vehicle Maneuvering , 2005 .

[156]  N. Cressie Spatial prediction and ordinary kriging , 1988 .

[157]  Shawn E. Gano,et al.  Update strategies for kriging models used in variable fidelity optimization , 2006 .

[158]  Rudolf Reinhard,et al.  A Framework For Adaptive Data Integration In Digital Production , 2013 .

[159]  Armin Iske,et al.  Hierarchical Nonlinear Approximation for Experimental Design and Statistical Data Fitting , 2007, SIAM J. Sci. Comput..

[160]  J. V. van Wijk,et al.  HyperSlice: visualization of scalar functions of many variables , 1993, VIS '93.

[161]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[162]  Xin Wei,et al.  A new sequential optimal sampling method for radial basis functions , 2012, Appl. Math. Comput..

[163]  C. Micchelli Interpolation of scattered data: Distance matrices and conditionally positive definite functions , 1986 .

[164]  J. I The Design of Experiments , 1936, Nature.

[165]  Dennis K. J. Lin Generating Systematic Supersaturated Designs , 1995 .

[166]  Jon C. Helton,et al.  Survey of sampling-based methods for uncertainty and sensitivity analysis , 2006, Reliab. Eng. Syst. Saf..

[167]  Tobias Meisen,et al.  Application Integration of Simulation Tools Considering Domain Specific Knowledge , 2011, ICEIS.

[168]  Valerio Pascucci,et al.  Persistence-sensitive simplification functions on 2-manifolds , 2006, SCG '06.

[169]  T. Simpson,et al.  Computationally Inexpensive Metamodel Assessment Strategies , 2002 .

[170]  Shapour Azarm,et al.  Maximum Accumulative Error Sampling Strategy for Approximation of Deterministic Engineering Simulations , 2006 .

[171]  V. F. Solinov,et al.  Laser cutting of float glass during production , 2006 .

[172]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[173]  J. Paulo Davim,et al.  Lasers in Manufacturing: Davim/Lasers in Manufacturing , 2013 .

[174]  Kenny Q. Ye,et al.  Algorithmic construction of optimal symmetric Latin hypercube designs , 2000 .

[175]  K. Yamazaki,et al.  Sequential Approximate Optimization using Radial Basis Function network for engineering optimization , 2011 .

[176]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[177]  Urs Eppelt,et al.  Metamodeling of laser cutting , 2015 .

[178]  Oliver Rübel,et al.  Morse–Smale Regression , 2013, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[179]  Shirley Dex,et al.  JR 旅客販売総合システム(マルス)における運用及び管理について , 1991 .