Knowledge Transfer Through Machine Learning in Aircraft Design

The modern aircraft has evolved to become an important part of our society. Its design is multidisciplinary in nature and is characterized by complex analyses of mutually interdependent disciplines and large search spaces. Machine learning has, historically, played a significant role in aircraft design, primarily by approximating expensive physics-based numerical simulations. In this work, we summarize the current role of machine learning in this application domain, and highlight the opportunity of incorporating recent advances in the field to further its impact. Specifically, regression models (or surrogate models) that represent a major portion of the current efforts are generally built from scratch assuming a zero prior knowledge state, only relying on data from the ongoing target problem of interest. However, due to the incremental nature of design processes, there likely exists relevant knowledge from various related sources that can potentially be leveraged. As such, we present three relatively advanced machine learning technologies that facilitate automatic knowledge transfer in order to improve design performance. Subsequently, we demonstrate the efficacy of one of these technologies, i.e. transfer learning, on two use cases of aircraft engine design yielding noteworthy results. Our aim is to unveil this new application as a well-suited arena for the salient features of knowledge transfer in machine learning to come to the fore, thereby encouraging future research efforts.

[1]  Bernhard Sendhoff,et al.  Multi co-objective evolutionary optimization: Cross surrogate augmentation for computationally expensive problems , 2012, 2012 IEEE Congress on Evolutionary Computation.

[2]  A. Keane,et al.  Evolutionary Optimization of Computationally Expensive Problems via Surrogate Modeling , 2003 .

[3]  Jude Shavlik,et al.  Chapter 11 Transfer Learning , 2009 .

[4]  Gideon S. Mann,et al.  Efficient Transfer Learning Method for Automatic Hyperparameter Tuning , 2014, AISTATS.

[5]  Yolanda Mack,et al.  CFD-based surrogate modeling of liquid rocket engine components via design space refinement and sensitivity assessment , 2007 .

[6]  Qiang Yang,et al.  Adaptive Transfer Learning , 2010, AAAI.

[7]  Yew-Soon Ong,et al.  Evolutionary multitasking in bi-level optimization , 2015 .

[8]  Yew-Soon Ong,et al.  Multifactorial Evolution: Toward Evolutionary Multitasking , 2016, IEEE Transactions on Evolutionary Computation.

[9]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[10]  Anand Pratap Singh,et al.  New Approaches in Turbulence and Transition Modeling Using Data-driven Techniques , 2015 .

[11]  Nateri K. Madavan,et al.  AERODYNAMIC DESIGN USING NEURAL NETWORKS , 2000 .

[12]  Oliver Bandte,et al.  A probabilistic multi-criteria decision making technique for conceptual and preliminary aerospace systems design , 2000 .

[13]  Helge Aagaard Madsen,et al.  Optimization method for wind turbine rotors , 1999 .

[14]  Dimitri N. Mavris,et al.  Pattern Classification of a Civilian Turbofan's State Space for Real-Time Surrogate Modeling , 2013 .

[15]  Edwin V. Bonilla,et al.  Multi-task Gaussian Process Prediction , 2007, NIPS.

[16]  Robert J. Winter Cpt.  Agile Software Development: Principles, Patterns, and Practices , 2014 .

[17]  Angappa Gunasekaran,et al.  Agile Manufacturing: The 21st Century Competitive Strategy , 2001 .

[18]  Chunhui Yang,et al.  Recent developments in finite element analysis for laminated composite plates , 2009 .

[19]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[20]  Shumeet Baluja,et al.  Incorporating a priori Knowledge in Probabilistic-Model Based Optimization , 2006, Scalable Optimization via Probabilistic Modeling.

[21]  Bernhard Sendhoff,et al.  A Unified Framework for Symbiosis of Evolutionary Mechanisms with Application to Water Clusters Potential Model Design , 2012, IEEE Computational Intelligence Magazine.

[22]  G. Gary Wang,et al.  Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions , 2010 .

[23]  Bernhard Sendhoff,et al.  Generalizing Surrogate-Assisted Evolutionary Computation , 2010, IEEE Transactions on Evolutionary Computation.

[24]  Yew-Soon Ong,et al.  Evolutionary Multitasking: A Computer Science View of Cognitive Multitasking , 2016, Cognitive Computation.

[25]  C.J.H. Mann,et al.  Handbook of Approximation: Algorithms and Metaheuristics , 2008 .

[26]  Timothy W. Simpson,et al.  Design and Analysis of Computer Experiments in Multidisciplinary Design Optimization: A Review of How Far We Have Come - Or Not , 2008 .

[27]  Massimiliano Vasile,et al.  Robust design of a re-entry unmanned space vehicle by multi-fidelity evolution control , 2011, GECCO '11.

[28]  Raphael T. Haftka,et al.  Surrogate-based Analysis and Optimization , 2005 .

[29]  R. Choudharya,et al.  Analytic target cascading in simulation-based building design , 2004 .

[30]  Karthik Duraisamy,et al.  A paradigm for data-driven predictive modeling using field inversion and machine learning , 2016, J. Comput. Phys..

[31]  Andy J. Keane,et al.  Efficient Multipoint Aerodynamic Design Optimization Via Cokriging , 2011 .

[32]  Yew-Soon Ong,et al.  Towards a new Praxis in optinformatics targeting knowledge re-use in evolutionary computation: simultaneous problem learning and optimization , 2016, Evolutionary Intelligence.

[33]  Ivor W. Tsang,et al.  The Emerging "Big Dimensionality" , 2014, IEEE Computational Intelligence Magazine.

[34]  Dimitri N. Mavris,et al.  INVENT Surrogate Modeling and Optimization of Transient Thermal Responses , 2012 .

[35]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[36]  Massimiliano Vasile,et al.  Robust Design of a Reentry Unmanned Space Vehicle by Multifidelity Evolution Control , 2013 .

[37]  Kroo Ilan,et al.  Multidisciplinary Optimization Methods for Aircraft Preliminary Design , 1994 .

[38]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Jasper Snoek,et al.  Multi-Task Bayesian Optimization , 2013, NIPS.

[40]  Wei Huang,et al.  Design exploration of three-dimensional transverse jet in a supersonic crossflow based on data mining and multi-objective design optimization approaches , 2014 .

[41]  Liang Feng,et al.  Autoencoding Evolutionary Search With Learning Across Heterogeneous Problems , 2017, IEEE Transactions on Evolutionary Computation.

[42]  A. Forrester,et al.  Design and analysis of 'noisy' computer experiments , 2006 .

[43]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[44]  Brendan D. Tracey,et al.  A Machine Learning Strategy to Assist Turbulence Model Development , 2015 .

[45]  Pj Clarkson,et al.  Robust multi-fidelity aerodynamic design optimization using surrogate models , 2008 .

[46]  Dimitri N. Mavris,et al.  Gaussian Process Surrogate Model for Levering Similar Trends Across Concepts , 2015 .

[47]  Lee Soo-Young,et al.  A self-organizing neural network approach for automatic mesh generation , 1991 .

[48]  Donghyun You,et al.  Integrated RANS/LES computations of turbulent o w through a turbofan jet engine , 2006 .

[49]  Ali Kamran,et al.  Support Vector Regression-driven Multidisciplinary Design Optimization of Multistage Ground Based Interceptor , 2009 .

[50]  Lin Ma,et al.  Design exploration for a single expansion ramp nozzle (SERN) using data mining , 2013 .

[51]  Sham M. Kakade,et al.  Multi-view Regression Via Canonical Correlation Analysis , 2007, COLT.

[52]  Sham M. Kakade,et al.  Multi-view clustering via canonical correlation analysis , 2009, ICML '09.

[53]  Oliver Stegle,et al.  It is all in the noise: Efficient multi-task Gaussian process inference with structured residuals , 2013, NIPS.

[54]  Jian Chen,et al.  Optimal design of aeroengine turbine disc based on kriging surrogate models , 2011 .

[55]  Jaime G. Carbonell,et al.  Feature Selection for Transfer Learning , 2011, ECML/PKDD.

[56]  Kay Chen Tan,et al.  Multiobjective Multifactorial Optimization in Evolutionary Multitasking , 2017, IEEE Transactions on Cybernetics.

[57]  Yan Liu,et al.  Accelerating Active Learning with Transfer Learning , 2013, 2013 IEEE 13th International Conference on Data Mining.

[58]  Liu Dong,et al.  Optimization of Preform Shapes by RSM and FEM to Improve Deformation Homogeneity in Aerospace Forgings , 2010 .

[59]  Jack D. Mattingly,et al.  Aircraft engine design , 1987 .

[60]  R. Haftka,et al.  On options for interdisciplinary analysis and design optimization , 1992 .

[61]  Shiliang Sun,et al.  A survey of multi-view machine learning , 2013, Neural Computing and Applications.

[62]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[63]  Andy J. Keane,et al.  Recent advances in surrogate-based optimization , 2009 .

[64]  Joaquim R. R. A. Martins,et al.  Multidisciplinary design optimization: A survey of architectures , 2013 .

[65]  Kwang-Yong Kim,et al.  Design optimization of rib-roughened channel to enhance turbulent heat transfer , 2004 .

[66]  Abdelhamid Bouchachia,et al.  Incremental Learning , 2009, Encyclopedia of Data Warehousing and Mining.

[67]  Barbara J. Grosz,et al.  Natural-Language Processing , 1982, Artificial Intelligence.

[68]  Jaroslaw Sobieszczanski-Sobieski,et al.  Optimization of coupled systems : A critical overview of approaches , 1994 .

[69]  Andy J. Keane,et al.  Wing Optimization Using Design of Experiment, Response Surface, and Data Fusion Methods , 2003 .

[70]  Russell W. Claus,et al.  Multidisciplinary propulsion simulation using NPSS , 1992 .

[71]  John E. Renaud,et al.  Response surface based, concurrent subspace optimization for multidisciplinary system design , 1996 .

[72]  Bojan Dolsak,et al.  Finite element mesh design expert system , 2002, Knowl. Based Syst..

[73]  Jakub Marecek,et al.  Handbook of Approximation Algorithms and Metaheuristics , 2010, Comput. J..

[74]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[75]  Dan Givoli,et al.  Finite-Element Mesh Generation Using Self-Organizing Neural Networks , 1997 .

[76]  P. Moin,et al.  DIRECT NUMERICAL SIMULATION: A Tool in Turbulence Research , 1998 .

[77]  John E. Dennis,et al.  Problem Formulation for Multidisciplinary Optimization , 1994, SIAM J. Optim..

[78]  Wei Shyy,et al.  Shape optimization of supersonic turbines using global approximation methods , 2002 .

[79]  Man Mohan Rai,et al.  Neural Net-Based Redesign of Transonic Turbines for Improved Unsteady Aerodynamic Performance , 1998 .

[80]  David E. Goldberg,et al.  Using Previous Models to Bias Structural Learning in the Hierarchical BOA , 2012, Evolutionary Computation.

[81]  Ivor W. Tsang,et al.  Memetic Search With Interdomain Learning: A Realization Between CVRP and CARP , 2015, IEEE Transactions on Evolutionary Computation.

[82]  Rohitash Chandra,et al.  Coping with Data Scarcity in Aircraft Engine Design , 2017 .

[83]  Egbert Torenbeek,et al.  The New Textbook ”Advanced Aircraft Design – Conceptual Design, Technology and Optimization of Subsonic Civil Airplanes“ , 2013 .

[84]  Stefan Görtz,et al.  Hierarchical Kriging Model for Variable-Fidelity Surrogate Modeling , 2012 .

[85]  Mengjie Zhang,et al.  Reusing Building Blocks of Extracted Knowledge to Solve Complex, Large-Scale Boolean Problems , 2014, IEEE Transactions on Evolutionary Computation.