Bayesian Multiobjective Optimisation With Mixed Analytical and Black-Box Functions: Application to Tissue Engineering

Tissue engineering and regenerative medicine looks at improving or restoring biological tissue function in humans and animals. We consider optimising neotissue growth in a three-dimensional scaffold during dynamic perfusion bioreactor culture, in the context of bone tissue engineering. The goal is to choose design variables that optimise two conflicting objectives, first, maximising neotissue growth and, second, minimising operating cost. We make novel extensions to Bayesian multiobjective optimisation in the case of one analytical objective function and one black-box, i.e. simulation based and objective function. The analytical objective represents operating cost while the black-box neotissue growth objective comes from simulating a system of partial differential equations. The resulting multiobjective optimisation method determines the tradeoff between neotissue growth and operating cost. Our method exhibits better data efficiency than genetic algorithms, i.e. the most common approach in the literature, on both the tissue engineering example and standard test functions. The multiobjective optimisation method applies to real-world problems combining black-box models with easy-to-quantify objectives such as cost.

[1]  Hamid Khaloozadeh,et al.  Open- and Closed-Loop Multiobjective Optimal Strategies for HIV Therapy Using NSGA-II , 2011, IEEE Transactions on Biomedical Engineering.

[2]  L. Geris,et al.  A computational model for cell/ECM growth on 3D surfaces using the level set method: a bone tissue engineering case study , 2014, Biomechanics and modeling in mechanobiology.

[3]  Marc Peter Deisenroth,et al.  Design of Experiments for Model Discrimination Hybridising Analytical and Data-Driven Approaches , 2018, ICML.

[4]  Efstratios N. Pistikopoulos,et al.  Global superstructure optimisation of red blood cell production in a parallelised hollow fibre bioreactor , 2014, Comput. Chem. Eng..

[5]  Philip Kollmannsberger,et al.  Geometry as a Factor for Tissue Growth: Towards Shape Optimization of Tissue Engineering Scaffolds , 2013, Advanced healthcare materials.

[6]  Andy J. Keane,et al.  Statistical Improvement Criteria for Use in Multiobjective Design Optimization , 2006 .

[7]  Liesbet Geris,et al.  Maximizing neotissue growth kinetics in a perfusion bioreactor: An in silico strategy using model reduction and Bayesian optimization , 2018, Biotechnology and bioengineering.

[8]  C. Maranas,et al.  Recent advances in the reconstruction of metabolic models and integration of omics data. , 2014, Current opinion in biotechnology.

[9]  Thomas J. Santner,et al.  Multiobjective optimization of expensive-to-evaluate deterministic computer simulator models , 2016, Comput. Stat. Data Anal..

[10]  Jonas Mockus,et al.  The Application Of Bayesian Methods , 1989 .

[11]  Peter W Zandstra,et al.  Quality cell therapy manufacturing by design , 2016, Nature Biotechnology.

[12]  Xiying Wu,et al.  Culture effects of epidermal growth factor (EGF) and basic fibroblast growth factor (bFGF) on cryopreserved human adipose‐derived stromal/stem cell proliferation and adipogenesis , 2009, Journal of tissue engineering and regenerative medicine.

[13]  Luciano Lamberti,et al.  A Mechanobiology-based Algorithm to Optimize the Microstructure Geometry of Bone Tissue Scaffolds , 2016, International journal of biological sciences.

[14]  Frank Kursawe,et al.  A Variant of Evolution Strategies for Vector Optimization , 1990, PPSN.

[15]  Peter J. Fleming,et al.  An Overview of Evolutionary Algorithms in Multiobjective Optimization , 1995, Evolutionary Computation.

[16]  Marianthi G. Ierapetritou,et al.  Surrogate-Based Optimization of Expensive Flowsheet Modeling for Continuous Pharmaceutical Manufacturing , 2013, Journal of Pharmaceutical Innovation.

[17]  Y. Guyot,et al.  Coupling curvature-dependent and shear stress-stimulated neotissue growth in dynamic bioreactor cultures: a 3D computational model of a complete scaffold , 2016, Biomechanics and modeling in mechanobiology.

[18]  Liesbet Geris,et al.  Regenerative orthopaedics: in vitro, in vivo … in silico , 2014, International Orthopaedics.

[19]  Carl E. Rasmussen,et al.  Manifold Gaussian Processes for regression , 2014, 2016 International Joint Conference on Neural Networks (IJCNN).

[20]  Christodoulos A. Floudas,et al.  Global optimization advances in Mixed-Integer Nonlinear Programming, MINLP, and Constrained Derivative-Free Optimization, CDFO , 2016, Eur. J. Oper. Res..

[21]  T W Gardner,et al.  Effect of vascular endothelial growth factor on cultured endothelial cell monolayer transport properties. , 2000, Microvascular research.

[22]  Peter Auer,et al.  Using Confidence Bounds for Exploitation-Exploration Trade-offs , 2003, J. Mach. Learn. Res..

[23]  S. Harris,et al.  Stimulation of bone formation in vitro and in rodents by statins. , 1999, Science.

[24]  Liesbet Geris,et al.  Bayesian Multi-Objective Optimisation of Neotissue Growth in a Perfusion Bioreactor Set-up , 2017 .

[25]  Franck Patrick Vidal,et al.  Tuning of Patient-Specific Deformable Models Using an Adaptive Evolutionary Optimization Strategy , 2012, IEEE Transactions on Biomedical Engineering.

[26]  Harold J. Kushner,et al.  A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise , 1964 .

[27]  I. G. Kevrekidis,et al.  Enabling dynamic process simulators to perform alternative tasks: A time-stepper-based toolkit for computer-aided analysis , 2003 .

[28]  Roberto Calandra,et al.  Bayesian Modeling for Optimization and Control in Robotics , 2017 .

[29]  Naoki Kawazoe,et al.  Autologous extracellular matrix scaffolds for tissue engineering. , 2011, Biomaterials.

[30]  J. D. Schaffer,et al.  Some experiments in machine learning using vector evaluated genetic algorithms (artificial intelligence, optimization, adaptation, pattern recognition) , 1984 .

[31]  Efstratios N. Pistikopoulos,et al.  A systematic framework for the design, simulation and optimization of personalized healthcare: Making and healing blood , 2015, Comput. Chem. Eng..

[32]  Julio R. Banga,et al.  Optimization in computational systems biology , 2008, BMC Systems Biology.

[33]  Warren L Grayson,et al.  Bone tissue engineering bioreactors: a role in the clinic? , 2012, Tissue engineering. Part B, Reviews.

[34]  David Ginsbourger,et al.  Quantifying uncertainty on Pareto fronts with Gaussian process conditional simulations , 2015, Eur. J. Oper. Res..

[35]  Antonios G Mikos,et al.  In vitro localization of bone growth factors in constructs of biodegradable scaffolds seeded with marrow stromal cells and cultured in a flow perfusion bioreactor. , 2006, Tissue engineering.

[36]  D. Wendt,et al.  The role of bioreactors in tissue engineering. , 2004, Trends in biotechnology.

[37]  Mohd Almie Alias,et al.  Modeling the Effect of Curvature on the Collective Behavior of Cells Growing New Tissue. , 2016, Biophysical journal.

[38]  J. Peters,et al.  Pareto Front Modeling for Sensitivity Analysis in Multi-Objective Bayesian Optimization , 2014 .

[39]  Glynn J. Sundararaj,et al.  Ability of Objective Functions to Generate Points on Nonconvex Pareto Frontiers , 2000 .

[40]  Benoît Chachuat,et al.  Bayesian Optimization with Dimension Scheduling: Application to Biological Systems , 2015, ArXiv.

[41]  Daniel Hern'andez-Lobato,et al.  Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraints , 2016, Neurocomputing.

[42]  Michael T. M. Emmerich,et al.  Single- and multiobjective evolutionary optimization assisted by Gaussian random field metamodels , 2006, IEEE Transactions on Evolutionary Computation.

[43]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[44]  C. Hwang Multiple Objective Decision Making - Methods and Applications: A State-of-the-Art Survey , 1979 .

[45]  Nicholas I. M. Gould,et al.  Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results , 2011, Math. Program..

[46]  Liesbet Geris,et al.  Immersed Boundary Models for Quantifying Flow-Induced Mechanical Stimuli on Stem Cells Seeded on 3D Scaffolds in Perfusion Bioreactors , 2016, PLoS Comput. Biol..

[47]  Alexandre Bernardino,et al.  Unscented Bayesian optimization for safe robot grasping , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[48]  Gary B. Lamont,et al.  Multiobjective evolutionary algorithms: classifications, analyses, and new innovations , 1999 .

[49]  Cécile M. Bidan,et al.  A three-dimensional model for tissue deposition on complex surfaces , 2013, Computer methods in biomechanics and biomedical engineering.

[50]  Roman Garnett,et al.  Bayesian optimization for sensor set selection , 2010, IPSN '10.

[51]  Richard Balling,et al.  The Maximin Fitness Function; Multi-objective City and Regional Planning , 2003, EMO.

[52]  Ching-Lai Hwang,et al.  Fuzzy Multiple Attribute Decision Making - Methods and Applications , 1992, Lecture Notes in Economics and Mathematical Systems.

[53]  Jan Peters,et al.  Bayesian optimization for learning gaits under uncertainty , 2015, Annals of Mathematics and Artificial Intelligence.

[54]  Efstratios N. Pistikopoulos,et al.  Optimal design of energy systems using constrained grey-box multi-objective optimization , 2018, Comput. Chem. Eng..

[55]  Mark C Allenby,et al.  Stem cell biomanufacturing under uncertainty: A case study in optimizing red blood cell production , 2017, AIChE journal. American Institute of Chemical Engineers.

[56]  Lotfi A. Zadeh,et al.  Optimality and non-scalar-valued performance criteria , 1963 .

[57]  Yann Guyot,et al.  A multiphysics multiscale computational framework for the simulation of perfusion bioreactor processes in bone tissue engineering , 2015 .

[58]  P. Giannoudis,et al.  Fracture healing: the diamond concept. , 2007, Injury.

[59]  J. Schrooten,et al.  A three‐dimensional computational fluid dynamics model of shear stress distribution during neotissue growth in a perfusion bioreactor , 2015, Biotechnology and bioengineering.

[60]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.

[61]  Daniel A. Ashlock,et al.  Evolutionary computation for modeling and optimization , 2005 .

[62]  Victor Picheny,et al.  Multiobjective optimization using Gaussian process emulators via stepwise uncertainty reduction , 2013, Statistics and Computing.

[63]  J. Mockus Bayesian Approach to Global Optimization: Theory and Applications , 1989 .