Automated stem cell production by bio-inspired control

Abstract The potential in treating chronic and life-threatening diseases by stem cell therapies can greatly be exploited via the efficient automation of stem cell production. Working with living material though poses severe challenges to automation. Recently, production platforms has been developed and tested worldwide with the aim to increase the reproducibility, quality and throughput of the process, to minimize human errors, and to reduce costs of production. A distinctive feature of this domain is the symbiotic co-existence and co-evolution of the technical, information and communication, as well as biological ingredients in production structures. A challenging way to overcome the issues of automated production is the use of biologically inspired control algorithms. In the paper an approach is described which combines digital, agent-based simulation and reinforcement learning for this purpose. The modelling of the cell growth behaviour, which is an important prerequisite of the simulation, is also introduced, together with an appropriate model fitting procedure. The applicability of the proposed approach is demonstrated by the results of a comprehensive investigation.

[1]  Balázs Csanád Csáji.  ADAPTIVE RESOURCE CONTROL Machine Learning Approaches to Resource Allocation in Uncertain and Changing Environments , 2008 .

[2]  Christian Brecher,et al.  Simulation Framework for Virtual Robot Programming in Reconfigurable Production Systems , 2019, Procedia CIRP.

[3]  David P. Smith,et al.  On the Nature of the Function Expressive of the Law of Human Mortality , 2013 .

[4]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[5]  Soundar R. T. Kumara,et al.  Cyber-physical systems in manufacturing , 2016 .

[6]  Marc-André Dittrich,et al.  Cooperative multi-agent system for production control using reinforcement learning , 2020, CIRP Annals.

[7]  E. Tjørve,et al.  A unified approach to the Richards-model family for use in growth analyses: why we need only two model forms. , 2010, Journal of theoretical biology.

[8]  Gisela Lanza,et al.  Reinforcement learning for an intelligent and autonomous production control of complex job-shops under time constraints , 2020, Production Engineering.

[9]  László Monostori,et al.  Biologicalisation: Biological transformation in manufacturing , 2018 .

[10]  Brecher Christian,et al.  Dynamic integration of manual and automated biological process skills into MES , 2019, 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA).

[11]  Gisela Lanza,et al.  Application of Reinforcement Learning in Production Planning and Control of Cyber Physical Production Systems , 2018, ML4CPS.

[12]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[13]  Robert Schmitt,et al.  Metrology-based quality and process control in automated stem cell production , 2015 .

[14]  U. Rieder,et al.  Markov Decision Processes with Applications to Finance , 2011 .

[15]  László Monostori,et al.  Value Function Based Reinforcement Learning in Changing Markovian Environments , 2008, J. Mach. Learn. Res..

[16]  László Monostori,et al.  Agent-based systems for manufacturing , 2006 .

[17]  Nikolaos V. Sahinidis,et al.  A deep reinforcement learning approach for chemical production scheduling , 2020, Comput. Chem. Eng..

[18]  Robert Schmitt,et al.  Parallelization in Automated Stem Cell Culture , 2017 .

[19]  A. Tsoularis,et al.  Analysis of logistic growth models. , 2002, Mathematical biosciences.

[20]  G. Daley,et al.  The promise of induced pluripotent stem cells in research and therapy , 2012, Nature.

[21]  Andreas Kuhnle,et al.  Reinforcement learning for adaptive order dispatching in the semiconductor industry , 2018 .

[22]  Fritz Klocke,et al.  Automatic Production of Induced Pluripotent Stem Cells , 2013 .

[23]  U. Galderisi,et al.  Clinical Trials with Mesenchymal Stem Cells: An Update , 2016, Cell transplantation.

[24]  C. Hewitt,et al.  Characterization of human mesenchymal stem cells from multiple donors and the implications for large scale bioprocess development , 2016 .

[25]  Botond Kádár,et al.  Enhanced control of complex production structures by tight coupling of the digital and the physical worlds , 2010 .

[26]  David J. Williams,et al.  Precision manufacturing for clinical-quality regenerative medicines , 2012, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[27]  Herwig Winkler,et al.  Industrial scheduling with Monte Carlo tree search and machine learning , 2018 .

[28]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[29]  Christian Brecher,et al.  Modelling of devices in an adaptive and dynamic environment , 2019, Procedia CIRP.

[30]  Robert Schmitt,et al.  Automation in the context of stem cell production - where are we heading with Industry 4.0? , 2016 .

[31]  Christian Brecher,et al.  Bio-inspired control of automated stem cell production , 2020 .

[32]  E. Tjørve,et al.  The use of Gompertz models in growth analyses, and new Gompertz-model approach: An addition to the Unified-Richards family , 2017, PloS one.

[33]  L. Monostori,et al.  Adaptive Stochastic Resource Control: A Machine Learning Approach , 2008, J. Artif. Intell. Res..

[34]  John N. Tsitsiklis,et al.  Analysis of temporal-difference learning with function approximation , 1996, NIPS 1996.

[35]  J. Hyllner,et al.  Cell-based therapy technology classifications and translational challenges , 2015, Philosophical Transactions of the Royal Society B: Biological Sciences.

[36]  Lenz Belzner,et al.  Optimization of global production scheduling with deep reinforcement learning , 2018 .

[37]  M. Butler.  Animal Cell Culture and Technology , 1997 .

[38]  Benjamin Gompertz,et al.  XXIV. On the nature of the function expressive of the law of human mortality, and on a new mode of determining the value of life contingencies. In a letter to Francis Baily, Esq. F. R. S. &c , 1825, Philosophical Transactions of the Royal Society of London.

[39]  Zhaoqi Wang,et al.  Simulation-assisted machine learning , 2018, Bioinform..