Teaching MLOps in Higher Education through Project-Based Learning

Building and maintaining production-grade ML-enabled components is a complex endeavor that goes beyond the current approach of academic education, focused on the optimization of ML model performance in the lab. In this paper, we present a project-based learning approach to teaching MLOps, focused on the demonstration and experience with emerging practices and tools to automatize the construction of ML-enabled components. We examine the design of a course based on this approach, including laboratory sessions that cover the end-to-end ML component life cycle, from model building to production deployment. Moreover, we report on preliminary results from the first edition of the course. During the present year, an updated version of the same course is being delivered in two independent universities; the related learning outcomes will be evaluated to analyze the effectiveness of project-based learning for this specific subject.

[1]  P. Heck,et al.  What Is an AI Engineer? An Empirical Analysis of Job Ads in The Netherlands , 2022, 2022 IEEE/ACM 1st International Conference on AI Engineering – Software Engineering for AI (CAIN).

[2]  F. Lanubile,et al.  Pynblint: a Static Analyzer for Python Jupyter Notebooks , 2022, 2022 IEEE/ACM 1st International Conference on AI Engineering – Software Engineering for AI (CAIN).

[3]  F. Lanubile,et al.  Eliciting Best Practices for Collaboration with Computational Notebooks , 2022, Proc. ACM Hum. Comput. Interact..

[4]  Petra Heck,et al.  Lessons Learned from Educating AI Engineers , 2021, 2021 IEEE/ACM 1st Workshop on AI Engineering - Software Engineering for AI (WAIN).

[5]  Jan Bosch,et al.  DevOps for AI – Challenges in Development of AI-enabled Applications , 2020, 2020 International Conference on Software, Telecommunications and Computer Networks (SoftCOM).

[6]  Sameer Singh,et al.  Beyond Accuracy: Behavioral Testing of NLP Models with CheckList , 2020, ACL.

[7]  Eunsuk Kang,et al.  Teaching Software Engineering for Al-Enabled Systems , 2020, 2020 IEEE/ACM 42nd International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET).

[8]  Noah A. Smith,et al.  Green AI , 2019, 1907.10597.

[9]  Andrew McCallum,et al.  Energy and Policy Considerations for Deep Learning in NLP , 2019, ACL.

[10]  Inioluwa Deborah Raji,et al.  Model Cards for Model Reporting , 2018, FAT.

[11]  Li Deng,et al.  Artificial Intelligence in the Rising Wave of Deep Learning: The Historical Path and Future Outlook [Perspectives] , 2018, IEEE Signal Processing Magazine.

[12]  David Maxwell Chickering,et al.  Machine Teaching: A New Paradigm for Building Machine Learning Systems , 2017, ArXiv.

[13]  D. Sculley,et al.  Hidden Technical Debt in Machine Learning Systems , 2015, NIPS.

[14]  I. Miladinovic,et al.  Educating AI Software Engineers: Challenges and Opportunities , 2021, ICL.

[15]  Lysanne S. Post,et al.  A review of project-based learning in higher education: Student outcomes and measures , 2020 .

[16]  Charlie Gerard,et al.  Machine learning in production , 2020 .

[17]  Daniel Rubio,et al.  Great Expectations , 2015, Logic, Rationality, and Interaction.

[18]  H. D. Rombach,et al.  The Goal Question Metric Approach , 1994 .