Machine Learning Education for Artists, Musicians, and Other Creative Practitioners

This article aims to lay a foundation for the research and practice of machine learning education for creative practitioners. It begins by arguing that it is important to teach machine learning to creative practitioners and to conduct research about this teaching, drawing on related work in creative machine learning, creative computing education, and machine learning education. It then draws on research about design processes in engineering and creative practice to motivate a set of learning objectives for students who wish to design new creative artifacts with machine learning. The article then draws on education research and knowledge of creative computing practices to propose a set of teaching strategies that can be used to support creative computing students in achieving these objectives. Explanations of these strategies are accompanied by concrete descriptions of how they have been employed to develop new lectures and activities, and to design new experiential learning and scaffolding technologies, for teaching some of the first courses in the world focused on teaching machine learning to creative practitioners. The article subsequently draws on data collected from these courses—an online course as well as undergraduate and masters-level courses taught at a university—to begin to understand how this curriculum supported student learning, to understand learners’ challenges and mistakes, and to inform future teaching and research.

[1]  Joseph A. Paradiso,et al.  The gesture recognition toolkit , 2014, J. Mach. Learn. Res..

[2]  Larry P. Heck,et al.  Deep Music: Towards Musical Dialogue , 2017, AAAI.

[3]  Ali Momeni,et al.  Ml.lib: robust, cross-platform, open-source machine learning for max and pure data , 2015, NIME.

[4]  John Lansdown,et al.  Generative techniques in graphical computer art: some possibilities and practices , 1989 .

[5]  Katy Jordan,et al.  Massive Open Online Course Completion Rates Revisited: Assessment, Length and Attrition , 2015 .

[6]  Ben Shneiderman,et al.  Design Principles for Tools to Support Creative Thinking , 2005 .

[7]  David Wessel,et al.  Real-Time Neural Network Processing of Gestural and Acoustic Signals , 1991, ICMC.

[8]  Ann Morrison,et al.  Bodily Explorations in Space: Social Experience of a Multimodal Art Installation , 2009, INTERACT.

[9]  Alan Bundy,et al.  Dynamic Time Warping , 1984 .

[10]  Christopher Raphael,et al.  Music Plus One and Machine Learning , 2010, ICML.

[11]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[12]  Embry-Riddle Aeronautical,et al.  The Flipped Classroom: A Survey of the Research , 2013 .

[13]  Heiga Zen,et al.  WaveNet: A Generative Model for Raw Audio , 2016, SSW.

[14]  Chang-Heng Wang COS 511: Theoretical Machine Learning , 2018 .

[15]  Aleata Hubbard,et al.  Pedagogical content knowledge in computing education: a review of the research literature , 2018, Comput. Sci. Educ..

[16]  L. Shulman Those Who Understand: Knowledge Growth in Teaching , 1986 .

[17]  Charles Ames,et al.  The Markov Process as a Compositional Model: A Survey and Tutorial , 2017 .

[18]  Luke Dahl,et al.  TweetDreams: Making Music with the Audience and the World using Real-time Twitter Data , 2011, NIME.

[19]  Russell K. Schutt,et al.  Research Methods in Education , 2011 .

[20]  Ameet Talwalkar,et al.  MLlib: Machine Learning in Apache Spark , 2015, J. Mach. Learn. Res..

[21]  Chris Piech,et al.  Deconstructing disengagement: analyzing learner subpopulations in massive open online courses , 2013, LAK '13.

[22]  Kayur Patel,et al.  Lowering the barrier to applying machine learning , 2010, UIST '10.

[23]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[24]  Alexander Mordvintsev,et al.  Inceptionism: Going Deeper into Neural Networks , 2015 .

[25]  D. Kolb Experiential Learning: Experience as the Source of Learning and Development , 1983 .

[26]  Roy D. Pea,et al.  Addressing the Challenges of Inquiry-Based Learning Through Technology and Curriculum Design , 1999 .

[27]  Perry R. Cook,et al.  The Wekinator: Software for using machine learning to build real-time interactive systems , 2011 .

[28]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[29]  Hongji Yang,et al.  The creative turn: new challenges for computing , 2013, Int. J. Creative Comput..

[30]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[31]  Rae A. Earnshaw,et al.  Computers in art, design and animation , 1989 .

[32]  W. Huff EXPERIENCE OR EDUCATION , 1968 .

[33]  Jerry Alan Fails,et al.  Interactive machine learning , 2003, IUI '03.

[34]  Beth Logan,et al.  Mel Frequency Cepstral Coefficients for Music Modeling , 2000, ISMIR.

[35]  James A. Landay,et al.  Gestalt: integrated support for implementation and analysis in machine learning , 2010, UIST.

[36]  Shuji Hashimoto,et al.  EyesWeb: Toward Gesture and Affect Recognition in Interactive Dance and Music Systems , 2000, Computer Music Journal.

[37]  Kylie A. Peppler,et al.  STEAM-Powered Computing Education: Using E-Textiles to Integrate the Arts and STEM , 2013, Computer.

[38]  Juha Sorva,et al.  Notional machines and introductory programming education , 2013, TOCE.

[39]  Thomas J. Howard,et al.  Describing the creative design process by the integration of engineering design and cognitive psychology literature , 2008 .

[40]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[41]  Luis C. Moll,et al.  Funds of knowledge for teaching: Using a qualitative approach to connect homes and classrooms , 1992 .

[42]  Mark d'Inverno,et al.  Evidencing the Value of Inquiry Based, Constructionist Learning for Student Coders , 2017, Int. J. Eng. Pedagog..

[43]  Francisco Bernardo,et al.  User-Centred Design Actions for Lightweight Evaluation of an Interactive Machine Learning Toolkit , 2018 .

[44]  Roy D. Pea,et al.  Language-Independent Conceptual “Bugs” in Novice Programming , 1986 .

[45]  Alex J. Champandard,et al.  Semantic Style Transfer and Turning Two-Bit Doodles into Fine Artworks , 2016, ArXiv.

[46]  Perry R. Cook,et al.  Human model evaluation in interactive supervised learning , 2011, CHI.

[47]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[48]  P. Grossman The Making of a Teacher: Teacher Knowledge and Teacher Education , 1990 .

[49]  Matthew Wright,et al.  Open SoundControl: A New Protocol for Communicating with Sound Synthesizers , 1997, ICMC.

[50]  Meinard Müller,et al.  Information retrieval for music and motion , 2007 .

[51]  Perry R. Cook,et al.  Real-time human interaction with supervised learning algorithms for music composition and performance , 2011 .

[52]  Atau Tanaka,et al.  Adaptive Gesture Recognition with Variation Estimation for Interactive Systems , 2014, ACM Trans. Interact. Intell. Syst..

[53]  Mordechai Ben-Ari,et al.  Constructivism in computer science education , 1998, SIGCSE '98.

[54]  Jason Freeman,et al.  Engaging underrepresented groups in high school introductory computing through computational remixing with EarSketch , 2014, SIGCSE.

[55]  Norbert Schnell,et al.  Continuous Realtime Gesture Following and Recognition , 2009, Gesture Workshop.

[56]  Perry R. Cook,et al.  A Meta-Instrument for Interactive, On-the-Fly Machine Learning , 2009, NIME.

[57]  Matthew J. Koehler,et al.  Technological Pedagogical Content Knowledge: A Framework for Teacher Knowledge , 2006, Teachers College Record: The Voice of Scholarship in Education.