Self-directed Machine Learning

Conventional machine learning (ML) relies heavily on manual design from machine learning experts to decide learning tasks, data, models, optimization algorithms, and evaluation metrics, which is labor-intensive, time-consuming, and cannot learn autonomously like humans. In education science, self-directed learning, where human learners select learning tasks and materials on their own without requiring hands-on guidance, has been shown to be more effective than passive teacher-guided learning. Inspired by the concept of self-directed human learning, we introduce the principal concept of Selfdirected Machine Learning (SDML) and propose a framework for SDML. Specifically, we design SDML as a self-directed learning process guided by self-awareness, including internal awareness and external awareness. Our proposed SDML process benefits from self task selection, self data selection, self model selection, self optimization strategy selection and self evaluation metric selection through self-awareness without human guidance. Meanwhile, the learning performance of the SDML process serves as feedback to further improve self-awareness. We propose a mathematical formulation for SDML based on multi-level optimization. Furthermore, we present case studies together with potential applications of SDML, followed by discussing future research directions. We expect that SDML could enable machines to conduct human-like self-directed learning and provide a new perspective towards artificial general intelligence.

[1]  J. McNally Self-Directed Learning, A Guide for Learners and Teachers , 1976 .

[2]  Philip H. Winne,et al.  Theories of Instruction and of Intelligence for Designing Artificially Intelligent Tutoring Systems , 1989 .

[3]  Stephen Brookfield,et al.  Self-Directed Learning, Political Clarity, and the Critical Practice of Adult Education , 1993 .

[4]  Philip H. Winne,et al.  Experimenting to Bootstrap Self-Regulated Learning , 1997 .

[5]  D. Garrison Self-Directed Learning: Toward a Comprehensive Model , 1997 .

[6]  Craig A. Knoblock,et al.  PDDL-the planning domain definition language , 1998 .

[7]  Yishay Mansour,et al.  Policy Gradient Methods for Reinforcement Learning with Function Approximation , 1999, NIPS.

[8]  Nizar Grira,et al.  Unsupervised and Semi-supervised Clustering : a Brief Survey ∗ , 2004 .

[9]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[10]  Liyan Song,et al.  A Conceptual Model for Understanding Self-Directed Learning in Online Environments. , 2007 .

[11]  Alessandro Saffiotti,et al.  Robot task planning using semantic maps , 2008, Robotics Auton. Syst..

[12]  B. Zimmerman,et al.  Motivation and Self-Regulated Learning: Theory, Research, and Applications , 2009 .

[13]  Rachid Alami,et al.  A Hybrid Approach to Intricate Motion, Manipulation and Task Planning , 2009, Int. J. Robotics Res..

[14]  Jason Weston,et al.  Curriculum learning , 2009, ICML '09.

[15]  Yoshua Bengio,et al.  Algorithms for Hyper-Parameter Optimization , 2011, NIPS.

[16]  Meng Wang,et al.  Active learning in multimedia annotation and retrieval: A survey , 2011, TIST.

[17]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[18]  Jasper Snoek,et al.  Practical Bayesian Optimization of Machine Learning Algorithms , 2012, NIPS.

[19]  Todd M Gureckis,et al.  Self-Directed Learning , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[20]  Alex Graves,et al.  Playing Atari with Deep Reinforcement Learning , 2013, ArXiv.

[21]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[22]  Yulia Tsvetkov,et al.  Learning the Curriculum with Bayesian Optimization for Task-Specific Word Representation Learning , 2016, ACL.

[23]  Abhinav Gupta,et al.  Training Region-Based Object Detectors with Online Hard Example Mining , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Marc'Aurelio Ranzato,et al.  Gradient Episodic Memory for Continual Learning , 2017, NIPS.

[25]  Christoph H. Lampert,et al.  iCaRL: Incremental Classifier and Representation Learning , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  V. S. Ramachandran,et al.  Analogical Reasoning , 2012 .

[27]  José M. F. Moura,et al.  Visual Dialog , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Razvan Pascanu,et al.  Overcoming catastrophic forgetting in neural networks , 2016, Proceedings of the National Academy of Sciences.

[29]  Zhe Gan,et al.  StyleNet: Generating Attractive Visual Captions with Styles , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Pengtao Xie,et al.  A Constituent-Centric Neural Architecture for Reading Comprehension , 2017, ACL.

[31]  Quoc V. Le,et al.  Neural Architecture Search with Reinforcement Learning , 2016, ICLR.

[32]  Etienne Perot,et al.  Deep Reinforcement Learning framework for Autonomous Driving , 2017, Autonomous Vehicles and Machines.

[33]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[34]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[35]  Li Fei-Fei,et al.  MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels , 2017, ICML.

[36]  Svetlana Lazebnik,et al.  PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[37]  Jonghyun Choi,et al.  ScreenerNet: Learning Self-Paced Curriculum for Deep Neural Networks , 2018 .

[38]  Swarat Chaudhuri,et al.  An incremental constraint-based framework for task and motion planning , 2018, Int. J. Robotics Res..

[39]  Bin Yang,et al.  Learning to Reweight Examples for Robust Deep Learning , 2018, ICML.

[40]  Isabelle Guyon,et al.  Taking Human out of Learning Applications: A Survey on Automated Machine Learning , 2018, 1810.13306.

[41]  Joaquin Vanschoren,et al.  Meta-Learning: A Survey , 2018, Automated Machine Learning.

[42]  Vijay Vasudevan,et al.  Learning Transferable Architectures for Scalable Image Recognition , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[43]  Quoc V. Le,et al.  Efficient Neural Architecture Search via Parameter Sharing , 2018, ICML.

[44]  Anoop Cherian,et al.  Audio Visual Scene-Aware Dialog , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Frank Hutter,et al.  Neural Architecture Search: A Survey , 2018, J. Mach. Learn. Res..

[46]  Richard Socher,et al.  Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting , 2019, ICML.

[47]  Dahua Lin,et al.  Learning a Unified Classifier Incrementally via Rebalancing , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Alok Aggarwal,et al.  Regularized Evolution for Image Classifier Architecture Search , 2018, AAAI.

[49]  Stefan Wermter,et al.  Continual Lifelong Learning with Neural Networks: A Review , 2019, Neural Networks.

[50]  Yiming Yang,et al.  DARTS: Differentiable Architecture Search , 2018, ICLR.

[51]  Marco F. Huber,et al.  Survey on Automated Machine Learning , 2019, ArXiv.

[52]  Alexander Schwing,et al.  Fast, Diverse and Accurate Image Captioning Guided by Part-Of-Speech , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Fillia Makedon,et al.  A Survey on Contrastive Self-supervised Learning , 2020, Technologies.

[54]  Jimmy Ba,et al.  Dream to Control: Learning Behaviors by Latent Imagination , 2019, ICLR.

[55]  Demis Hassabis,et al.  Mastering Atari, Go, chess and shogi by planning with a learned model , 2019, Nature.

[56]  Jie Tang,et al.  Self-Supervised Learning: Generative or Contrastive , 2020, IEEE Transactions on Knowledge and Data Engineering.

[57]  Yitian Yuan,et al.  Controllable Video Captioning with an Exemplar Sentence , 2020, ACM Multimedia.

[58]  Jungong Han,et al.  Few-Cost Salient Object Detection with Adversarial-Paced Learning , 2021, NeurIPS.

[59]  Sailik Sengupta,et al.  RADAR: automated task planning for proactive decision support , 2020, Hum. Comput. Interact..

[60]  Reinhard Koch,et al.  A survey on Semi-, Self- and Unsupervised Techniques in Image Classification , 2020, ArXiv.

[61]  Jingzhou Liu,et al.  Violin: A Large-Scale Dataset for Video-and-Language Inference , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[62]  Atul Gupta,et al.  Active Learning Query Strategies for Classification, Regression, and Clustering: A Survey , 2020, Journal of Computer Science and Technology.

[63]  Svein Loeng Self-Directed Learning: A Core Concept in Adult Education , 2020, Education Research International.

[64]  Timothy M. Hospedales,et al.  Meta-Learning in Neural Networks: A Survey , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[65]  Kaiyong Zhao,et al.  AutoML: A Survey of the State-of-the-Art , 2019, Knowl. Based Syst..

[66]  Wenwu Zhu,et al.  Explainable Automated Graph Representation Learning with Hyperparameter Importance , 2021, ICML.

[67]  Yudong Chen,et al.  A Survey on Curriculum Learning , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[68]  Wenwu Zhu,et al.  AutoAttend: Automated Attention Representation Search , 2021, ICML.

[69]  Self-Directed Learning Online: An Opportunity to Binge , 2021 .

[70]  Wenwu Zhu,et al.  Graph Differentiable Architecture Search with Structure Learning , 2021, NeurIPS.

[71]  Xin Wang,et al.  Meta Hyperparameter Optimization with Adversarial Proxy Subsets Sampling , 2021, CIKM.

[72]  Yudong Chen,et al.  Curriculum Disentangled Recommendation with Noisy Multi-feedback , 2021, NeurIPS.

[73]  Yudong Chen,et al.  Curriculum Meta-Learning for Next POI Recommendation , 2021, KDD.

[74]  Zhikun Wei,et al.  AutoIAS: Automatic Integrated Architecture Searcher for Click-Trough Rate Prediction , 2021, CIKM.

[75]  Yasin Yilmaz,et al.  Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey , 2020, IEEE Transactions on Intelligent Transportation Systems.

[76]  Jiebo Luo,et al.  Small Data Challenges in Big Data Era: A Survey of Recent Progress on Unsupervised and Semi-Supervised Methods , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[77]  Zhihui Li,et al.  A Survey of Deep Active Learning , 2020, ACM Comput. Surv..