Studying the Usability of an Intervention to Promote Teachers’ Use of Robotics in STEM Education

We developed RoboSTEM, a portal for open educational resources for the use of robotics in teaching and learning, to help teachers learn how to design and implement lessons using robotics. In this article, the design and functionality of RoboSTEM and the theoretical foundations behind its design are described. Two usability testing studies are also presented. Participants were 13 pre- and in-service teachers. Results from both quantitative and qualitative data analyses show that (a) usability survey ratings were positive overall and improved after revisions from the first usability study to the second; (b) data from participants’ screen activities and interviews were aligned with the survey results; (c) high levels of behavioral, cognitive, and emotional engagement in RoboSTEM were observed; and (d) the positive impact of RoboSTEM on example-based learning was acknowledged among teachers. Applications of RoboSTEM as well as future research and development directions are discussed.

[1]  Prashant Doshi,et al.  Robotics to promote elementary education pre-service teachers' STEM engagement, learning, and teaching , 2015, Comput. Educ..

[2]  Margus Pedaste,et al.  LEARNING APPROACHES TO APPLYING ROBOTICS IN SCIENCE EDUCATION , 2013, Journal of Baltic Science Education.

[3]  W Kerry Mummery,et al.  Applying usability testing techniques to improve a health promotion website. , 2008, Health promotion journal of Australia : official journal of Australian Association of Health Promotion Professionals.

[4]  Shaunna F. Smith,et al.  The Power of Educational Robotics as an Integrated STEM Learning Experience in Teacher Preparation Programs , 2015 .

[5]  Miguel Nussbaum,et al.  Collaborative robotic instruction: A graph teaching experience , 2009, Comput. Educ..

[6]  Angeliki Poulymenakou,et al.  Developing a Usability Evaluation Method for e-Learning Applications: Beyond Functional Usability , 2009, Int. J. Hum. Comput. Interact..

[7]  Jörgen Lindh,et al.  Does lego training stimulate pupils' ability to solve logical problems? , 2007, Comput. Educ..

[8]  Susan M. Land,et al.  A conceptual framework for scaffolding III-structured problem-solving processes using question prompts and peer interactions , 2004 .

[9]  Yuxin Ma,et al.  Acquisition of Physics Content Knowledge and Scientific Inquiry Skills in a Robotics Summer Camp , 2007 .

[10]  Belén Curto,et al.  Resources and features of robotics learning environments (RLEs) in Spain and Latin America , 2013, TEEM '13.

[11]  Andrina Granic,et al.  Usability Testing and Expert Inspections Complemented by Educational Evaluation: A Case Study of an e-Learning Platform , 2011, J. Educ. Technol. Soc..

[12]  A. Bandura.  Social Foundations of Thought and Action: A Social Cognitive Theory , 1985 .

[13]  D. Brian Larkins,et al.  Application of the cognitive apprenticeship framework to a middle school robotics camp , 2013, SIGCSE '13.

[14]  J. Finn,et al.  Student engagement: What is it? Why does it matter? , 2012 .

[15]  K. A. Ericsson,et al.  Valid and Non-Reactive Verbalization of Thoughts During Performance of Tasks Towards a Solution to the Central Problems of Introspection as a Source of Scientific Data , 2003 .

[16]  M. Hannafin,et al.  A Framework for Designing Scaffolds That Improve Motivation and Cognition , 2013, Educational psychologist.

[17]  Dimitris Alimisis,et al.  Educational robotics: Open questions and new challenges , 2013 .

[18]  M. L. Silva.  Camtasia in the Classroom: Student Attitudes and Preferences for Video Commentary or Microsoft Word Comments During the Revision Process , 2012 .

[19]  Ru-Chu Shih,et al.  The impact of problem-based learning strategies on STEM knowledge integration and attitudes: an exploratory study among female Taiwanese senior high school students , 2011 .

[20]  Berit Eika,et al.  Effects of modelling examples in complex procedural skills training: a randomised study , 2013, Medical education.

[21]  Kamini Jaipal-Jamani,et al.  Effect of Robotics on Elementary Preservice Teachers’ Self-Efficacy, Science Learning, and Computational Thinking , 2017 .

[22]  So Yoon Yoon,et al.  Validation of the Teaching Engineering Self‐Efficacy Scale for K‐12 Teachers: A Structural Equation Modeling Approach , 2014 .

[23]  Florence R. Sullivan,et al.  Robotics and Discovery Learning: Pedagogical Beliefs, Teacher Practice, and Technology Integration , 2009 .

[24]  Fabiane Barreto Vavassori Benitti,et al.  Exploring the educational potential of robotics in schools: A systematic review , 2012, Comput. Educ..

[25]  Mesut Duran,et al.  Urban High School Students’ IT/STEM Learning: Findings from a Collaborative Inquiry- and Design-Based Afterschool Program , 2013, Journal of Science Education and Technology.

[26]  Mark Hawkes,et al.  Variables of Interest in Exploring the Reflective Outcomes of Network-Based Communication , 2001 .

[27]  Monica McGill,et al.  Learning to Program with Personal Robots: Influences on Student Motivation , 2012, TOCE.

[28]  Teresa Roselli,et al.  An approach to usability evaluation of e-learning applications , 2006, Universal Access in the Information Society.

[29]  Randall D. Beer,et al.  Using autonomous robotics to teach science and engineering , 1999, Commun. ACM.

[30]  Michael Hansen,et al.  Investigating the Relationship between STEM Learning Principles and Student Achievement in Math and Science , 2014, American Journal of Education.

[31]  S. Loucks-Horsley,et al.  Research on Professional Development for Teachers of Mathematics and Science: The State of the Scene , 1999 .

[32]  Nikol Rummel,et al.  Learning to collaborate while being scripted or by observing a model , 2009, Int. J. Comput. Support. Collab. Learn..

[33]  Giovanni Toffetti Carughi,et al.  Web Usability: Principles and Evaluation Methods , 2006, Web Engineering.

[34]  Aykut Hamit Turan,et al.  ASSESSING THE USABILITY OF UNIVERSITY WEBSITES: AN EMPIRICAL STUDY ON NAMIK KEMAL UNIVERSITY , 2012 .

[35]  Elizabeth Sklar,et al.  Using RoboCup in university-level computer science education , 2004, JERC.

[36]  Bradley Barker,et al.  Examining fidelity of program implementation in a STEM-oriented out-of-school setting , 2014 .

[37]  James Levin,et al.  Teaching Teleapprenticeships: Electronic Network-Based Educational Frameworks for Improving Teacher Education , 1998, Interact. Learn. Environ..

[38]  C. Kim,et al.  Guidelines for facilitating the development of learning communities in online courses , 2014, J. Comput. Assist. Learn..

[39]  Alexander Renkl,et al.  Toward an Instructionally Oriented Theory of Example-Based Learning , 2014, Cogn. Sci..

[40]  Kelly E. Knight,et al.  School Disengagement as a Predictor of Dropout, Delinquency, and Problem Substance Use During Adolescence and Early Adulthood , 2012, Journal of youth and adolescence.

[41]  B. Glaser.  The Constant Comparative Method of Qualitative Analysis , 1965 .

[42]  Kurt Becker,et al.  Integrative Approaches among Science,Technology, Engineering and Mathematics (STEM) Subjects on Students' Learning: AMeta-Analysis , 2009 .

[43]  Megan Tschannen-Moran,et al.  Teacher efficacy: capturing an elusive construct , 2001 .

[44]  Kristin E. Voelkl.  Identification with School , 1997, American Journal of Education.

[45]  Amy Eguchi,et al.  Educational Robotics Theories and Practice: Tips for how to do it Right , 2012 .

[46]  Chanmin Kim,et al.  Design and implementation of volitional control support in mathematics courses , 2013 .

[47]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[48]  Jennifer A. Fredricks,et al.  School Engagement: Potential of the Concept, State of the Evidence , 2004 .

[49]  T. Kindermann,et al.  A Motivational Perspective on Engagement and Disaffection , 2009 .

[50]  Shi-Jer Lou,et al.  Attitudes towards science, technology, engineering and mathematics (STEM) in a project-based learning (PjBL) environment , 2013 .

[51]  D. Schunk,et al.  Peer Models : Influence on Children's Self-Efficacy and Achievement , 1985 .

[52]  Lev Vygotsky.  Mind in society , 1978 .

[53]  Marta Díaz,et al.  Comparing two LEGO Robotics-based interventions for social skills training with children with ASD , 2013, 2013 IEEE RO-MAN.

[54]  Bradley S. Barker,et al.  Impact of Robotics and Geospatial Technology Interventions on Youth STEM Learning and Attitudes , 2010 .

[55]  Marina Umaschi Bers,et al.  Teaching Partnerships: Early Childhood and Engineering Students Teaching Math and Science Through Robotics , 2005 .

[56]  S. Christenson,et al.  Jingle, Jangle, 1 and Conceptual Haziness 2 : Evolution and Future Directions of the Engagement Construct , 2012 .

[57]  Elizabeth R. Kazakoff,et al.  The Effect of a Classroom-Based Intensive Robotics and Programming Workshop on Sequencing Ability in Early Childhood , 2013 .

[58]  M. David Merrill,et al.  First principles of instruction , 2012 .

[59]  BenittiFabiane Barreto Vavassori.  Exploring the educational potential of robotics in schools , 2012 .

[60]  Eric Wang,et al.  LEGO-based Robotics in Higher Education: 15 Years of Student Creativity , 2014 .

[61]  Bruno S. Silvestre,et al.  Social Media? Get Serious! Understanding the Functional Building Blocks of Social Media , 2011 .

[62]  Amy Fry,et al.  Usability Testing for e-Resource Discovery: How Students Find and Choose e-Resources Using Library Web Sites , 2011 .

[63]  Amela Karahasanovic,et al.  Comparing of feedback-collection and think-aloud methods in program comprehension studies , 2009, Behav. Inf. Technol..