Assessment in Smart Learning Environments: Psychological factors affecting perceived learning

Abstract Smart learning environments (SLEs) are spaces that incorporate technology to improve students learning outcomes. This growing body of literature typically examines day-to-day learning, but very little research considers the use of these environments for formative and summative assessment. We investigate how SLEs can be used to conduct successful real time simulations for summative assessment, using an innovative crisis management simulation in a unique smart learning environment at the University of South Wales. Using a PLS SEM methodology, this research demonstrates that the psychological factors of social support and career relatedness show significant positive effects upon perceived learning. Surprisingly, the perceived ease of use of the technology does not significantly affect perceived learning. This research contributes a valuable understanding of simulation learning within SLEs, and its relationship to positive outcomes in students' perceived learning. Implications for both academics and practitioners are discussed, and suggestions for future research are given.

[1]  Oscar Camacho Nieto,et al.  Collaborative learning in postgraduate level courses , 2015, Comput. Hum. Behav..

[2]  Fred D. Davis,et al.  User Acceptance of Computer Technology: A Comparison of Two Theoretical Models , 1989 .

[3]  Rafael Bravo,et al.  Analysing teamwork in higher education: an empirical study on the antecedents and consequences of team cohesiveness , 2019 .

[4]  Luigi Proserpio,et al.  Teaching without the teacher? Building a learning environment through computer simulations , 2012, Int. J. Inf. Manag..

[5]  Rosemary Luckin,et al.  Beyond Web 2.0: mapping the technology landscapes of young learners , 2009, J. Comput. Assist. Learn..

[6]  L. Vos,et al.  Marketing simulation games: student and lecturer perspectives , 2010 .

[7]  P. Wilcox,et al.  ‘It was nothing to do with the university, it was just the people’: the role of social support in the first‐year experience of higher education , 2005 .

[8]  Leonard A. Annetta,et al.  A meta-analysis with examination of moderators of student cognition, affect, and learning outcomes while using serious educational games, serious games, and simulations , 2018, Comput. Hum. Behav..

[9]  Monica L. Forret,et al.  Learning The Art of Networking: A Critical Skill for Enhancing Social Capital and Career Success , 2008 .

[10]  Joseph P. Robinson Evidence of a Differential Effect of Ability Grouping on the Reading Achievement Growth of Language-Minority Hispanics , 2008 .

[11]  Ana Isabel Canhoto,et al.  Learning From Simulation Design to Develop Better Experiential Learning Initiatives , 2016 .

[12]  R. Eynon,et al.  Digital natives: where is the evidence? , 2010 .

[13]  C. Carver,et al.  Assessing coping strategies: a theoretically based approach. , 1989, Journal of personality and social psychology.

[14]  Mehdi Farashahi,et al.  Effectiveness of teaching methods in business education: A comparison study on the learning outcomes of lectures, case studies and simulations , 2018 .

[15]  R. Fuller The big breach: An experiential learning exercise in mindful crisis communication , 2016 .

[16]  Erman Yükseltürk,et al.  Team shared mental model as a contributing factor to team performance and students' course satisfaction in blended courses , 2011, Comput. Hum. Behav..

[17]  Matt Elbeck,et al.  Toward Universal Definitions for Direct and Indirect Assessment , 2015 .

[18]  Marko Sarstedt,et al.  PLS-SEM: Indeed a Silver Bullet , 2011 .

[19]  Debra R. Comer,et al.  Factors That Affect Students’ Capacity to Fulfill the Role of Online Learner , 2015 .

[20]  R. Bosker,et al.  The influence of personal networks and social support on study attainment of students in university education , 2008 .

[21]  Daniel Bodemer,et al.  Group awareness in CSCL environments , 2011, Comput. Hum. Behav..

[22]  Frederik Beuk,et al.  Sales Simulation Games , 2016 .

[23]  Y. Lou,et al.  Within-Class Grouping: A Meta-Analysis , 1996 .

[24]  Natalie Simper,et al.  Assessing teamwork in undergraduate education: a measurement tool to evaluate individual teamwork skills , 2017 .

[25]  Nian-Shing Chen,et al.  Evolution Is not enough: Revolutionizing Current Learning Environments to Smart Learning Environments , 2016, International Journal of Artificial Intelligence in Education.

[26]  Alexander Serenko,et al.  Integrating Technology Addiction and Use: An Empirical Investigation of Online Auction Users , 2011, MIS Q..

[27]  Fred D. Davis Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[28]  Ejaz Ahmed,et al.  Investigating the influence of the most commonly used external variables of TAM on students' Perceived Ease of Use (PEOU) and Perceived Usefulness (PU) of e-portfolios , 2016, Comput. Hum. Behav..

[29]  K. Wentzel,et al.  Emotional Support and Expectations From Parents, Teachers, and Peers Predict Adolescent Competence at School , 2016 .

[30]  Qimei Chen,et al.  Regulatory focus and technology acceptance: Perceived ease of use and usefulness as efficacy , 2018 .

[31]  Making It Real: Using a Collaborative Simulation to Teach Crisis Communications , 2012 .

[32]  R. Arteaga Sánchez,et al.  Motivational factors that influence the acceptance of Moodle using TAM , 2010, Comput. Hum. Behav..

[33]  J. P. Maloney,et al.  Social support: a conceptual analysis. , 1997, Journal of advanced nursing.

[34]  Mahmoud Abd Ellatif,et al.  A proposed paradigm for smart learning environment based on semantic web , 2017, Comput. Hum. Behav..

[35]  R. Brennan Reflecting on Experiential Learning in Marketing Education , 2014 .

[36]  A. Girardi,et al.  A Systematic Literature Review of Teamwork Pedagogy in Higher Education , 2016 .

[37]  Penny Thompson How digital native learners describe themselves , 2013, Education and Information Technologies.

[38]  Kenneth G. Brown,et al.  Self-Assessment of Knowledge: A Cognitive Learning or Affective Measure? , 2010 .

[39]  Thomas Menkhoff,et al.  Enhancing students' learning process through interactive digital media: New opportunities for collaborative learning , 2015, Comput. Hum. Behav..

[40]  Rupert Ward,et al.  Developing a General Extended Technology Acceptance Model for E-Learning (GETAMEL) by analysing commonly used external factors , 2016, Comput. Hum. Behav..

[41]  Ying-Tien Wu,et al.  A review of features of technology-supported learning environments based on participants' perceptions , 2015, Comput. Hum. Behav..

[42]  K. Wentzel Social Relationships and Motivation in Middle School: The Role of Parents, Teachers, and Peers , 1998 .

[43]  Marie Good,et al.  Linking academic social environments, ego-identity formation, ego virtues, and academic success. , 2008, Adolescence.

[44]  Using Simulations in the Marketing Classroom , 2016 .

[45]  A. Burns,et al.  Perceived Communication Barriers to Learning with Computer Simulation , 1983 .

[46]  Blanka Klimova Assessment in Smart Learning Environment – A Case Study Approach , 2015 .

[47]  Dezhi Wu,et al.  Are your users actively involved? A cognitive absorption perspective in mobile training , 2015, Comput. Hum. Behav..

[48]  S. Folkman,et al.  Journal of Personality and Social Psychology If It Changes It Must Be a Process: Study of Emotion and Coping during Three Stages of a College Examination , 2022 .

[49]  Polona Tominc,et al.  Impact of support of teacher and compatibility with needs of study on usefulness of SPSS by students , 2015, Comput. Hum. Behav..

[50]  A. Tymon,et al.  The student perspective on employability , 2013 .

[51]  Fred D. Davis,et al.  Dead Or Alive? The Development, Trajectory And Future Of Technology Adoption Research , 2007, J. Assoc. Inf. Syst..

[52]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[53]  M. Levitt,et al.  Social support and achievement in childhood and early adolescene: A multicultural study , 1994 .

[54]  A. Bandura Perceived Self-Efficacy in Cognitive Development and Functioning , 1993, Educational Psychologist.

[55]  Viswanath Venkatesh,et al.  Determinants of Perceived Ease of Use: Integrating Control, Intrinsic Motivation, and Emotion into the Technology Acceptance Model , 2000, Inf. Syst. Res..

[56]  Declan Doyle,et al.  Using a business simulation to teach applied skills – the benefits and the challenges of using student teams from multiple countries , 2000 .

[57]  Princely Ifinedo,et al.  Business undergraduates' perceived use outcomes of Moodle in a blended learning environment: The roles of usability factors and external support , 2018, Telematics Informatics.

[58]  Christopher R. Jones,et al.  Net generation or Digital Natives: Is there a distinct new generation entering university? , 2010, Comput. Educ..

[59]  Gabriele Frankl,et al.  Collaboration Is Smart: Smart Learning Communities , 2013, CHI-KDD.

[60]  Mohamed Yeou,et al.  An Investigation of Students’ Acceptance of Moodle in a Blended Learning Setting Using Technology Acceptance Model , 2016 .

[61]  Szu-Yuan Sun,et al.  Usability, quality, value and e-learning continuance decisions , 2005, Comput. Educ..

[62]  Murat Akçayir,et al.  What makes you a digital native? Is it enough to be born after 1980? , 2016, Comput. Hum. Behav..

[63]  Michael J. Hannafin,et al.  Scaffolding problem solving in technology-enhanced learning environments (TELEs): Bridging research and theory with practice , 2011, Comput. Educ..

[64]  Jane E. Klobas,et al.  Evaluation of the efficacy of collaborative learning in face-to-face and computer-supported university contexts , 2006, Comput. Hum. Behav..

[65]  J. Bernhardt,et al.  Designing and Testing an Inventory for Measuring Social Media Competency of Certified Health Education Specialists , 2015, Journal of medical Internet research.

[66]  B. Weijters,et al.  The effect of rating scale format on response styles: the number of response categories and response catgory labels , 2010 .

[67]  Steven Shaw,et al.  Designing curriculum to shape professional social media skills and identity in virtual communities of practice , 2017, Comput. Educ..

[68]  J. Frankham Employability and higher education: the follies of the ‘Productivity Challenge’ in the Teaching Excellence Framework , 2017 .

[69]  D. Bacon Reporting Actual and Perceived Student Learning in Education Research , 2016 .

[70]  T. Wills,et al.  Stress, social support, and the buffering hypothesis. , 1985, Psychological bulletin.

[71]  Isabel Buil,et al.  Exploring students' flow experiences in business simulation games , 2018, J. Comput. Assist. Learn..

[72]  Adolfo Obaya Getting Cooperative Learning. , 1999 .

[73]  Mehdi Darban,et al.  Do emotions matter in technology training? Exploring their effects on individual perceptions and willingness to learn , 2016, Comput. Hum. Behav..

[74]  Kathryn R. Wentzel,et al.  Social-motivational processes and interpersonal relationships: Implications for understanding motivation at school. , 1999 .

[75]  Clement C. Chen,et al.  The Association between Students' Style of Learning Preferences, Social Presence, Collaborative Learning and Learning Outcomes. , 2018 .

[76]  Ying-Chieh Liu,et al.  Self-efficacy as the moderator: Exploring driving factors of perceived social support for mainland Chinese students in Taiwan , 2016, Comput. Hum. Behav..

[77]  William H. Redmond,et al.  On the Duration of Simulations: An Exploration of Minimum Effective Length , 1989 .

[78]  L. Lawton,et al.  Blissful Ignorance: The Problem of Unrecognized Incompetence and Academic Performance , 2002 .

[79]  Kamla Ali Al-Busaidi,et al.  An empirical investigation linking learners’ adoption of blended learning to their intention of full e-learning , 2013, Behav. Inf. Technol..

[80]  James T. Low Guidelines for the Use of Business Simulation Games , 1980 .

[81]  W. Bennis,et al.  How business schools lost their way. , 2005, Harvard business review.

[82]  Chyan Yang,et al.  Why do we blog? From the perspectives of technology acceptance and media choice factors , 2013, Behav. Inf. Technol..

[83]  Y. Sohn,et al.  Effects of grit on academic achievement and career-related attitudes of college students in Korea , 2017 .

[84]  Lynn Vos,et al.  Effects of Participation in a Simulation Game on Marketing Students’ Numeracy and Financial Skills , 2013 .

[85]  D. Bacon Comparing Direct Versus Indirect Measures of the Pedagogical Effectiveness of Team Testing , 2011 .

[86]  R. Bagozzi,et al.  On the evaluation of structural equation models , 1988 .

[87]  Marko Sarstedt,et al.  Partial least squares structural equation modeling (PLS-SEM): An emerging tool in business research , 2014 .

[88]  Jing Lv,et al.  Social presence in relation to students' satisfaction and learning in the online environment: A meta-analysis , 2017, Comput. Hum. Behav..

[89]  Joseph F. Hair,et al.  Partial least squares structural equation modeling (PLS-SEM): An emerging tool in business research , 2014 .

[90]  Jonathan Michael Spector,et al.  Conceptualizing the emerging field of smart learning environments , 2014, Smart Learning Environments.

[91]  Paul A. Pavlou,et al.  Understanding and Predicting Electronic Commerce Adoption: An Extension of the Theory of Planned Behavior , 2006, MIS Q..

[92]  Meredith Ringel Morris,et al.  Beyond "social protocols": multi-user coordination policies for co-located groupware , 2004, CSCW.

[93]  Louis Leung,et al.  Multiple determinants of life quality: the roles of Internet activities, use of new media, social support, and leisure activities , 2005, Telematics Informatics.

[94]  Christian Steglich,et al.  Emergent achievement segregation in freshmen learning community networks , 2018 .

[95]  Ashok Ranchhod,et al.  Evaluating the educational effectiveness of simulation games: A value generation model , 2014, Inf. Sci..

[96]  M. Sarstedt,et al.  A new criterion for assessing discriminant validity in variance-based structural equation modeling , 2015 .

[97]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[98]  A. K. M. Najmul Islam,et al.  E-learning system use and its outcomes: Moderating role of perceived compatibility , 2016, Telematics Informatics.

[99]  Jonte Bernhard,et al.  What matters for students’ learning in the laboratory? Do not neglect the role of experimental equipment! , 2018, Instructional Science.

[100]  Guan-Yu Lin,et al.  Anonymous versus identified peer assessment via a Facebook-based learning application: Effects on quality of peer feedback, perceived learning, perceived fairness, and attitude toward the system , 2018, Comput. Educ..

[101]  Henry Adobor,et al.  Management simulations: determining their effectiveness , 2006 .

[102]  S. Cobb Presidential Address-1976. Social support as a moderator of life stress. , 1976, Psychosomatic medicine.

[103]  D. Russell,et al.  Perceived parental social support and academic achievement: an attachment theory perspective. , 1994, Journal of personality and social psychology.

[104]  Matthew J. Xerri,et al.  Student engagement in academic activities: a social support perspective , 2018 .

[105]  A. K. M. Najmul Islam,et al.  Investigating e-learning system usage outcomes in the university context , 2013, Comput. Educ..

[106]  Thomas Ashby Wills,et al.  Social support and interpersonal relationships. , 1991 .