Development and Validation of an Instrument for Assessing Distance Education Learning Environments in Higher Education: The Distance Education Learning Environments Survey (DELES)

The purpose of this study was to develop and validate a new learning environments instrument designed to aid investigators and practitioners in measuring and researching the psychosocial learning environment in post-secondary distance education. Using a three-stage approach, the Distance Education Learning Environments Survey (DELES) was developed, field-tested with 680 distance education students, and then validated. The DELES has 34 items allocated to six scales: (1) Instructor Support; (2) Student Interaction and Collaboration; (3) Personal Relevance; (4) Authentic Learning; (5) Active Learning; and (6) Student Autonomy. An additional scale of Enjoyment was included in this study to explore associations between the psychosocial learning environment and student affective traits. Each learning environment item had a factor loading of at least 0.50 with its own scale, and less than 0.50 with all other scales. The alpha reliability coefficient for each scale ranged from 0.75 to 0.94. Simple correlations between Enjoyment and the DELES scales ranged from 0.12 to 0.31, with the scale of Personal Relevance having the strongest correlation with Enjoyment when all other scales were mutually controlled. The DELES, an online instrument that can be utilized by students at any location, eliminates data transfer errors and does not allow for non-responses, adding to the overall validity of the instrument. The development of DELES relied extensively on literature pertaining to high-quality distance education and expert content validation techniques. It treats distance learning as having a distinct social-psychological climate unlike those found in other post-secondary classroom environments.

[1]  T. Newcomb.  The consistency of certain extrovert-introvert behavior patterns in 51 problem boys. , 1929 .

[2]  M. May,et al.  Studies in the nature of character, [part] 1: Studies in deceit: book 1, General methods and results; book 2, Statistical methods and results. , 1930 .

[3]  F. Heider,et al.  Principles of topological psychology , 1936 .

[4]  H. Murray.  Explorations in personality : a clinical and experimental study of fifty men of college age , 1939 .

[5]  Benjamin S. Bloom,et al.  Methods in Personality Assessment , 1957 .

[6]  C. R. Pace,et al.  An approach to the measurement of psychological characteristics of college environments. , 1958 .

[7]  William F. Ogburn,et al.  On Culture And Social Change , 1964 .

[8]  G. Stern,et al.  B==F(P,E). , 1964, Journal of projective techniques & personality assessment.

[9]  E. Thoma.  Interpersonal Diagnosis of Personality , 1965 .

[10]  L. R. Goldberg,et al.  Comparative validity of different strategies of constructing personality inventory scales. , 1967, Psychological bulletin.

[11]  Herbert J. Walberg,et al.  CLASSROOM CLIMATE AND INDIVIDUAL LEARNING. , 1968 .

[12]  R. G. Owens.  Organizational behavior in education , 1970 .

[13]  Eva Dobozy,et al.  Psychology Applied to Teaching , 1971 .

[14]  J. M. Steele,et al.  An Instrument for Assessing Instructional Climate through Low-Inference Student Judgments1 , 1971 .

[15]  Howard B. Lee,et al.  Foundations of Behavioral Research , 1973 .

[16]  Rudolf H. Moos,et al.  Manual : classroom environment scale , 1974 .

[17]  R. Moos.  Determinants of Physiological Responses to Symbolic Stimuli: The Role of the Social Environment , 1974, International journal of psychiatry in medicine.

[18]  Herbert J. Walberg,et al.  Evaluating educational performance : a sourcebook of methods, instruments, and examples , 1974 .

[19]  R. Moos,et al.  Health and the social environment , 1974 .

[20]  Rudolf H. Moos,et al.  The human context: Environmental determinants of behavior , 1976 .

[21]  B. Fraser.  Selection and validation of attitude scales for curriculum evaluation , 1977 .

[22]  Herbert J. Walberg,et al.  Predictive Validity of Student Perception: A Cross-Cultural Replication , 1977 .

[23]  Jae-On Kim,et al.  Introduction to Factor Analysis: What It Is and How To Do It , 1978 .

[24]  Rudolf H. Moos,et al.  Evaluating Educational Environments , 1979 .

[25]  Chester A. Schriesheim,et al.  Controlling Acquiescence Response Bias by Item Reversals: The Effect on Questionnaire Validity , 1981 .

[26]  Richard M. Jaeger,et al.  Statistics, a spectator sport , 1983 .

[27]  David F. Treagust,et al.  Development of an instrument for assessing classroom psychosocial environment at universities and colleges , 1986 .

[28]  R. Slavin.  Educational Psychology: Theory and Practice , 1986 .

[29]  B. Fraser,et al.  Using Short Forms of Classroom Climate Instruments to Assess and Improve Classroom Psychosocial Environment. , 1986 .

[30]  Jeri Benson,et al.  Detecting Item BIAS in Affective Scales , 1987 .

[31]  Jack J. Phillips,et al.  Handbook of Training Evaluation and Measurement Methods , 1987 .

[32]  Chester A. Schriesheim,et al.  The Effect of Negation and Polar Opposite Item Reversals on Questionnaire Reliability and Validity: An Experimental Investigation , 1991 .

[33]  Max Oelschlaeger,et al.  The Idea of Wilderness , 1991 .

[34]  B. Fraser.  Assessing the Climate of Science Laboratory Classes. , 1992 .

[35]  Jack Levy,et al.  Do You Know What You Look Like?: Interpersonal Relationships In Education , 1993 .

[36]  B. Fraser,et al.  An evaluation of computer assisted learning in geography in Singapore , 1994 .

[37]  Jannette Kirkwood,et al.  Students' Anxiety in Learning through Distance Education. , 1994 .

[38]  Darrell L. Fisher,et al.  Interpersonal behaviour in senior high school biology classes , 1995 .

[39]  Campbell J. McRobbie,et al.  Science Laboratory Classroom Environments at Schools and Universities: A Cross‐National Study∗ , 1995 .

[40]  Barry J. Fraser,et al.  The development and validation of a distance and open learning environment scale , 1995 .

[41]  Raymond B. Cattell,et al.  Personality Structure and the New Fifth Edition of the 16PF , 1995 .

[42]  B. Fraser,et al.  Evolution and Validation of a Personal Form of an Instrument for Assessing Science Laboratory Classroom Environments. , 1995 .

[43]  R. Moos.  Understanding environments: The key to improving social processes and program outcomes , 1996, American journal of community psychology.

[44]  Assessing Teacher-Student Interpersonal Relationships in Science Classes. , 1996 .

[45]  Swee Chiew Goh,et al.  Validation of an Elementary School Version of the Questionnaire on Teacher Interaction , 1996 .

[46]  Angela F. L. Wong,et al.  Perceptions of interpersonal teacher behaviour in secondary science classrooms: A cross-national story , 1996 .

[47]  Judi Harris,et al.  Virtual Architecture: Designing and Directing Curriculum-Based Telecomputing , 1997 .

[48]  Editorial: Assessing students' understanding of science , 1997 .

[49]  Michael Newby,et al.  An Instrument for Assessing the Learning Environment of a Computer Laboratory , 1997 .

[50]  Darrell L. Fisher,et al.  Monitoring constructivist classroom learning environments , 1997 .

[51]  Darren George,et al.  SPSS for Windows Step by Step: A Simple Guide and Reference , 1998 .

[52]  Barry J. Fraser,et al.  Teacher Interpersonal Behaviour, Classroom Environment and Student Outcomes in Primary Mathematics in Singapore , 1998 .

[53]  J. Bain,et al.  Describing Computer-facilitated Learning Environments in Higher Education , 1998 .

[54]  J. Campbell,et al.  Personal and Class Forms of Classroom Environment Instruments , 1998 .

[55]  Wim Veen,et al.  Computer-Mediated Communication Environments in Teacher Education: Computer Conferencing and the Supervision of Student Teachers , 1998 .

[56]  Barry J. Fraser,et al.  Classroom Environment Instruments: Development, Validity and Applications , 1998 .

[57]  Kenneth Tobin,et al.  Qualitative and Quantitative Landscapes of Classroom Learning Environments , 1998 .

[58]  B. Fraser.  Science Learning Environments: Assessment, Effects and Determinants , 1998 .

[59]  D. Noble.  Digital diploma mills: The automation of higher education , 1998 .

[60]  Kenneth Tobin,et al.  Issues and Trends in the Teaching of Science , 1998 .

[61]  D. Robinson,et al.  Individual study, interactive multimedia, or cooperative learning : Which activity best supplements lecture-based distance education? , 1999 .

[62]  Ronald A. Phipps,et al.  What''s the Difference? A Review of Contemporary Research on the Effectiveness of Distance Learning , 1999 .

[63]  Dorit Maor,et al.  A Teacher Professional Development Program on Using a Constructivist Multimedia Learning Environment , 1999 .

[64]  Rena M. Palloff,et al.  Building Learning Communities in Cyberspace: Effective Strategies for the Online Classroom. Jossey-Bass Higher and Adult Education Series. , 1999 .

[65]  Oleg Liber,et al.  A framework for pedagogical evaluation of virtual learning environments , 1999 .

[66]  Olugbemiro J. Jegede,et al.  Differences between low and high achieving distance learners in locus of control and metacognition , 1999 .

[67]  George Puan Loon Teh.  Assessing Student Perceptions of Internet-Based Online Learning Environments. , 1999 .

[68]  J. Michael Spector,et al.  Designing Collaborative Distance Learning Environments for Complex Domains , 1999 .

[69]  Kenneth Tobin,et al.  Student and Teacher Perspectives in Computer-Mediated Learning Environments in Teacher Education , 1999 .

[70]  Active Learning in a Large First Year Biology Class: A Collaborative Resource-Based Study Project on "AIDS in Science and Society". , 1999 .

[71]  Amy S. C. Leh.  Computer-Mediated Communication and Foreign Language Learning via Telecommunication Technology , 1999 .

[72]  Jill M. Aldridge,et al.  Investigating Classroom Environments in Taiwan and Australia With Multiple Research Methods , 1999 .

[73]  Catalysing Changes in Research on Learning Environments: Regional Editor's Introduction , 1999 .

[74]  Mary Frances Grasinger.  Successful Distance Learning Teaching via Synchronous Video , 1999 .

[75]  David B. Zandvliet,et al.  The physical and psychosocial environment associated with classrooms using new information technologies : a cross-national study. , 1999 .

[76]  Peter Jamieson,et al.  IMPROVING TEACHING BY TELECOMMUNICATIONS MEDIA: EMPHASISING PEDAGOGY RATHER THAN TECHNOLOGY , 1999 .

[77]  D. Diaz,et al.  Students' Learning Styles in Two Classes: Online Distance Learning and Equivalent On-Campus , 1999 .

[78]  Graham R. Gibbs.  Learning how to learn using a virtual learning environment for philosophy , 1999, J. Comput. Assist. Learn..

[79]  Darrell L. Fisher,et al.  Assessment and Investigation of Constructivist Science Learning Environments in Korea , 1999 .

[80]  PaK ivi TynjaK,et al.  Towards expert knowledge ? A comparison between a constructivist and a traditional learning environment in the university , 1999 .

[81]  Enhancing Quality through Distributed Education , 1999 .

[82]  Mike Markel,et al.  Distance Education and the Myth of the New Pedagogy , 1999 .

[83]  T. Ponzurick,et al.  Delivering Graduate Marketing Education: An Analysis of Face-to-Face versus Distance Education , 2000 .

[84]  Scott Walker,et al.  Internet-Based Distance Education: Barriers, Models, and New Research , 2000, WebNet.

[85]  Darrell L. Fisher,et al.  Classroom Environment and Teacher Interpersonal Behaviour in Secondary Science Classes in Korea , 2000 .

[86]  Dorit Maor,et al.  Assessing the efficacy of online teaching with the Constructivist online learning environment survey , 2000 .

[87]  Margaret Haughey,et al.  Building Learning Communities in Cyberspace: Effective Strategies for the Online Classroom by Rena M. Palloff and Keith Pratt , 2000 .

[88]  P. Whitten,et al.  Re‐thinking instructional immediacy for web courses: A social cognitive exploration , 2000 .

[89]  Darrell L. Fisher,et al.  Interpersonal Behavior, Laboratory Learning Environments, and Student Outcomes in Senior Biology Classes. , 2000 .

[90]  M. Fish,et al.  The Classroom Systems Observation Scale: Development of an instrument to assess classrooms using a systems perspective , 2000 .

[91]  Karen L. Murphy,et al.  Images of Texan and Mexican cultures shared in a telecommunications partnership , 2000 .

[92]  Michael Molenda,et al.  Issues and Trends in Instructional Technology. , 2000 .

[93]  Campbell J. McRobbie,et al.  Changing the Learning Environment to Enhance Explaining and Understanding in a Year 12 Chemistry Classroom , 2000 .

[94]  Swee Chiew Goh,et al.  Teacher Interpersonal Behavior and Elementary Students' Outcomes , 2000 .

[95]  Uschi Felix,et al.  A multivariate analysis of students' experience of web based learning , 2001 .

[96]  Kelli Cargile Cook.  Online Professional Communication: Pedagogy, Instructional Design, and Student Preference in Internet-Based Distance Education , 2000 .

[97]  Christine B. Johnson,et al.  Supporting distributed communities of practice with interactive television , 2000 .

[98]  Martin Owen,et al.  Structure and Discourse in a telematic learning environment , 2000, J. Educ. Technol. Soc..

[99]  Farhad Saba.  Research in Distance Education: A Status Report , 2000 .

[100]  Dorothy Harnish,et al.  Issues in the Evaluation of Large-Scale Two-Way Interactive Distance Learning Systems. , 2000 .

[101]  Jill M. Aldridge,et al.  Constructivist learning environments in a crossnational study in Taiwan and Australia , 2000 .

[102]  Effects of Gender and Collaboration on College Students' Performance on a Piagetian Spatial Task , 2000 .

[103]  Rafi Nachmias,et al.  Web-Based Learning Environments , 2000 .

[104]  Gail Flanagan,et al.  Teaching and Learning at a Distance - Foundations of Distance Education: Simonson, M., Smaldino, S., Albright, M., and Zvacek, S. (2000). Upper Saddle River, NJ: Prentice-Hall, 241 pages, ISBN: 0-13769258-7 , 2000, Internet and Higher Education.

[105]  Hayley J. Mayall,et al.  Measuring the Effect of Distance Education on the Learning Experience: Teaching Accounting via PictureTel. , 2000 .

[106]  Robert Tinker.  Ice Machines, Steamboats, and Education: Structural Change and Educational Technologies. , 2000 .

[107]  J. Jackson Barnette,et al.  Effects of Stem and Likert Response Option Reversals on Survey Internal Consistency: If You Feel the Need, There is a Better Alternative to Using those Negatively Worded Stems , 2000 .

[108]  Jill M. Aldridge,et al.  Junior Secondary Mathematics Student's Learning Environment and Satisfaction in Brunei Darussalam , 2001 .

[109]  James C. Taylor,et al.  Fifth generation distance education , 2001 .

[110]  Karen Swan,et al.  Virtual interaction: Design factors affecting student satisfaction and perceived learning in asynchronous online courses , 2001 .

[111]  A Nurturing Learning Environment—On‐or OffLine , 2001 .

[112]  Vanessa Chang,et al.  The validation and application of a new learning environment instrument to evaluate online learning in higher education , 2001 .

[113]  Swee Chiew Goh,et al.  Investigation of tertiary classroom learning environment in Singapore , 2001 .

[114]  Choon Lang Quek,et al.  Determinants and effects of perceptions of chemistry classroom learning environments in secondary school gifted education classes in Singapore , 2001 .

[115]  David de Vaus,et al.  Research Design in Social Research , 2001 .

[116]  M M Cannon,et al.  "We're living what we're learning": student perspectives in distance learning degree and certificate programs in public health. , 2001, Journal of public health management and practice : JPHMP.

[117]  Cross-National Validation and Use of Classroom Environment Scales. , 2001 .

[118]  C. Tu,et al.  A paradigm shift for online community research , 2001 .

[119]  Charlotte N. Gunawardena,et al.  A cross‐cultural study of group process and development in online conferences , 2001 .

[120]  Lauren Cifuentes,et al.  Using web tools, collaborating, and learning online , 2001 .

[121]  Mary Graham,et al.  Enhancing the learning environment for distance education students , 2001 .

[122]  Validation of the Chinese Language Classroom Environment Inventory (CLCEI) for use in Singapore secondary schools , 2001 .

[123]  Myint Swe Khine.  STUDY OF LEARNING ENVIRONMENT FOR IMPROVING SCIENCE EDUCATION IN BRUNEI , 2002 .

[124]  Jan Wilson.  The Power of Distance Learning , 2002 .

[125]  R. Moos.  2001 INVITED ADDRESS: The Mystery of Human Context and Coping: An Unraveling of Clues , 2002, American journal of community psychology.

[126]  S. Marginson.  Education in the Global Market: Lessons from Australia. , 2002 .

[127]  Jill M. Aldridge,et al.  INVESTIGATING FACTORS THAT PREVENT SCIENCE TEACHERS FROM CREATING POSITIVE LEARNING ENVIRONMENTS IN TAIWAN , 2002 .

[128]  MEASURING CULTURALLY SENSITIVE FACTORS OF CLASSROOM LEARNING ENVIRONMENTS WITH THE CLEQ , 2002 .

[129]  M. O'Reilly,et al.  Interaction online: Above and beyond requirements of assessment , 2002 .

[130]  Laboratory Classroom Environments in Korean High Schools. , 2002 .

[131]  P. Kollock,et al.  Communities in Cyberspace , 2002 .

[132]  Perry den Brok,et al.  Perceptions of Asian-American and Hispanic-American teachers and their students on teacher interpersonal communication style , 2002 .

[133]  The Role of Learning Environment, Achievement, and Student and Teacher Attitudes in a Science Mentoring Program for Beginning Elementary School Teachers. , 2002 .

[134]  CURRICULUM DEVELOPMENT AND LEARNING ENVIRONMENT RESEARCH IN KOREA , 2002 .

[135]  Alvin Y. Wang,et al.  Predictors of Performance in the Virtual Classroom: Identifying and Helping At-Risk Cyber-Students , 2002 .

[136]  Myint Swe Khine,et al.  Studies In Educational Learning Environments: An International Perspective , 2002 .

[137]  Robert A. Wisher,et al.  The Effectiveness of Web-Based Instruction: An Initial Inquiry. , 2002 .

[138]  Bridging Walls and Crossing Borders in the Caribbean. , 2002 .

[139]  Peter S. Cookson.  Editorial ~ Online Postgraduate Education: Some reflections , 2002 .

[140]  Peter Charles Taylor,et al.  Interpretive analysis of an internet-based course constructed using a new courseware tool called Moodle , 2002 .

[141]  Swee Chiew Goh.  STUDIES ON LEARNING ENVIRONMENTS IN SINGAPORE CLASSROOMS , 2002 .

[142]  David B. Zandvliet.  LEARNING ENVIRONMENTS IN TECHNOLOGY-RICH CLASSROOMS , 2002 .

[143]  Joi L. Moore,et al.  Student Perceptions as Distance Learners in Internet-Based Courses , 2002 .

[144]  Barry J. Fraser,et al.  Investigating the Learning Environment in Canadian Mathematics and Science Classrooms in Which Laptop Computers Are Used , 2002 .

[145]  P. Twining.  Oversold and underused: computers in the classroom , 2002 .

[146]  Barry J. Fraser,et al.  LEARNING ENVIRONMENTS RESEARCH: YESTERDAY, TODAY AND TOMORROW , 2002 .

[147]  S. Brookfield.  Overcoming Alienation as the Practice of Adult Education: The Contribution of Erich Fromm to a Critical Theory of Adult Learning and Education , 2002 .

[148]  David Mioduser,et al.  The Development of Social Climate in Virtual Learning Discussion Groups , 2002 .

[149]  Jill M. Aldridge,et al.  Learning Environment and its Association with Student Satisfaction Among Mathematics Students in Brunei Darussalam , 2002 .

[150]  John G. Hedberg,et al.  Recommendations for Teaching in a Distributed Learning Environment: The Students' Perspective. , 2002 .

[151]  M. David Merrill,et al.  First principles of instruction , 2012 .

[152]  LEARNING ENVIRONMENT RESEARCH IN INDONESIA , 2002 .

[153]  Paul A. Kirschner,et al.  The sociability of computer-supported collaborative learning environment , 2002, J. Educ. Technol. Soc..

[154]  Christopher K. Morgan,et al.  Is Enough Too Much? The dilemma for online distant learner supporters , 2003 .

[155]  P. Fahy.  Indicators of Support in Online Interaction , 2003 .

[156]  Jeffrey P. Dorman,et al.  Cross-National Validation of the What is Happening in this Class? (WIHIC) Questionnaire using Confirmatory Factor Analysis , 2003 .

[157]  Hendrik Witt,et al.  Preface , 2004, Molecular and Cellular Endocrinology.

[158]  H. Mcclelland,et al.  Learning environments. , 2004, Accident and emergency nursing.

[159]  Jill M. Aldridge,et al.  Use of Multitrait-Multimethod Modelling to Validate Actual and Preferred Forms of the Technology-Rich Outcomes-Focused Learning Environment Inventory (TROFLEI). , 2004 .

[160]  B. Fraser,et al.  Learning environments in information and communications technology classrooms , 2004 .

[161]  S. C. Ehrmann.  IMPLEMENTING THE SEVEN PRINCIPLES: Technology as Lever , 2004 .

[162]  Bridget Somekh,et al.  Taking the sociological imagination to school: an analysis of the (lack of) impact of information and communication technologies on education systems , 2004 .