No comparison: Distance education finds a new use for ‘No significant difference’

Media comparison studies have long been criticized as an inappropriate research design for measuring the effectiveness of instructional technology. However, a resurgence in their use has recently been noted in distance education for program evaluation purposes. An analysis of the research design will detail why such a methodology is an inappropriate approach to such an investigation. Increased access to such programming does not seem to serve as a satisfactory benefit for the implementation of distance education efforts. Stakeholders desire to prove that participants in distance-delivered courses receive the same quality of instruction off-campus as those involved in the “traditional” classroom setting. However, the desire to prove that the quality of such distributed offerings is equal to the quality of on-campus programming often results in comparisons of achievement between the two groups of student participants. Statistically, such a research design almost guarantees that the desired outcome will be attained—that indeed distance learners perform as well as campus-based students.

[1]  Michael Barry,et al.  A review of distance‐learning studies in the U. S. military , 1995 .

[2]  K. Popper,et al.  Conjectures and refutations;: The growth of scientific knowledge , 1972 .

[3]  P. Ewell.  No Significant Difference , 1999 .

[4]  Charles Schlosser,et al.  Distance Education: Review of the Literature , 1994 .

[5]  H. Keselman,et al.  Consequences of Assumption Violations Revisited: A Quantitative Review of Alternatives to the One-Way Analysis of Variance F Test , 1996 .

[6]  D. R. Garrison,et al.  An analysis and evaluation of audio teleconferencing to facilitate education at a distance , 1990 .

[7]  L. P. Saettler.  A history of instructional technology , 1967 .

[8]  John K. Burton,et al.  Computer programming and generalized problem-solving skills: in search of direction , 1987 .

[9]  Gary R. Morrison,et al.  The media effects question: “Unresolvable” or asking the right question , 1994 .

[10]  The Social Scientist's Bestiary: A Guide to Fabled Threats to, and Defences of, Naturalistic Social Science , 1992 .

[11]  Gavriel Salomon,et al.  Transcending the Qualitative-Quantitative Debate: The Analytic and Systemic Approaches to Educational Research , 1991 .

[12]  Alasdair McLean,et al.  The Potential of Live Teacher Supported Distance Learning: A Case-Study of the Use of Audio Conferencing at the University of Aberdeen , 1996 .

[13]  R. Clark.  Media will never influence learning , 1994 .

[14]  David M. Moore,et al.  Media Research: a graduate student's primer , 1986 .

[15]  B. Thompson.  Editorial Policies Regarding Statistical Significance Testing : Three Suggested Reforms , 2012 .

[16]  David H. Jonassen,et al.  Learningwith media: Restructuring the debate , 1994 .

[17]  Margaret Landstrom.  Evaluating Open and Distance Learning , 1989 .

[18]  Bruce Thompson,et al.  The Use of Statistical Significance Tests in Research: Bootstrap and Other Alternatives , 1993 .

[19]  B. Thompson.  Research news and Comment: AERA Editorial Policies Regarding Statistical Significance Testing: Three Suggested Reforms , 1996 .

[20]  J. Levin,et al.  Reflections on Statistical and Substantive Significance, with a Slice of Replication. , 1997 .

[21]  Rand R. Wilcox,et al.  New monte carlo results on the robustness of the anova f, w and f statistics , 1986 .

[22]  R. Kozma.  Learning with Media , 1991 .

[23]  Carl J. Huberty,et al.  Statistical Practices of Educational Researchers: An Analysis of their ANOVA, MANOVA, and ANCOVA Analyses , 1998 .

[24]  Richard E. Clark,et al.  Media and method , 1994 .

[25]  R. Clark.  Reconsidering Research on Learning from Media , 1983 .

[26]  G. Keppel,et al.  Data analysis for research designs: Analysis of variance and multiple regression/correlation approaches. , 1990 .

[27]  Margaret Gorts Morabito.  Foundations of Distance Education. , 1997 .

[28]  E. Wagner.  Interactivity: From Agents to Outcomes. , 1997 .

[29]  Thomas C. Reeves,et al.  Pseudoscience in Computer-Based Instruction: The Case of Learner Control Research. , 1993 .

[30]  R. P. Carver.  The Case Against Statistical Significance Testing, Revisited , 1993 .

[32]  John R. Verduin,et al.  Distance Education: The Foundations of Effective Practice , 1991 .

[33]  Daniel H. Robinson,et al.  Research news and Comment: Reflections on Statistical and Substantive Significance, With a Slice of Replication , 1997 .

[34]  Sharon A. Shrock,et al.  The media influence debate: Read the fine print, but don't lose sight of the big picture , 1994 .

[35]  Robert A. Reiser.  Clark's invitation to the dance: An instructional Designer's response , 1994 .

[36]  Desmond Keegan.  Foundations of Distance Education. Third Edition. Routledge Studies in Distance Education. , 1996 .

[37]  P. Meehl.  Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. , 1978 .

[38]  Gayle Douglas.  MLIS Distance Education at the University of South Carolina: Report of a Case Study , 1996, J. Am. Soc. Inf. Sci..

[39]  J. Levin.  Statistical Significance Testing From Three Perspectives , 1993 .

[40]  Gene V. Glass,et al.  Research and Evaluation in Education and the Social Sciences , 1986 .

[41]  Robert D. Tennyson,et al.  The big wrench vs. integrated approaches: The great media debate , 1994 .

[42]  Deborah L. Whetzel,et al.  A real world comparison of the effectiveness of satellite training and classroom training , 1996 .

[43]  J Reiss,et al.  Enhancing the role public health nurses play in serving children with special health needs: an interactive videoconference on Public Law 99-457 Part H. , 1996, Public health nursing.

[44]  Robert B. Kozma,et al.  A reply: Media and methods , 1994 .

[45]  T. Cyrs,et al.  Teaching and learning at a distance : what it takes to effectively design, deliver, and evaluate programs , 1997 .

[46]  M. Orey.  A Philosophical Critique of Null Hypothesis Testing. , 1989 .

[47]  Bruce A. Thyer,et al.  Research Notes: Distance Learning in Social Work Education: A Preliminary Evaluation , 1997 .

[48]  R. P. Carver.  The Case Against Statistical Significance Testing , 1978 .

[49]  Michael Simonson,et al.  Proceedings of Selected Research and Development Presentations at the 1995 National Convention of the Association for Educational Communications and Technology Sponsored by the Research and Theory Division (17th, Anaheim, CA, 1995). , 1995 .

[50]  Teaching a Large Enrollment, Introductory Geography Course by Television , 1996 .

[51]  P. Meehl.  Theory-Testing in Psychology and Physics: A Methodological Paradox , 1967, Philosophy of Science.

[52]  R. Kozma.  Will media influence learning? Reframing the debate , 1994 .