Comparisons of Improvement-Over-Chance Effect Sizes for Two Groups Under Variance Heterogeneity and Prior Probabilities

The authors examined the distributional properties of 3 improvement-over-chance, I, effect sizes each derived from linear and quadratic predictive discriminant analysis and from logistic regression analysis for the 2-group univariate classification. These 3 classification methods (3 levels) were studied under varying levels of data conditions, including population separation (3 levels), variance pattern (3 levels), total sample size (3 levels), and prior probabilities (5 levels). The results indicated that the decision of which effect size to choose is primarily determined by the variance pattern and prior probabilities. Some of the I indices performed well for some small sample cases and quadratic predictive discriminant analysis I tended to work well with extreme variance heterogeneity and differing prior probabilities.

[1]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[2]  William D. Schafer,et al.  measurement and evaluation in counseling and Development , 2013 .

[3]  Rink Hoekstra,et al.  Are Assumptions of Well-Known Statistical Techniques Checked, and Why (Not)? , 2012, Front. Psychology.

[4]  L. Velasco,et al.  The role of psychological variables in explaining depression in older people with chronic pain , 2008, Aging & mental health.

[5]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[6]  B. Thompson,et al.  Extending Improvement-Over-Chance I-Index Effect Size Simulation Studies to Cover Some Small-Sample Cases , 2007 .

[7]  Anil K. Ghosh,et al.  Applied MANOVA and Discriminant Analysis (2nd ed.). Carl J. Huberty and Stephen Olejnik , 2007 .

[8]  R. Henson.  Effect-Size Measures and Meta-Analytic Thinking in Counseling Psychology Research , 2006 .

[9]  Pamela A. Moss,et al.  Standards for Reporting on Empirical Social Science Research in AERA Publications American Educational Research Association , 2006 .

[10]  Carl J. Huberty,et al.  Applied MANOVA and discriminant analysis , 2006 .

[11]  W. Holmes Finch,et al.  Misclassification Rates for Four Methods of Group Classification , 2006 .

[12]  B. Schneider,et al.  Standards for Reporting on Empirical Social Science Research in AERA Publications American Educational Research Association , 2006 .

[13]  Roger E. Kirk,et al.  Effect Size Measures , 2005 .

[14]  Anthony J. Onwuegbuzie,et al.  Editorial: Evidence-Based Guidelines for Publishing Articles in Research in the Schools and Beyond , 2005 .

[15]  C. J. Huberty,et al.  A History of Effect Size Indices , 2002 .

[16]  B. Thompson.  What Future Quantitative Social Science Research Could Look Like: Confidence Intervals for Effect Sizes , 2002 .

[17]  C. J. Huberty,et al.  The Efficacy of two Improvement-Over-Chance Effect Sizes for Two-Group Univariate Comparisons under Variance Heterogeneity and Nonnormality , 2001 .

[18]  Dae-Yeop Hwang.  Issues in Predictive Discriminant Analysis: Using and Interpreting the Leave-One-Out Jackknife Method and the Improvement-Over-Change "I" Index Effect Size. , 2001 .

[19]  B. Thompson.  Significance, Effect Sizes, Stepwise Methods, and Other Issues: Strong Arguments Move the Field , 2001 .

[20]  Bruce Thompson,et al.  Statistical Techniques Employed in AERJ and JCP Articles from 1988 to 1997: A Methodological Review , 2001 .

[21]  Raymond Hubbard,et al.  The Historical Growth of Statistical Significance Testing in Psychology--and Its Future Prospects. , 2000 .

[22]  Raymond Hubbard,et al.  Statistical Significance with Comments by Editors of Marketing Journals , 2000 .

[23]  Carl J. Huberty,et al.  Group Overlap as a Basis for Effect Size , 2000 .

[24]  Alvaro J. Arce Ferrer,et al.  Comparing the Classification Accuracy among Nonparametric, Parametric Discriminant Analysis and Logistic Regression Methods. , 1999 .

[25]  Xitao Fan,et al.  Comparing Linear Discriminant Function with Logistic Regression for the Two-Group Classification Problem. , 1999 .

[26]  Carl J. Huberty,et al.  Statistical Practices of Educational Researchers: An Analysis of their ANOVA, MANOVA, and ANCOVA Analyses , 1998 .

[27]  C. Heckler.  Applied Discriminant Analysis , 1995 .

[28]  Assuming Equal vs. Unequal Prior Probabilities of Group Membership in Discriminant Analysis: Effect on Predictive Accuracy. , 1995 .

[29]  Patrick Dattalo,et al.  A Comparison of Discriminant Analysis and Logistic Regression , 1995 .

[30]  R. Rosenthal.  Parametric measures of effect size. , 1994 .

[31]  Patricia Snyder,et al.  Evaluating Results Using Corrected and Uncorrected Effect Size Estimates , 1993 .

[32]  Bruce Thompson,et al.  The Use of Statistical Significance Tests in Research: Bootstrap and Other Alternatives , 1993 .

[33]  Elazar J. Pedhazur,et al.  Measurement, Design, and Analysis: An Integrated Approach , 1994 .

[34]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[35]  R. L. Greene,et al.  Use of the MMPI to identify malingering and exaggeration of psychiatric symptomatology in male prison inmates. , 1988, Journal of consulting and clinical psychology.

[36]  Rand R. Wilcox,et al.  New designs in analysis of variance. , 1987 .

[37]  D. Marsal.  An Introduction to Discriminant Analysis , 1987 .

[38]  Graham K. Rand,et al.  Quantitative Applications in the Social Sciences , 1983 .

[39]  Carl J. Huberty,et al.  Two-Group Comparisons and Univariate Classification , 1983 .

[40]  S. J. Press,et al.  Choosing between Logistic Regression and Discriminant Analysis , 1978 .

[41]  G. Glass.  Primary, Secondary, and Meta-Analysis of Research1 , 1976 .

[42]  R. Carroll,et al.  Sampling Characteristics of Kelley's ε and Hays' ω , 1975 .

[43]  N. L. Johnson,et al.  Multivariate Analysis , 1958, Nature.