Statistical analysis of ordinal user opinion scores

Data-sets derived from subjective experiments are often exploited to construct objective quality models using parametric statistics such as MOS and multiple regression. In this paper, using data type and normality tests, we verify that non-normally distributed user opinion scores in nominal or ordinal responses should not be analysed using parametric statistics. The paper introduces a number of non-parametric statistics for valid model building and parameter estimation based on user opinion scores. A set of modelling results are also presented to demonstrate the effectiveness of non-parametric statistics.

[1]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[2]  A. O'connell.  Logistic Regression Models for Ordinal Response Variables , 2005 .

[3]  J. S. Long,et al.  Confidence Intervals for Predicted Outcomes in Regression Models for Categorical Outcomes , 2005 .

[4]  P. McCullagh.  Regression Models for Ordinal Data , 1980 .

[5]  Lucjan Janowski,et al.  Modeling subjective tests of quality of experience with a Generalized Linear Model , 2009, 2009 International Workshop on Quality of Multimedia Experience.

[6]  A. Agresti.  Categorical data analysis , 1993 .

[7]  J. Guzmán.  Regression Models for Categorical Dependent Variables Using Stata , 2013 .

[8]  J. Chimka.  Categorical Data Analysis, Second Edition , 2003 .

[9]  R. Fisher,et al.  On the Mathematical Foundations of Theoretical Statistics , 1922 .

[10]  S. Siegel,et al.  Nonparametric Statistics for the Behavioral Sciences , 2022, The SAGE Encyclopedia of Research Design.

[11]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[12]  Sugato Chakravarty,et al.  Methodology for the subjective assessment of the quality of television pictures , 1995 .

[13]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[14]  S Lemeshow,et al.  The importance of assessing the fit of logistic regression models: a case study. , 1991, American journal of public health.

[15]  L. Toothaker.  Book Review : Nonparametric Statistics for the Behavioral Sciences (Second Edition) , 1989 .

[16]  Jusak Jusak,et al.  A Pilot Study to Assess Quality of Experience Based on Varying Network Parameters and User Behaviour , 2011 .

[17]  Mu Mu,et al.  Discrete quality assessment in IPTV content distribution networks , 2011, Signal Process. Image Commun..

[18]  ITU-T Rec. P.910 (04/2008) Subjective video quality assessment methods for multimedia applications , 2009 .

[19]  J. S. Long,et al.  Regression models for categorical dependent variables using Stata, 2nd Edition , 2005 .

[20]  M. S. Bartlett,et al.  Statistical methods and scientific inference. , 1957 .