Standards for the development and review of instrumental studies: considerations about test selection in psychological research

This paper discusses the criteria that should be considered when selecting psychological assessment tests in a research context. Traditionally attention has focused - and still does - on the stages that must govern any process of test construction/ adaptation. This work is guided by internationally accepted standards, whose scientific importance is agreed by the scientific community. However, beyond any construction/ adaptation process, the use of tests is a constant feature of psychological research, so it is of vital importance to select the tests adequately. For this reason, in this theoretical study we provide a summary of the criteria that should guide test construction/adaptation as well as some general guidelines to consider when selecting tests for psychological research. The information presented is organized into six sections, each of which corresponds to a different stage in the process of test creation: a) conceptual definition of the construct to assess; b) information about item construction and qualitative assessment; c) results of the statistical analysis of the items; d) empirical evidence of the internal structure of the test; e) results of the reliability estimation; and f) external evidence of score validity. The study ends with a reflection on the scope of the proposed guidelines and the importance of using clear criteria to select the tests used in research.

[1]  L. Wise Encouraging and Supporting Compliance with Standards for Educational Tests , 2006 .

[2]  A. Kazdin Methodological issues & strategies in clinical research , 1992 .

[3]  D. Rubio,et al.  Objectifying Content Validity: Conducting a Content Validity Study in Social Work Research , 2003 .

[4]  D. Watson,et al.  Constructing validity: Basic issues in objective scale development , 1995 .

[5]  Ross E. Traub,et al.  Reliability for the Social Sciences: Theory and Applications , 1994 .

[6]  Nancy E. Betz,et al.  Tests and assessment , 1985 .

[7]  J. M. Cortina,et al.  What Is Coefficient Alpha? An Examination of Theory and Applications , 1993 .

[8]  J. Alonso,et al.  Análisis factorial confirmatorio. Su utilidad en la validación de cuestionarios relacionados con la salud , 2004 .

[9]  Paula Elosua,et al.  Evaluación progresiva de la invarianza factorial entre las versiones original y adaptada de una escala de autoconcepto , 2005 .

[10]  F. Floyd,et al.  Factor analysis in the development and refinement of clinical assessment instruments. , 1995 .

[11]  M. Spaan Test and Item Specifications Development , 2006 .

[12]  Rafael Moreno,et al.  New Guidelines for Developing Multiple-Choice Items , 2006 .

[13]  J. Loevinger Objective Tests as Instruments of Psychological Theory , 1957 .

[14]  EVALUACIÓN DE LA UNIDIMENSIONALIDAD DE LOS ITEMS MEDIANTE ANÁLISIS FACTORIAL , 1996 .

[15]  J. Muñiz,et al.  La evaluación de las consecuencias del uso de los tests en la teoría de la validez , 2006 .

[16]  C. Cleeland,et al.  Content validity of self-report measurement instruments: an illustration from the development of the Brain Tumor Module of the M.D. Anderson Symptom Inventory. , 2005, Oncology nursing forum.

[17]  R. Hambleton Adaptación de tests para su uso en diferentes idiomas y culturas: fuentes de error, posibles soluciones y directrices prácticas , 1996 .

[18]  J. M. Fernández,et al.  Teoría clásica de los tests , 1992 .

[19]  K. Murphy,et al.  Psychological Testing: Principles and Applications , 1988 .

[20]  Juan Botella,et al.  Doing and reporting a meta-analysis , 2006 .

[21]  M. Ash,et al.  Roles of instruments in psychological research. , 2005, History of psychology.

[22]  R. Hambleton Guidelines for adapting educational and psychological tests: A progress report. , 1994 .

[23]  Thomas P. Hogan,et al.  An Empirical Study of Reporting Practices Concerning Measurement Validity , 2004 .

[24]  Steven M Downing,et al.  Validity threats: overcoming interference with proposed interpretations of assessment data , 2004, Medical education.

[25]  J. Muñiz,et al.  Esquema conceptual y procedimientos para analizar la validez de las consecuencias del uso de los test , 2007 .

[26]  H. G. Osburn Coefficient alpha and related internal consistency reliability coefficients. , 2000 .

[27]  James Jaccard,et al.  Arbitrary metrics in psychology. , 2006, The American psychologist.

[28]  Hugo Carretero-Dios,et al.  Normas para el desarrollo y revisión de estudios instrumentales , 2015 .

[29]  R. Hambleton,et al.  Advances in translating and adapting educational and psychological tests , 2003 .

[30]  Educational Evaluation Standards for Educational and Psychological Testing , 1999 .

[31]  Gregory T. Smith On construct validity: issues of method and measurement. , 2005, Psychological assessment.

[32]  A. Evers,et al.  Testing practices in European countries , 2001 .

[33]  Ignacio Montero,et al.  Sistema de clasificación del método en los informes de investigación en Psicología , 2005 .

[34]  Juana Gómez Benito,et al.  Esquema conceptual y procedimientos para analizar la validez de las consecuencias del uso de los test , 2007 .

[35]  Steven J. Osterlind,et al.  What Is Constructing Test Items , 1989 .

[36]  Robert L. Linn Following the Standards: Is it Time for Another Revision? , 2006 .

[37]  Developing and improving assessment instruments , 2004 .

[38]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[39]  S. Haynes,et al.  Content validity in psychological assessment: A functional approach to concepts and methods. , 1995 .

[40]  Dimensiones de la apreciación del humor , 2006 .

[41]  Paula Elosua Olidén Sobre la validez de los tests , 2003 .