The Combination of Grey System Theory and Receiver Operating Characteristic Method for Setting the Standard of Tests

How can you make a decision for students from their test scores and test information? The purpose of this study is to propose a standard setting method for test based on the combination of grey prediction models (GM) and Receiver Operating Characteristic (ROC) method. Using the previous measurement data, grey prediction models predict the ability scores of students during testing. ROC method uses this result as a reference to find out the optimal cut score based on students’ performance. The experiment result showed that the proposed method found out objective pass mark that can differentiate passing and failing students. This method is considered easy to apply and not time consuming, but the results obtained are reliable, it helps educators more convenient to make their decision in teaching process.

[1]  N. Perkins,et al.  The inconsistency of "optimal" cutpoints obtained using two criteria based on the receiver operating characteristic curve. , 2006, American journal of epidemiology.

[2]  Yen-Hsien Lee,et al.  Accurately Forecasting Model for the Stochastic Volatility Data in Tourism Demand , 2011 .

[3]  Fang Yang,et al.  APPLICATION OF GREY SYSTEM THEORY TO FORECAST THE GROWTH OF LARCH , 2009 .

[4]  K. Zou,et al.  Receiver-Operating Characteristic Analysis for Evaluating Diagnostic Tests and Predictive Models , 2007, Circulation.

[5]  Rajeev Kumar,et al.  Receiver operating characteristic (ROC) curve for medical researchers , 2011, Indian pediatrics.

[6]  GuoDong Li,et al.  New Proposal and Accuracy Evaluation of Grey Prediction GM , 2007, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[7]  Samuel A. Livingston,et al.  Passing Scores: A Manual for Setting Standards of Performance on Educational and Occupational Tests. , 1982 .

[8]  Memmedaga Memmedli,et al.  An application of fuzzy time series to improve ISE forecasting , 2010 .

[9]  Isaac I. Bejar.  Standard Setting: What Is It? Why Is It Important? , 2008 .

[10]  M. Shu,et al.  Forecasting Inbound Tourism Demand in Thailand with Grey Model , 2013 .

[11]  L. Shepard.  Standard Setting Issues and Methods , 1980 .

[12]  N. Obuchowski.  Receiver operating characteristic curves and their use in radiology. , 2003, Radiology.

[13]  Geraldine Lear,et al.  Standard setting. , 2003, Nursing standard (Royal College of Nursing (Great Britain) : 1987).

[14]  C. V. D. van der Vleuten,et al.  A standard setting method with the best performing students as point of reference: Practical and affordable , 2010, Medical teacher.

[15]  Dengxu He,et al.  Optimized Grey Model Based on Cuckoo Search Algorithm and Its Prediction Application , 2014 .

[16]  C A Roe,et al.  Statistical Comparison of Two ROC-curve Estimates Obtained from Partially-paired Datasets , 1998, Medical decision making : an international journal of the Society for Medical Decision Making.

[17]  J. Deng,et al.  Introduction to Grey system theory , 1989 .

[18]  M. Tavakol,et al.  Standard Setting: the application of the Receiver Operating Characteristic method , 2012 .

[19]  Chen Bing,et al.  An Optimized Unbiased GM (1, 1) Power Model for Forecasting MRO Spare Parts Inventory , 2012 .

[20]  Martin W. Bunder,et al.  The inconsistency of , 1976, Journal of Symbolic Logic.

[21]  A. Barman.  Standard setting in student assessment: is a defensible method yet to come? , 2008, Annals of the Academy of Medicine, Singapore.

[22]  Tom Fawcett,et al.  ROC Graphs: Notes and Practical Considerations for Researchers , 2007 .

[23]  K. Hajian‐Tilaki,et al.  Receiver Operating Characteristic (ROC) Curve Analysis for Medical Diagnostic Test Evaluation. , 2013, Caspian journal of internal medicine.

[24]  W. Marsden.  I and J , 2012 .

[25]  J A Colliver,et al.  Using a receiver operating characteristic (ROC) analysis to set passing standards for a standardized‐patient examination of clinical competence , 1994, Academic medicine : journal of the Association of American Medical Colleges.

[26]  W. Youden,et al.  Index for rating diagnostic tests , 1950, Cancer.

[27]  Ara Tekian,et al.  RESEARCH METHODOLOGY: Procedures for Establishing Defensible Absolute Passing Scores on Performance Examinations in Health Professions Education , 2006, Teaching and learning in medicine.