Intelligent interpretation of validation data

Abstract This paper describes the characteristics of an expert system that will make an intelligent interpretation of the quantitative data obtained following the usual validation process for intelligent systems. This expert system converts numerical data into information that will indicate whether the performance of the validated intelligent system is comparable to the performance of human experts from the domain taken as reference. The process is carried out in two parts: an algorithmic part which identifies the basic characteristics of the quantitative data; a heuristic part which analyses these characteristics so as to be able to draw conclusions as to the results of the system.

[1]  G. W. Williams,et al.  Comparing the joint agreement of several raters with another rater. , 1976, Biometrics.

[2]  Leonard Adelman,et al.  Handbook for Evaluating Knowledge-Based Systems , 1997, Springer US.

[3]  Eduardo Mosqueira-Rey,et al.  Validation of intelligent systems: a critical study and a tool , 2000 .

[4]  C. H. Chen,et al.  Handbook of Pattern Recognition and Computer Vision , 1993 .

[5]  G. De Soete,et al.  Clustering and Classification , 2019, Data-Driven Science and Engineering.

[6]  Pedro Meseguer,et al.  Knowledge Based Systems Validation: A State of the Art , 1990, AI Commun..

[7]  G. W. Milligan,et al.  An examination of procedures for determining the number of clusters in a data set , 1985 .

[8]  Amparo Alonso-Betanzos,et al.  Applying statistical, uncertainty-based and connectionist approaches to the prediction of fetal outcome: a comparative study , 1999, Artif. Intell. Medicine.

[9]  Osman Balci,et al.  Validating Expert System Performance , 1987, IEEE Expert.

[10]  Robert Plant,et al.  A survey of tools for the validation and verification of knowledge-based systems: 1985-1995 , 1997, Decis. Support Syst..

[11]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[12]  Wei-Tek Tsai,et al.  Verification and Validation of Knowledge-Based Systems , 1999, IEEE Trans. Knowl. Data Eng..

[13]  B. Chandrasekaran.  On Evaluating Artificial Intelligence Systems for Medical Diagnosis , 1983, AI Mag..

[14]  V. Moret-Bonillo,et al.  The PATRICIA project: a semantic-based methodology for intelligent monitoring in the ICU , 1993, IEEE Engineering in Medicine and Biology Magazine.

[15]  Natalia Juristo Juzgado,et al.  Common framework for the evaluation process of KBS and conventional software , 1998, Knowl. Based Syst..

[16]  Balakrishnan Chandrasekaran,et al.  On evaluating ai systems for medical diagnosis , 1989 .

[17]  Richard C. Dubes,et al.  Cluster Analysis and Related Issues , 1993, Handbook of Pattern Recognition and Computer Vision.

[18]  M. Kendall,et al.  Rank Correlation Methods , 1949 .

[19]  P. Groenen,et al.  Modern multidimensional scaling , 1996 .

[20]  Eduardo Mosqueira Rey.  SHIVA : un sistema heurístico e integrado para la validación de sistemas inteligentes , 1998 .

[21]  Uma G. Gupta.  Validation and verification of knowledge-based systems: A survey , 1993, Applied Intelligence.

[22]  Amparo Alonso-Betanzos,et al.  Information analysis and validation of intelligent monitoring systems in intensive care units , 1997, IEEE Transactions on Information Technology in Biomedicine.

[23]  Elena Pereira,et al.  Técnicas de inteligencia artifical e ingeniería del software para un sistema inteligente de monitorización de apneas en sueño , 2000 .

[24]  R. Sokal,et al.  THE COMPARISON OF DENDROGRAMS BY OBJECTIVE METHODS , 1962 .