Descriptive Data Analysis: A Concept between Confirmatory and Exploratory Data Analysis

Confirmatory Data Analysis (CDA) in randomized comparative (“controlled”) studies with many variables and/or time points of interest finds its limitations in the multiplicity of desired inferential statements which leads to unfeasibly small adjusted significance levels (“Bon-ferronization”) and, thereby, to unduly increased risks of not rejecting false hypotheses. In general, analytical models adequate for such complex data structures and suitable for practical use do not exist as yet. Exploratory Data Analysis (EDA), on the other hand, is usually intended to generate hypotheses and not to lead to final conclusions based on the results of the study. In this paper, it is proposed to fill the conceptual gap between CDA and EDA by “Descriptive Data Analysis” (“DDA”) which concept is mainly based on descriptive inferential statements. The results of a DDA in a controlled study are interpreted simultaneously on the basis of the investigator’s experience with respect to numerically relevant treatment effect differences and on “descriptive significances” as they appear in “near regular” patterns corresponding to the resulting relevant effect differences. A DDA may also contain confirmatory parts and/or tests on global hypotheses at a prechosen maximum risk α of erroneously rejecting true hypotheses. The paper is in parts expository and is addressed to investigators as well as statisticians.

[1]  C. Dunnett.  A Multiple Comparison Procedure for Comparing Several Treatments with a Control , 1955 .

[2]  J. Tukey.  The Future of Data Analysis , 1962 .

[3]  F. J. Anscombe.  Sequential Medical Trials , 1963 .

[4]  Rupert G. Miller.  Simultaneous Statistical Inference , 1966 .

[5]  M A Schneiderman,et al.  The role of hypothesis testing in clinical trials. Biometrics seminar. , 1966, Journal of chronic diseases.

[6]  Klaus Abt,et al.  On the identification of the significant independent variables in linear models , 1967 .

[7]  Jean D. Gibbons,et al.  P-values: Interpretation and Methodology , 1975 .

[8]  B. Rüger.  Das maximale signifikanzniveau des Tests: “LehneHo ab, wennk untern gegebenen tests zur ablehnung führen” , 1978 .

[9]  S. Holm.  A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[10]  Stellung der Explorativen Datenanalyse (EDA) im Rahmen der Statistik , 1980 .

[11]  P. Ihm.  Explorative und Konfirmatorische Datenanalyse - Gegensatz oder Ergänzung - , 1980 .

[12]  F. Mosteller,et al.  Reporting standards and research strategies for controlled trials , 1980 .

[13]  B. W. Brown,et al.  Statistical controversies in the design of clinical trials—Some personal views* , 1980 .

[14]  John W. Tukey,et al.  We Need Both Exploratory and Confirmatory , 1980 .

[15]  K Abt,et al.  Problems of repeated significance testing. , 1981, Controlled clinical trials.

[16]  The role of hypothesis testing in clinical trials. , 1981, Methods of information in medicine.

[17]  N. Victor.  Exploratory Data Analysis and Clinical Research , 1982, Methods of Information in Medicine.

[18]  G. Hommel.  Tests of the overall hypothesis for arbitrary dependence structures , 1983 .

[19]  K. Abt,et al.  Pharmaco-EEG and Psychometric study of the effect of single doses of temazepam and nitrazepam. , 1983, Neuropsychobiology.

[20]  K Abt,et al.  Significance testing of many variables. Problems and solutions. , 1983, Neuropsychobiology.

[21]  Risikofaktoren der Schwangerschaft , 1983 .

[22]  P. Armitage,et al.  Controversies and achievements in clinical trials. , 1984, Controlled clinical trials.

[23]  B. Schneider.  Bayesian Models for Clinical Studies , 1984, Methods of Information in Medicine.

[24]  Eugene S. Edgington,et al.  Randomization Tests , 2011, International Encyclopedia of Statistical Science.