New approaches to using scientific data - statistics, data mining and related technologies in research and research training

This paper surveys technological changes that affect the collection, organization, analysis and presentation of data. It considers changes or improvements that ought to influence the research process and direct the use of technology. It explores implications for graduate research training. The insights of Evidence-Based Medicine are widely relevant across many different research areas. Its insights provide a helpful context within which to discuss the use of technological change to improve the research process. Systematic data-based overview has to date received inadequate attention, both in research and in research training. Sharing of research data once results are published would both assist systematic overview and allow further scrutiny where published analyses seem deficient. Deficiencies in data collection and published data analysis are surprisingly common. Technologies that offer new perspectives on data collection and analysis include data warehousing, data mining, new approaches to data visualization and a variety of computing technologies that are in the tradition of knowledge engineering and machine learning. There is a large overlap of interest with statistics. Statistics is itself changing dramatically as a result of the interplay between theoretical development and the power of new computational tools. I comment briefly on other developing mathematical science application areas – notably molecular biology. The internet offers new possibilities for cooperation across institutional boundaries, for exchange of information between researchers, and for dissemination of research results. Research training ought to equip students both to use their research skills in areas different from those in which they have been immediately trained, and to respond to the challenge of steadily more demanding standards. There should be an increased emphasis on training to work cooperatively. “At bottom my critique is pretty simple-minded: Nobody pays much attention to the assumptions, and technology tends to overwhelm common sense.” [Freedman 1987.] “I personally look forward to the proper balance that will emerge from the mixing of computational algorithm-oriented approaches characterizing the database and computer science communities with the powerful mathematical theories and methods for estimation developed in statistics.” [Fayyad 1998.] “Statistics has been the most successful information science. Those who ignore statistics are condemned to re-invent it.” [Efron, quoted in Friedman 1997.] “Some members of the profession are trying hard to make changes, by teaching courses in which substantive questions come first and technique is introduced to find answers. Of course, all too often, technique comes first; data come in as purely decorative illustrations – a practice not confined to statistics departments.” [Freedman 1991.]