Experimental Computer Science

This paper will focus on experimental computer science, why it is of current importance, its links to other disciplines, and some examples of the work being done at IBM’s Research Division and elsewhere. It will also touch on the future of computer science and how private industry can best support this emerging discipline. What makes experimental computer science distinct is the positing of hypotheses and the rigorous testing of these hypotheses under controlled conditions. Its goal is to acquire new knowledge about computers and computing. The questions posed and testing methods vary widely. But, in each case there must be an explicit underlying conceptual model that is tested by an experimental device with features approximating the model being questioned. Techniques like simulation and prototyping are essential elements of the experimental apparatus. As in the empirical sciences much effort must be devoted to developing “better microscopes” to capture, in greater detail, experimental results. Additionally, the data must be carefully interpreted within the context of the model. In the future, experimental computer science will increase in importance. What may have been a luxury for a small number of institutions will become a necessity for many if this science is to advance beyond the ad hoc methods of the past. I hope to see a strong collaboration between industry and academia. From this relationship a sharing of expertise that will maintain the vitality of computer research in both sectors will emerge. One example of experimental computer science is the work on comparing microprocessor architectures at the University of Massachusetts,* which is similar to the comparative analysis of architectures for military applications at Carnegie-Mellon University’ and work done at Stanford Univer~ity.~.~ It was hypothesized that architectures could be meaningfully distinguished by their efficiency independent of the technology of their embodiments. However, in the past, debating the merits of different architectures often took on the flavor of a religious schism. An experiment was designed in which a set of equivalent tasks were programmed on implementations of the VAX-11/780, 2-8000, HP-300, LSI-11, Motorola 6801, and Intel 8080 architectures. Restrictions on the style of coding such as that all kernels except the input/output (I/O) had to be reentrant were made to focus on the basic aspects of the instruction set. Five measures of efficiency were used: the number of bytes of