Evaluation of the performance of an information‐retrieval system by modified mooers plan

Information retrieval for law research has been carried out by the Health Law Center of the University of Pittsburgh. In this case, the document collection consists of all Pennsylvania Statutes. There is no indexing; the full text of the laws is scanned. Utilizing proposals by Mooers, randomly selected statute sections are the bases of retrieval requests. In response to such requests, statute sections are retrieved, but only the “hidden” statutes are counted in the evaluation of the system efficiency. Two systems are compared: retrieval by electronic computer vs. retrieval by manual search, i.e., by lawyers using the index parts of statute collections. Mooers' ratios are computed and interpreted as point estimates of corresponding retrieval probabilities. Interval estimates and tests of significance for the differences in computed ratios are also carried out, with some consideration for the power function of the test. Cost limitations prescribed very small sample sizes.