Meaningful metrics for observatory publication statistics

Observatories have wrestled for decades with the questions how to measure their importance to the astronomical community, what their scientific impact is, and how their performance in that respect compares to that of other observatories. There is a general sense that the answer is to be found in the publication record - specifically, in the refereed journal articles. However, simple parameters (such as the number of papers) are not helpful, because in isolation (applied to a single observatory) they are meaningless, while in comparison between observatories they are subject to external influences that all but invalidate the comparisons. We were fortunate in having the Chandra X-ray Observatory's bibliographic database with its rich variety of metadata available as a resource for experimenting with more sophisticated metrics. Out of this project we propose a modest set that contains meaningful information when viewed in the isolation of a single observatory as well as in comparison with other observatories. Even so, we urge users not to draw conclusions on the basis of the face value of the comparisons, but only after a serious analysis of potential causes for any differences or similarities. We have designed our metrics to provide useful information in three main areas of interest: speed of publication; fraction of observing time published; and archival usage. The basic measured parameters are the percentage of available observing time published as a function of the data's age, at a few specific age values; the median time it takes to publish observations; and similar parameters for multiple publications of the same observations. Citation of results is a fourth category, but it does not lend itself well to comparisons and defies the search for definite statements.