Identifying and Evaluating the Universe of Outlets for Information Systems Research: Ranking the Journals

ABSTRACT We use a survey instrument to identify the universe of journal publication outlets for information systems (IS) research, to identify the IS journals, and to observe the value of the outlets to IS researchers. In an online survey we asked IS researchers to rate the value of IS publication outlets and to categorize them into IS journals, allied discipline journals, and professional and managerial magazines and journals. 1129 validated and non-duplicate respondents rated 326 journals, which we present in three rank-ordered lists, one for each of IS journals, allied discipline journals, and professional and managerial magazines and journals. In addition, we graphically present trends in the ranking of five selected journals from 1991 through 2003. This is the first attempt to identify the universe of IS journal publishing outlets, the first to rank the value of IS journals separately, and the first attempt to rank the value of IS publication outlets without pre-selecting the set of journals to be ranked. INTRODUCTION Motivation At the time of the first ICIS conference, more than twenty years ago, information systems was an ill-defined research focus, with little disciplinary recognition and with few credible research publication outlets of its own. It included researchers from a variety of disciplines, such as strategic management, operations research, accounting, computer science, and engineering, who saw the need to work on problems related to computer information systems and organizations. Quite naturally, they published the results of their research in the journals of a variety of disciplines, not only because there were few credible IS journals, but also because the researchers themselves had ties to other disciplines. Since then, IS has become an accepted discipline at most universities, albeit grudgingly at some, if for no other reason than because of the very substantial demand for teaching capacity in business schools for MIS major students and as part of the core curriculum for other business students. In the subsequent two decades researchers in IS developed a large number of new journals that specifically focus exclusively on IS research issues. At the same time, IS researchers have continued the tradition of publishing articles that address IS research questions in the journals of allied disciplines. As a result, IS research has been published in a wide variety of journal outlets, both within and outside the IS discipline. Over this period, there have been at least 15 published attempts to evaluate and rank journals in terms of their importance for IS research. Generally these attempts have been consistent with information system's origins as a multidisciplinary focus. In every case the articles have evaluated a mixed set of IS journals and journals from other disciplines. In addition, each of the prior studies has limited itself to a pre-selected set of journals. To date there has been no attempt, * To identify the universe of publication outlets in which IS research is published, * To identify all of the IS research journals, or * To measure the value of the outlets to IS researchers and their audiences. In consequence, researchers have made do with articles that identify and evaluate only a small part of the IS research publication capacity. The number of outlets identified in each article varied, but some of the articles identified and evaluated capacity for only a few dozen articles per year. For example, Shim, English, and Yoon (1991) identified eight journals, mostly from other disciplines, publishing approximately 100 MIS articles per year, as the most important IS research outlets. Other ranking articles have similarly rated a small number of research outlets. Since that time, however, the IS discipline has grown and matured substantially, such that today a small number of publication outlets can hardly serve the needs of 3000 or more IS researchers, who investigate problems in dozens of different research streams. …