It takes time: A remarkable example of delayed recognition

The way in which scientific publications are picked up by the research community can vary. Some articles become instantly cited, whereas others go unnoticed for some time before they are discovered or rediscovered. Papers with delayed recognition have also been labeled “sleeping beauties.” I briefly discuss an extreme case of a sleeping beauty. Peirce's short note in Science in 1884 shows a remarkable increase in citations since around 2000. The note received less than 1 citation per year in the decades prior to 2000, 3.5 citations per year in the 2000s, and 10.4 in the 2010s. This increase was seen in several domains, most notably meteorology, medical prediction research, and economics. The paper outlines formulas to evaluate a binary prediction system for a binary outcome. This citation increase in various domains may be attributed to a widespread, growing research focus on mathematical prediction systems and the evaluation thereof. Several recently suggested evaluation measures essentially reinvented or extended Peirce's 120-year-old ideas. © 2012 Wiley Periodicals, Inc.

[1]  S. Rabe-Hesketh,et al.  Latent Variable Modelling: A Survey * , 2007 .

[2]  ANTHONY F. J. VAN RAAN,et al.  Sleeping Beauties in science , 2004, Scientometrics.

[3]  Joseph Brent,et al.  Charles Sanders Peirce: A Life , 1993 .

[4]  Oscar Jorda,et al.  Discussion of "Anchoring Countercyclical Capital Buffers: The Role of Credit , 2011 .

[5]  Thed N. van Leeuwen,et al.  Is scientific literature subject to a ‘Sell-By-Date’? A general methodology to analyze the ‘durability’ of scientific documents , 2010 .

[6]  C S Peirce,et al.  The numerical measure of the success of predictions. , 1884, Science.

[7]  Nancy R Cook,et al.  Using relative utility curves to evaluate risk prediction , 2009, Journal of the Royal Statistical Society. Series A,.

[8]  M. White,et al.  Measuring and comparing the accuracy of species distribution models with presence–absence data , 2011 .

[9]  Stephen Cole,et al.  Professional Standing and the Reception of Scientific Discoveries , 1970, American Journal of Sociology.

[10]  Ahmed Albatineh,et al.  On Similarity Indices and Correction for Chance Agreement , 2006, J. Classif..

[11]  T. Haiden,et al.  A new equitable score suitable for verifying precipitation in numerical weather prediction , 2010 .

[12]  Hester F. Lingsma,et al.  Assessing the incremental value of diagnostic and prognostic markers: a review and illustration , 2012, European journal of clinical investigation.

[13]  K. Henkens,et al.  Signals in Science - on the Importance of Signaling in Gaining Attention in Science , 2004 .

[14]  Juan Miguel Campanario.  Have referees rejected some of the most-cited articles of all times? , 1996 .

[15]  Leo Egghe,et al.  Thoughts on uncitedness: Nobel laureates and Fields medalists as case studies , 2011, J. Assoc. Inf. Sci. Technol..

[16]  Norio Ohba,et al.  Sleeping beauties in ophthalmology , 2012, Scientometrics.

[17]  Eric Loken,et al.  Peirce's 19th Century Mixture Model Approach to Rater Agreement , 2006 .

[18]  Thed N. van Leeuwen,et al.  Is scientific literature subject to a 'Sell-By-Date'? A general methodology to analyze the 'durability' of scientific documents , 2009, J. Assoc. Inf. Sci. Technol..

[19]  Jiang Li,et al.  The phenomenon of all-elements-sleeping-beauties in scientific literature , 2012, Scientometrics.

[20]  Quentin L. Burrell,et al.  Are “Sleeping Beauties” to be expected? , 2005, Scientometrics.

[21]  W. Youden,et al.  Index for rating diagnostic tests , 1950, Cancer.

[22]  Ewout W Steyerberg,et al.  Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers , 2011, Statistics in medicine.

[23]  Wolfgang Glänzel,et al.  Better late than never? On the chance to become highly cited only beyond the standard bibliometric time horizon , 2004, Scientometrics.

[24]  Agostino Manzato,et al.  A Note On the Maximum Peirce Skill Score , 2007 .

[25]  B. Kramer,et al.  Peirce, Youden, and Receiver Operating Characteristic Curves , 2007 .

[26]  E. Elkin,et al.  Decision Curve Analysis: A Novel Method for Evaluating Prediction Models , 2006, Medical decision making : an international journal of the Society for Medical Decision Making.