Network Signatures of Success: Emulating Expert and Crowd Assessment in Science, Art, and Technology

The success of scientific, artistic, and technological works is typically judged by human experts and the public. Recent empirical literature suggests that exceptionally creative works might have distinct patterns of citation. Given the recent availability of large citation and reference networks, we investigate how highly successful works differ from less successful ones in terms of a broad selection of centrality indices. Our experiments show that expert opinion is better emulated than popular judgment even with a single well-chosen index. Our findings further provide insights into otherwise implicit assumptions about indicators of success by evaluating the success of works based on the patterns of references that they receive.

[1]  Daniel M. Romero,et al.  The nearly universal link between the age of past knowledge and tomorrow’s breakthroughs in science and technology: The hotspot , 2017, Science Advances.

[2]  Vittorio Loreto,et al.  Significance and popularity in music production , 2017, Royal Society Open Science.

[3]  Albert-László Barabási,et al.  Quantifying Long-Term Scientific Impact , 2013, Science.

[4]  Matthew J. Salganik,et al.  Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market , 2006, Science.

[5]  ThelwallMike,et al.  When are readership counts as useful as citation counts? Scopus versus Mendeley for LIS journals , 2016 .

[6]  D J PRICE,et al.  NETWORKS OF SCIENTIFIC PAPERS. , 1965, Science.

[7]  Andreas Spitz,et al.  Measuring Long-Term Impact Based on Network Centrality: Unraveling Cinematic Citations , 2014, PloS one.

[8]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[9]  Benjamin F. Jones,et al.  Atypical Combinations and Scientific Impact , 2013, Science.

[10]  M. Weitzman,et al.  Recombinant Growth , 2009 .

[11]  D. Watts Everything Is Obvious: *Once You Know the Answer , 2011 .

[12]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[13]  Enys Mones,et al.  Universal hierarchical behavior of citation networks , 2014, ArXiv.

[14]  Mike Thelwall,et al.  When are readership counts as useful as citation counts? Scopus versus Mendeley for LIS journals , 2016, J. Assoc. Inf. Sci. Technol..

[15]  Sergei Maslov,et al.  Finding scientific gems with Google's PageRank algorithm , 2006, J. Informetrics.

[16]  Brian A. Nosek,et al.  Using prediction markets to estimate the reproducibility of scientific research , 2015, Proceedings of the National Academy of Sciences.

[17]  A. D. Jackson,et al.  Measures for measures , 2006, Nature.

[18]  Paul C. Tetlock,et al.  Using Information Markets to Improve Public Decision Making , 2005 .

[19]  E. Garfield,et al.  Citation indexes for science. , 1956, Science.

[20]  Simon M. Reader,et al.  Animal and human innovation: novel problems and novel solutions , 2016, Philosophical Transactions of the Royal Society B: Biological Sciences.

[21]  R. E. Burton,et al.  The “half‐life” of some scientific and technical literatures , 1960 .

[22]  Luís A Nunes Amaral,et al.  Cross-evaluation of metrics to estimate the significance of creative works , 2015, Proceedings of the National Academy of Sciences.

[23]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[24]  Melissa A. Schilling A "Small-World" Network Model of Cognitive Insight , 2005 .

[25]  Don A. Moore,et al.  Confidence Calibration in a Multiyear Geopolitical Forecasting Competition , 2017, Manag. Sci..

[26]  E GARFIELD,et al.  Citation indexes for science; a new dimension in documentation through association of ideas. , 2006, Science.

[27]  E. Todeva Networks , 2007 .

[28]  S. Rosen The Economics of Superstars , 1981 .

[29]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[30]  E. Garfield The history and meaning of the journal impact factor. , 2006, JAMA.

[31]  Gilles Louppe,et al.  Understanding variable importances in forests of randomized trees , 2013, NIPS.

[32]  Filippo Radicchi,et al.  Correlations between user voting data, budget, and box office for films in the internet movie database , 2013, J. Assoc. Inf. Sci. Technol..

[33]  E. Bonabeau Decisions 2.0: the power of collective intelligence , 2009 .

[34]  Anita Elberse,et al.  Blockbusters: Hit-making, Risk-taking, and the Big Business of Entertainment , 2001 .