Extraordinary claims: the 0.000029% solution

The five-standard-deviation threshold is an established standard for discovery claims in experimental particle physics; however, the criterion is an ad-hoc recipe with no solid foundations. In this report I discuss its origins and the issues it was designed to address, pointing out its shortcomings and the need for a more flexible approach to decide when a new observed effect should be taken seriously.

[1]  A. Heinson,et al.  Observation of Single Top Quark Production , 2011, 1101.1275.

[2]  G. Piacentino,et al.  OBSERVATION OF TOP QUARK PRODUCTION IN ANTI-P P COLLISIONS. , 1995 .

[3]  E. Gross,et al.  Trial factors for the look elsewhere effect in high energy physics , 2010, 1005.1891.

[4]  J. Cvach.  Evidence for a narrow anti-charmed baryon state , 2004 .

[5]  H. Jeffreys,et al.  The Theory of Probability , 1896 .

[6]  G De Lorenzo,et al.  Observation of electroweak single top-quark production. , 2009, Physical review letters.

[7]  D. Lindley.  A STATISTICAL PARADOX , 1957 .

[8]  A. Savoy-Navarro,et al.  Experimental observation of lepton pairs of invariant mass around 95 GeV/c2 at the Cern SPS collider , 1983 .

[9]  D. Whiteson,et al.  Invariant-mass distribution of jet pairs produced in association with a W boson in pp ̄ collisions at √s = 1.96TeV using the full CDF Run II data set , 2014, 1402.7044.

[10]  G. Unal,et al.  Evidence for top quark production in p p collisions at s = 1.8 TeV , 1994 .

[11]  G. Bauer,et al.  Experimental observation of isolated large transverse energy electrons with associated missing energy at $\sqrt s$ = 540 GeV , 1983 .

[12]  George K. Karagiannidis,et al.  An Improved Approximation for the Gaussian Q-Function , 2007, IEEE Communications Letters.

[13]  Cen,et al.  Evidence for top quark production in p-barp collisions at sqrt s =1.8 TeV. , 1994, Physical review letters.

[14]  V. Tioukov,et al.  Measurement of the neutrino velocity with the OPERA detector in the CNGS beam , 2011, 1109.4897.

[15]  Shapiro,et al.  Study of the heavy flavor content of jets produced in association with W bosons in p(p)over-bar collisions at root s=1.8 TeV , 2002 .

[16]  P. G. Verdini,et al.  Four-jet final state production in e+e− collisions at centre-of-mass energies of 130 and 136 GeV , 1996 .

[17]  Robert D. Cousins,et al.  The Jeffreys–Lindley paradox and discovery criteria in high energy physics , 2013, Synthese.

[18]  V. Tioukov,et al.  Measurement of the neutrino velocity with the OPERA detector in the CNGS beam using the 2012 dedicated data , 2012, 1212.1276.

[19]  A. Ribon,et al.  Observation of top quark production in $\bar{p}p$ collisions , 1995 .

[20]  Louis Lyons,et al.  Discovering the Significance of 5 sigma , 2013, 1310.1284.

[21]  J. Appel,et al.  Observation of High Mass Dilepton Pairs in Hadron Collisions at 400-GeV , 1976 .

[22]  R. Kephart,et al.  Observation of a Dimuon Resonance at 9.5 GeV in 400-GeV Proton-Nucleus Collisions , 1977 .

[23]  G De Lorenzo,et al.  Invariant Mass Distribution of Jet Pairs Produced in Association with a W boson in ppbar Collisions at sqrt(s) = 1.96 TeV , 2011, 1104.0699.