The 3-sigma bump reported by Fermilab on Wednesday has garnered a lot of attention. Understandable, since it might be a precious sign of particle physics beyond the Standard Model — but it’s also just a 3-sigma bump, and usually those go away.
Via Matt Strassler and Lisa Randall, here’s a set of plots that helps indicate exactly how hard this game really is. The plots can all be found on this web page at CERN. In comments Matt and Peter note that they were made by Tommaso Tabarelli de Fatis, as explained on Tommaso Dorigo’s blog. Here is the original plot from the CDF paper:
We’re looking at the number of events that produce a W boson and two jets, as a function of the energy of the jets. The bottom plot is all the data, while at the top they’ve subtracted off most of the Standard Model background, leaving only the predicted red curve from WW/WZ events. You see the extra little bump around 150 GeV, that’s what’s getting everyone so excited. It’s unlikely that the data are a good fit to the prediction; the “KS (Kolmogorov-Smirnov) probability” is given as 5×10-5, which means that it’s not bloody likely.
But, just for giggles, let’s imagine that the energy of the jets wasn’t measured very accurately. Obviously the experimenter worked hard to get it right, and I would trust their judgment over my own any day, but you never know. Jets are complicated things with many particles in them, you can imagine being off by a bit.
So here is the same plot, just scaling the jet energy by two percent:
You see that makes a lot of the excitement go away. The KS probability is now 9×10-2, which essentially means … the bump has gone away. If we scale by 4%, it goes away almost entirely:
Now the data fit the Standard Model very well. If we keep cranking up the supposed error, we stop fitting again, because the data at very low energies begin to go astray:
There is an animated version of the plot which is fun to look at. Nobody is saying (I don’t think) that this is certainly what’s going on; it’s just an illustration of how difficult this game really is, and why people shouldn’t get too excited about three-sigma events. A little bit of excitement is good, and descending deep into cynicism is bad, but patience is really warranted. Let’s collect more data and see what happens.
Tommaso Dorigo has an informative blog posting up today about this, see
http://www.science20.com/quantum_diaries_survivor/blog/jet_energy_scale_explanation_cdf_signal-77886
He explains that the plots are work of Tommaso Tabarelli de Fatis.
The plots were made by Tommaso Tabarelli de Fatis, I first saw them on Tommaso Dorigo’s blog
Update: I see Peter Woit beat me to it
A little bit of excitement is good, and descending deep into cynicism is bad, but patience is really warranted. Let’s collect more data and see what happens.
Then why make such a big deal at this stage?
Has anyone ever thought about the possible negative effects from publicizing something so much to later say that it’s not true? If someone wants to hype scientific results up, it should be done after it’s been confirmed.
And, why does it happen a lot more often with particle physics?
Do the authors of the plot explain whether one is allowed to shift the energy by 4% arbitrarily?
Thanks Sean (I think you actually have four times the same plot though)
Peter and Matt, thanks for the find; I’ve updated the post.
T– look more carefully!
Quoted from the article:
“But, just for giggles, let’s imagine that the energy of the jets wasn’t measured very accurately. Obviously the experimenter worked hard to get it right, and I would trust their judgment over my own any day, but you never know. Jets are complicated things with many particles in them, you can imagine being off by a bit.”
Sean,
How accurate is the measurement of jet energy in a normal experiment? Does a difference of 4% in that measurement happen often enough to casually suggest it as an alternative explanation in this case?
If 4% errors in measurement are common, then would you agree that the “excitement” generated by the report of this data is more of a media exercise than it is an indication of some new physics?
I am very eager to hear what DZero comes back with, even if it confirms nothing more than a statistical fluke.
Sean,
If 3-sigma results in particle physics usually go away, then why has this latest result gotten so much attention? In my opinion, it seems that we are entering a period of time in which it is truth or consequences for many of our cherished theories, as no gravitational waves have been detected, no dark matter has been directly detected, and the Higgs boson remains unfound as well.
My personal theory is that the simplest explanation for this universe is that we are living in a simulation that makes up the rules on the fly and generates new regions of spacetime as we explore them. If there is a “Great Programmer” of this simulation I’m sure He is having quite a laugh at our attempts to make sense of His algorithms! 😉
@charro: I’m sure ‘hype’ happens in other fields too but that hep gets mainstream press while many other fields do not. In any event, I have no issue with someone putting up 3 sigma preliminary results or those which will require a more extensive search by other researchers. And certainly why shouldn’t experimentalists get some slack when theorists are always churning stuff out which is not five sigma and having it hyped?
Pingback: 8 April 2011 pm « blueollie
@ #7 (David)
Last I heard the uncertainty in the jet energy scale at CDF was about 3%, which seems enough to reconcile this difference but I would have expected the author to include this systematic before presenting this bump as 3-sigma. There are other ways you could accidentally find a bump too, if the Monte Carlo prediction of the background was normalized incorrectly (through no fault of the author) it might skew the background subtracted result. Basically there are a lot of things that *might* cause such a bump and the best way to test it will be more data.
Tellingly, CDF didn’t see this bump in a similar analysis with leptons implying that any new particle would have to be very lepton shy. D0 didn’t see anything either but I suspect they’re going over all their data again. The LHC will hopefully be ready to release some results on a similar analysis by the end of the year, a lot of people will be very disappointed if CDF pipped them to the post.
While we’re on the subject I saw a very good article written at a layman’s level over at io9:
http://io9.com/#!5789759/has-fermilab-really-discovered-an-entirely-new-subatomic-particle
Apologies for the hashbang
Your point is valid, but I hardly think it’s fair (beyond being illustrative) to apply a rote scaling to every point there. It means not only did they get the jet energy wrong by 4%, but *in every bin* of the bump.
Put differently, if we apply the same type of shift to the standard model prediction near 90 GEV, we can make it go away as well.
I am *not* saying that the blip is real, nor am I saying it’s energy miscalculations. I’m just pointing out that shifting *every* bin is a tad misleading….
Pingback: New physics on Tevatron? « Symmetry
Pingback: Nachrichten aus der extremen Physik kompakt « Skyweek Zwei Punkt Null
thats what happens when funding runns out on a big experiment
you do have a charming way of saying they screwed up the jet energy measurment ^_^
Although it’s probably going to vanish like dreams in the dawn, this result has generated interest precisely because it is the first indication of something going on beyond the standard model . It would represent fundamentally new physics.
And this is going to be as good as it gets as the Tevatron is now a memory.
Pingback: Anomaly at the Tevatron Might Be Something Real? | Cosmic Variance | Discover Magazine