A little late, but I didn’t want to let slip this interesting discussion about the agonizing process of making experimental particle physics results ready for public consumption from Tommaso Dorigo and Gordon Watts. You’ll recall that we mentioned a couple of weeks ago the new results from Fermilab’s Tevatron on B-mixing, a measurement that puts interesting new constraints on the possibilities for physics beyond the Standard Model. The first announcement was from the D0 (“D-Zero”) experiment; as Collin pointed out in the comments, the CDF experiment followed with their own results soon thereafter.
From the CDF point of view, this is not how things are supposed to be; CDF is supposed to get there first, and D0 is supposed to confirm their results. Speaking from the CDF side, Tommaso talks about the process:
The publication process of CDF data analyses is baroque, bordering the grotesque. Once a group finalizes their result and presents it at internal meetings, the result has to be blessed. This involves three rounds of scrutiny, the full documentation of the analysis in internal notes, and often the fight with skeptics who like to sit at meetings and play “shoot the sitting duck” with the unfortunate colleague presenting the result. Usually, when an important result is on, the physicists who produced it are asked to perform additional checks of various kinds, and defend it with internal referees. When all of that is through, and not a day earlier, the result can be shown at Physics conferences.
After that happens, one would like to get the result on a Physics journal as soon as possible – to be cited!!! But just then, another much longer nightmare starts, when a process called “godparenting” begins and three knowledgeable colleagues (the godparents) are designated to scrutinize every detail of the work. Then a draft paper is produced, and in the following two weeks all the collaborators can play “shoot the duck” in written form, by sending criticism and demanding yet more checks. Then a second draft follows, and the process repeats…. In the end, usually six months pass between the blessing of a result at the physics meeting and the forwarding of a paper to a journal.
Gordon, from the D0 side, agrees with the general outline.
I don’t think it is that much different than what CDF has to go through — perhaps a bit more streamlined. We are all afraid that something wrong will make it out; hence all the layers of cross checking that go on. All of the collaboration is on the author list; this is the way the collaboration makes sure that the results that get out are correct. It can be a pain!
Read the whole things.
Of course there’s a lot more to the sociology of particle physics experiments than deciding when to release results. Interestingly, there are a lot of great books that take high-energy experiments and experimenters as their source material. Even novels — I recently read A Hole in Texas by Herman Wouk (best known for The Caine Mutiny and The Winds of War). It’s a short book set in the aftermath of the cancellation of the Superconducting Super Collider, imagining the hysteria if China managed to beat us to the Higgs boson. As a novel, I’ve read better; the romantic and political plots are somewhat perfunctory and not very believable. (And obviously written by a man; where else can you find no fewer than three attractive and accomplished women throwing themselves at a somewhat over-the-hill and not especially charming male physicist?) But the physics is surprisingly good; Wouk really put some effort into getting it right, including field trips to Fermilab and the SSC site.
And then you have your honest social-science explorations of the anthropology of the tribe of particle physicists. Beamtimes and Lifetimes, by anthropologist Sharon Traweek, treats HEP experimenters the same way we would treat an isolated tribe in the Amazon jungle, trying to figure out what makes them tick. (I’m still not sure.) But for my money, far and away the most insightful book is Nobel Dreams by Gary Taubes, the story of how Carlo Rubbia smashed the competition, not always using the most fair-minded tactics, to discover the W boson and win the Nobel Prize. Oh yes, and how he then failed to win another Nobel for discovering supersymmetry, despite repeatedly suggesting that his UA1 experiment had found evidence for it. A fascinating read, one that makes you tremble at the ambition of Rubbia and his lieutenants, admire the superhuman dedication of the many physicists on the project, and thank your lucky stars that your own working hours are a bit more sensible.
Update: Tommaso and Gordon explain more about the physics of the result.
Another interesting book that gives a lot of details about the inner workings of Fermilab is:
The Evidence for the Top Quark
Objectivity and Bias in Collaborative Experimentation
by Kent W. Staley (Cambridge 2004).
The book also discusses “… the introduction inot the Standard Model of the fifth (“bottom”) and sixth (“top”) quarks by Makoto Kobayashi andToshihide Maskakawa in 1973. The book says in part:
“… Sakata and several other Nagoya physicists were committed to dialectical materialism … The Nagoya model … prompted Kobayashi and Maskawa toward … studies … that led to their introduction of an entirely new “generation” of matter … some Japanese physicists felt strongly that Western physicists, especially in the United States, systematically ignored their work … In exploring the literature … one encounters the unpleasant reality of physicsts accusing one another of various forms of cultural bias. Worse, I [Staley] have found myself unable to resolve the worry that these accusations raise … “.
Tony Smith
http://www.valdostamuseum.org/hamsmith/
Experimentalists should indeed have very thorough internal reviews before releasing major important results. Just look at the example of the ‘cold fusion’ fiasco to see why. It’s absolutely necessary, or science will get the reputation of Uri Geller and the ‘spoon benders’.
If you write a theoretical paper, the essence of what you did can be checked within the paper itself.
But where the experiment is not easily replicated or at least requires costly equipment, inaccessible to the masses, it depends on lab credibility. Jan Hendrik Schön is the perfect example of how a laboratory protects its reputation as soon as allegations of exaggeration emerge, by firing the alleged culprit: http://en.wikipedia.org/wiki/Jan_Hendrik_Sch%C3%B6n
It is always necessary to sacrifice individuals to spare the majority. Don’t expect comradeship in the lab.
Whoa!! Herman Wouk is still alive?
It’s interesting to compare this to the recent kerfuffle about “fraudulent” articles in the biosciences and the whole vioxx mess with NEJM. Whereas the labs in those cases really did no internal verification, the labs in your field are doing extreme amounts. I wonder if it has to do with funding source and proximity to industry?
Tony, Do you agree that the Japanese should be given credit for predicting the 3rd generation of quarks first, although they used perhaps less rigorous arguments?
Wigner wrote in his autobiography, The Recollections of Eugene P. Wigner, that when he came to America, he was depressed to find that everyone preferred gruelling calculations to the lively exchange of speculative ideas that occurred in Hungary. Gell-Mann sensibly decided to try publishing the quark paper in a European journal, not an American one. American physics is probably too “old British” (stiff upper lip) to be comfortably relaxed in resolving priority disputes. But Europe had problems with this back in the days of Newton vs. Leibniz.
No rare that outsiders long shots are not very appreciated. But it is the same in the theory side; I get occasional messages about (against) releasing early 🙂. Even if in this case (theory in general) it is only an individual or small group problem, it is considered a lack of quality; and this consideration overweights the want of early discussion implied in an early release.
I think it was not the same in the old times, the 1923 revolution, or the shelter island backenvelope calculations.
The SNO collaboration goes a step further still. It is collaboration policy not to show any result in public unless it has both gone through extensive internal review and has been submitted for publication to a refereed journal. In other words, there is no such thing as a “SNO preliminary result”. The attitude is that if it’s not ready to submit for publication, it’s not ready to show in public either. Obviously this slows down the publication process, but personally I think there’s a lot of be said for it as well.
This scenario also plays out in medical research. By the time the entire process is completed, when the report is released, not just in the journals, but as interpreted and discussed in the public media, some in the medical community have moved on to other research that discredits the recently reported. In some cases this time line can be up to three or four years between the completion of the experiments and the final approved release of the report to the public. Three months is a long time to wait for proper health care, three years is much too long. With physics, this doesn’t kill people, in medicine and pharmacology it does. Something has to happen, and it might be that the physicists engaged in trying to work this out will come up with a standard formulation that will help in the other sciences.
Science asks if I “… agree that the Japanese should be given credit for predicting the 3rd generation of quarks first …”.
Yes.
If I ran the Nobel foundation (no danger of that actually happening) I would this year give 3 physics prizes for the T-quark:
Kobayashi (for theory prediction);
Maskawa (for theory prediction); and
Fermilab (for experimental observation).
Kobayashi and Maskawa are individuals, so their prizes would be easy to award.
However, Fermilab is a laboratory and Nobel prizes go to individuals, so what to do? Here is a suggestion (similar to one I made back in 1999):
Since machines don’t qualify, the director of Fermilab, as of the year 1994, should get the Nobel prize.
However,
I think that the director should accept the award, not only for himself, but also as a representative of everybody who worked at Fermilab, CDF, D0 during the runs leading up to the 1994 announcement FERMILAB-PUB-94/097-E, and, further, as a representative of all the USA taxpayers who paid the bills to get the work done. If and when such an award were to be made by the Nobel committee, the Fermilab Public Affairs office could make replicas of the Nobel medallions, not only for workers at Fermilab, CDF, and D0 (free to them, paid out of the Nobel Prize funds) but also for each USA taxpayer who wants to buy one. For taxpayers, maybe the Nobel medallion could be in a plaque that says something like “to _____, a taxpayer who paid the bills and made the T-quark discovery possible”, with the taxpayer’s name engraved on the plaque.) If the price for taxpayers were to be about $20 over cost, and if a million taxpayers bought their own Nobel prize, then Fermilab could raise $20 million.
(I know that’s not much money nowadays, but, to paraphrase Sen. Dirksen, who got Fermilab located in Illinois in the first place, “$20 million here, $20 million there, sooner or later it adds up to real money”).
Tony Smith
http://www.valdostamuseum.org/hamsmith/
I agree that the large number of authors of these experimental papers see e.g. here slows things down.
Tony and Science: yes, Kobayashi and Maskawa are, by and large, given credit for predicting the existence of the third generation with their famous paper in 1974. I’ve discussed this with some of my elders and they said that at the time, the Kobayashi amd Maskawa paper did not get so much attention as it was considered just another wild speculation.
And, don’t forget that the third generation was discovered in 1976 with the discovery of the tau lepton! This has, of course, already been given a Nobel prize. It was after the tau was discovered that people took the KM speculation seriously.
Thanks Tony,
“Since machines don’t qualify, the director of Fermilab, as of the year 1994, should get the Nobel prize.”
I agree. They have already given the Nobel Prize to figureheads such as the Director-General of the IAEA http://nobelprize.org/peace/laureates/2005/
As to U.S. taxpayers having the right to have a share in the reward because they funded the research: this argument would also be a good leveller to some of the arrogance in mainstream THEORETICAL physics. Some humility would be inspired if they were reminded that responsibility comes packaged with publishing privileges (arXiv), etc.
I’m uneasy with the idea that big spending Americans can buy Nobel Prizes for dollars. I’m not sure why. For example, if American spending on cancer research led to a breakthrough (say, automated chemical experiments or computer modelling), I’d agree without reserve.
It’s just in physics that it seems a bit unethical money can buy discoveries. Contrast that to the myth of Faraday working an a shoestring even at the height of popularity. However, I think even Faraday was still corrupted by fame.
For all his widely praised ‘humility’ Faraday was a powerful loud showman, who ended up overshadowing others by fame, not necessarily always genius/ability/hard work.
Henry published his discovery of self-induction in the 1832 American Journal of Science. Nobody said ‘wow’! But when in December 1834 Faraday rediscovered it (without citing Henry) and made a big exciting media show of it, people got excited.
A recent example, still very sour, is Einstein’s genius in putting the year 1905 ahead of 1889 when FitzGerald discovered the contraction from an empirical argument based on the Michelson-Morley result. FitzGerald preserves absolute speed of light and has the instrument contract so that you always detect the same relative speed, which is the physics of special relativity.
Once someone is famous for marketing a scientific product (equation or experimental result), they get credited 100% with every pie they had their finger stuck in.
While a small number might be held in mind, I agree, that many individuals can slow the process? 🙂
How many “additional details” would have been needed to progress at any point?
I was never aware of this organizational process before, and the issues faced with who constitues qualification to back tracking of the archive of arxiv, might qualify, comes to mind as well.
It might be interesting to see the “whole system displayed” as to the responsibility of the science process?
While now, our points as layman, might be seen as very insignificant? 🙂 Had always been. Ah, it’s nice we can participate in “some kind of” process anyway.:)
I have been looking recently at Pauli mail during the 1923 angular momentum/spin revolution (it starts with angular momentum being j(j+1) instead of j^2 and it goes, as every revolution, to a rear turmoil finishing with the new quantum mechanics and a new quantum number, spin). Initially it seemed to me that a lot was being discussed as “preliminar” in the letters, but looking at the dates it appears that Pauli first publishs a result in the Z. f. Physik, then proceeds to write to the people speaking more informally about this result. So the letters to Lande and Sommerfeld about (j-1) j are dated May and July respectively, the related published paper can be considered to be the one of April, even if late in the year, in Oktober, another paper goes back about the more speculative content of the letter.
Of course it seems that publication lag was a lot smaller than nowadays, so perhaps the SNO method (publish, then discuss) was the better approach. In some other place Sommerfeld tells that the experimentalist result must be arranged to be published always before the modeller results, but this is during a dispute against Lande for the Paschen-Back results. very muddled with the need of Back (but also of Lande) to claim a “new result” for his habilitation exam.
Hi all,
two things.
One, I totally agree about attributing a Nobel Prize to Kobajashi and Maskawa. They broke a lot of ground, and did so in a number of ways.
But most important, they did it very early on – when quarks were not yet taken very seriosly.
As for an experimentalist who deserves the prize, let me be partisan: Giorgio Bellettini did a lot for CDF, who saw the first top quarks and measured the top mass to be 174 GeV ( a GeV away from current world averages) in April 1994. Along with Giorgio, Alvin Tollestrup would surely be a good candidate for the recipient of the prize.
As for the meaningless discussion about US taxpayers deserving the prize: do not forget that CDF (and D0) are international collaborations. If you believe the US could have done without italians, for instance (once again I feel partisan tonight) thing again. The SVX is almost entirely an italian achievement, without which no top quarks would have been published in 1994 or 1995 by CDF.
So should we award the prize to all italian taxpayers as well ? Give me a break. I vote for Giorgio Bellettini, who deserves it for his career as much as for his huge enthusiasm and energy who allowed CDF to put together a great detector and the first top quark observation.
With respect to my suggestion that USA taxpayers be recognized for their contribution to Fermilab’s T-quark experimental observations, Tommaso Dorigo said:
“… As for the meaningless discussion about US taxpayers deserving the prize: do not forget that CDF (and D0) are international collaborations. … The SVX is almost entirely an italian achievement, without which no top quarks would have been published in 1994 or 1995 by CDF. So should we award the prize to all italian taxpayers as well ?
Give me a break.
I vote for Giorgio Bellettini …”.
Yes, I would include Italian taxpayers and taxpayers from all other governments that participated in Fermilab during the runs leading up to the 1994 announcement FERMILAB-PUB-94/097-E. I hereby apologize for ignoring the other governments in my earlier post.
I don’t see anything wrong with including ALL taxpayers, or why Tommaso would be so scornful of it as to say “Give me a break”.
As to recognition of Giorgio Bellettini individually, I think that he (like many others whose contributions may have been necessary, if not sufficient, for the Fermilab observations) was covered in my earlier comment when I said:
“… the director should accept the award, not only for himself, but also as a representative of everybody who worked at Fermilab, CDF, D0 during the runs leading up to the 1994 announcement FERMILAB-PUB-94/097-E … the Fermilab Public Affairs office could make replicas of the Nobel medallions … for workers at Fermilab, CDF, and D0 (free to them, paid out of the Nobel Prize funds) …”.
Tony Smith
http://www.valdostamuseum.org/hamsmith/
PS – Personally, I am very much biased in favor of a Nobel Prize going to Giorgio Bellettini. Many years ago I attended a conference at Fermilab. I did not rent a car, but rode a bus from the hotel to Fermilab. IIRC (regretfully my memory fades with age) I was the only non-Italian on the bus, and Giorgio Belletini was among the Italians on the bus, and I very much enjoyed bus-ride discussions of things from physics to comparing USA and Italian cultures (including discussion of the USA practice of conversion of food-producing farmland to housing subdivisions with flimsy wood-frame construction).
Pingback: Life as a Physicist » More On Competition
Many thanks to Tony Smith for mentioning my book The Evidence for the Top Quark. I have just a couple of comments pertaining to previous posts. I apologize for coming in rather late on this — I only just came across this discussion.
Regarding Japanese physics, Kobayashi, and Maskawa:
As indicated by Tony’s original comment, my emphasis is not so much on making sure that K-M get the credit they deserve (and certainly they deserve a lot). In the end, they did become institutionalized in the literature, with very high rankings among the SPIRES most cited papers. Prior to about 1977, they were ignored not only in the US, but also in Japan, with almost no citations even in the journal in which they published their paper. (In the book I discuss a number factors that plausibly contributed to the increased attention to their paper, among which certainly is the tau discovery in 76, but also a prominent citation and discussion by Ellis, Gaillard, and Nanopoulos, also in 76.)
Although K-M are often cited, the usual histories leave out the context of their work: the “Nagoya model” developed by Sakata, Maki, Ohnuki, and Nakagawa in papers published in early 60s. Also forgotten is the fact that K-M were reacting to the “X-particle”, a 1971 cosmic ray finding by Japanese experimentalists of a possibly charmed state that they were interpreting as the fourth fundamental “ur-baryon” postulated by the Nagoya model. The usual histories see K-M’s 73 paper as a kind of miraculous anticipation of quarks five and six before quark number four had even been found. Putting their paper in its original context shows that there were specific but forgotten problems they were trying to solve. (Great physicists rather than prophets!)
Regarding the top discovery and the Nobel:
I have hesitated to form any opinion regarding who, if anyone, apart from Kobayashi and Maskawa, should get a Nobel for the top discovery (although such an announcement might give a little (rather badly needed) kick to sales of my book). When I do think about it, Alvin Tollestrup does come to mind — not so much for his direct involvement in the top analysis and write-up itself, but for his role in making the CDF detector a reality. Certainly, as Tommaso Dorigo notes, the role of the Italians, and Giorgio Bellettini in particular, is very important in CDF. So much depends on what kind of contribution we value, though (the obscurity of this issue is what makes me reluctant to form firm opinions on the Nobel questions). My investigations convinced me that a crucial but perhaps unsung hero was Aldo Menzione, a Pisa physicist who adovcated strongly and persistently and with little encouragement for the building of the silicon vertex detector at CDF, based on a kind of fundamental insight into the value of having qualitatively new information about particle processes very close to the beam line. Without that component of the detector, there would have been no result in 94 and a much less impressive result in 95.
Speaking of these two papers — I think this poses another difficulty. In the period 94-95 there were three top results from Fermilab, CDF’s “Evidence for top quark” paper of 94, and the two “Observation of top” papers from CDF and D0 in 95. Although Tony Smith’s proposal is in the spirit of “Nobel prizes all around,” the hallmark he cites is the posting of the 94 CDF paper. This is problematic, though, insofar as many physicists in D0, and some in CDF, regarded the 94 paper as anything but exemplary of experimental physics at its best. In my book I discuss a number of concerns raised about possible biases in that result (based mostly on criticisms from CDF members themselves). I don’t claim that all those criticisms are warranted, but there is a legitimate question as to whether the paper — for all its many remarkable features — should be granted iconic status.
For the 95 results, the story is quite different, but now you do have two results. And you get quite different stories about the relative merits of the two papers from talking to CDF or D0 members respectively.
The experimental confirmation of the top quark is an amazing achievement in experimental physics — and one that can only be regarded as more amazing in light of the complicated “experimental sociology” (maybe “sociology of experiment” would be better?) surrounding it. I suspect that in the end the problem is that the model for scientific credit presupposed by the Nobel mechanism is simply inappropriate for the kind of achievement this is.
Kent– Thanks for chipping in. “Experimental sociology” was intentionally ambiguous, of course.
My personal opinion is that there needn’t be a Nobel given for the top discovery; there’s no rule that says every new particle has to have a Nobel prize that comes along with it, and everyone knew that the top would be there. (Not to slight the actual experimental effort, which was tremendous.) If and when the Higgs and supersymmetry are found, these questions will be very sticky.