Despite the fact that the arxiv has made it possible to disseminate papers well before they are sent to a journal, the process of anonymous peer review is still crucial to physics and the rest of science. Anyone who has at least a couple of published papers has appeared on the radar screen of various journals as a potential referee, and pretty soon the requests to review papers come fast and furious. And it’s not a matter of rubber-stamping; I’ve personally refereed about 100 papers, and recommended less than half of them for publication. Of course, individual referees can behave quite differently; editors like referees who will actually read the paper, are willing to reject it if it’s bad, and get the reviews back quickly. I used to be good at all three of those, although my record on the last point has deteriorated seriously of late.
Every paper sent to a journal like Physical Review (in all of its contemporary manifestations) is sent to a referee as a matter of course. It wasn’t always thus. The current issue of Physics Today has a great article about Albert Einstein’s run-in with the journal in 1936.
In his salad days, Einstein published in German journals such as Annalen der Physik, but he eventually switched to American journals after he moved to the U.S. He had published a couple of papers in the Physical Review, which were apparently accepted by editor John Tate without being sent to a referee. These included the famous Einstein, Podolsky and Rosen paper on nonlocality in quantum mechanics, “Can Quantum-Mechanical Description of Physical Reality Be Considered Complete?”
But in 1936 Einstein and Rosen submitted a paper on the existence of gravitational waves that struck Tate as suspicious, and he decided to send it to the referee. The Physics Today article reveals that the referee was relativist Howard Percy Robertson. Soon after the initial formulation of general relativity, Einstein predicted the existence of gravitational waves by doing the obvious thing — examining the behavior of small fluctuations in the gravitational field using perturbation theory. But Einstein and Rosen had attempted to solve the full equations without any approximations, and were able to prove that there were no non-singular solutions; they therefore claimed that gravitational waves didn’t exist! Robertson figured out that they had made a classic error in GR — essentially, they had used a bad coordinate system. He wrote a ten-page report explaining why the conclusions of the paper were incorrect.
Einstein explained that he had submitted his paper for publication, not for refereeing.
Dear Sir,
We (Mr. Rosen and I) had sent you our manuscript for publication and had not authorized you to show it to specialists before it is printed. I see no reason to address the — in any case erroneous — comments of your anonymous expert. On the basis of this incident I prefer to publish the paper elsewhere.
Respectfully,
P.S. Mr. Rosen, who has left for the Soviet Union, has authorized me to represent him in this matter.
After this incident, Einstein vowed never again to publish in Physical Review — and he didn’t. The Einstein-Rosen paper eventually appeared in the Journal of the Franklin Institute, but its conclusions were dramatically altered — the authors chose new coordinates, and showed that they had actually discovered a solution for cylindrical gravitational waves, now known as the “Einstein-Rosen metric.” It’s a little unclear how exactly Einstein changed his mind — whether it was of his own accord, through the influence of the referee’s report, or by talking to Robertson personally. But it’s pretty clear that he would have loved the innovation of arxiv.org.
I suppose Lubos Motl can take comfort in the fact that even Einstein got confused about coordinate systems in GR on occasion.
Er, Sean, what kind of Pandora box are you opening here?
In any case, I’d love to see ten pages in a referee report, nowadays.
Via Daniel Doro Ferrante:
The Convergence of Digital-Libraries and the Peer-Review Process
cs.DL/0504084
I agree that peer review is important, but I do not agree that it needs to be anonymous. The only part that sometimes needs to be anonymous is privately rejecting or criticizing a paper.
I talk about what I think peer review should look like in the last part of math.HO/0210144.
This is amusing just for the fack that it shows Einstein in a more “human” albeit stubborn mindset then one often gets from the history of science. Scientists who get into the history books are brilliant individuals but they can be just as frustratingly obtuse sometimes as their lessers.
I read the Einstein article with interest. Is it my imagination though or is peer review in physics and mathematical physics getting tougher? It seems tougher now than in the 90s (or maybe I am now just losing my edge:). I get the impression that after the Jan Schon affair, with the guy from Bell Labs publishing a lot of deliberately fraudulent papers in top journals, that most journals are subjecting submissions to even greater scrutiny. Also the Bogdanov affair highlighted issues of peer review in physics.
I strongly feel that peer review should be rigorous but fair. Ideally, papers should be judged without bias as regards to the sex, race, country or home institution of the authors. In this regard, I think referees should get just a manuscript with no name or affiliation so that it is judged purely on the contents, i.e double-blind reviewing. I also feel there is a danger that if peer review is too conservative and rigid that important new ideas will get strangled at birth. Often very important new ideas can emerge in an incomplete or confused form, with a deeper and clearer understanding coming later, and allowances must be made for that. I think the arxiv serves a very important function in this respect though.
“…it’s pretty clear that he would have loved the innovation of arxiv.org”
He would have loved it while he was deep in umbrage at having his paper rejected. But then he wouldn’t have had the chance to correct his reasoning before publication, and his error would have been common knowledge. It might have been a short-lived affair.
Marty, true enough. Especially now that you can’t remove previous versions, only add updated ones. Also, Einstein was notoriously uninterested in citing other people’s work — the Einstein-Rosen paper cited precisely zero other papers.
“…he would have loved the innovation of arxiv.org”
I wonder. Maybe the way it used to work in the past, but after the introduction of the current endorsement system,
http://arxiv.org/help/endorsement
would he even have been able to get his papers uploaded? Who would he have asked, and who would have agreed to endorse him? Approaching people he didn’t know personally, he would have had to show them his work first – and run the obvious risk of it being stolen. It would have been the word of a professor against that of a total unknown.
Turn it around: suppose there is some new Einstein out there, right now, with the Ultime Unified Theory all LaTeX-ed up and ready to publish. Alas, he is not in academia; he works at the Indoesian Patent Office. What should he do?
Sean, suppose he came across this site and got the idea that you, or Mark, or Clifford seem like nice people whom he could ask for endorsement on the arxiv. So he takes a chance and sends you a mail asking for endorsement. What would you do?
If there are any new Einsteins out there with a correct theory of everything all LaTeXed up, they should feel quite willing to ask me for an endorsement for the arxiv; I’d be happy to bask in the reflected glory and earn a footnote in their triumphant autobiography. More likely, however, they will just send their paper to Physical Review, where it will be accepted and published, and they will become famous without my help.
If, on the other hand, there is anyone out there who thinks they are the next Einstein, but really they are just a crackpot, don’t bother; I get things like that all the time. Sadly, the real next-Einsteins only come along once per century, whereas the crackpots are far too common.
p.s. Risa and JoAnne are also nice people, even if they are of a different gender.
I’m surprised that nobody has commented on what seems to many people to be the catastrophic effect of the arxiv on refereeing standards. Many referees seem to think that since the arxiv has, at the very least, made paper journals less important — and of course lots of people would say “completely superfluous” — therefore they don’t have to take their job seriously any more. Certainly I have recently come across many examples of refereeing that was not just slipshod but actually irresponsible. For example, a colleague had a paper on brane worlds rejected because the referee thought that that brane worlds was a nonsensical subject. That was the whole report: brane worlds are nonsense. This was a famous journal. The editors backed him up. I myself had a paper rejected because the referee thought that equation A followed from equation B — it didn’t, but nor had I claimed that it did. Again the editor backed up the referee even though the latter had demonstrably made a simple error. I know people who have given up submitting papers to journals — the arxiv is good enough for them, and they are sick and tired of dealing with the ignorant twits who make up such a large proportion of the refereeing community.
“Risa and JoAnne are also nice people, even if they are of a different gender.”
Ah yes, but this hypothetical Indonesian also happens to be a traditionalist Muslim with very strong views on the place of women, see. ;^) Let’s throw in badly broken English and plentiful spellling errors for good measure. Imagine somebody like the early Srinivasa Ramanujan, just… more so; I have this notion that the next great towering figure in physics is likely to be from the “third” world, to use an obsolete and un-PC term (on purely statistical grounds if nothing else, but I also have cultural factors in mind) and that he (or she, though statistics favour it being a “he”) is most definitely likely to be an outsider, like his predecessors. Would somebody fitting this description really stand a reasonable chance of getting his work published in the current system? If not, and I’m afraid the answer indeed is not, we have a problem.
Dissident, do we really have a problem?
So many scientists are earning a living and having fun trying to advance our theories of the universe. If someone comes along and publishes a (manifestly) correct theory of everything, then bang, game over. Time for a career change.
Of course, if the correct theory of everything has useful practical applications, then we might be poorer for not knowing it right now… but I very much doubt that.
😉
Greg, we may agree about the “having fun” part; the part about “earning a living” strikes me as rather more questionable, especially under the premise you put forth. Considering that most fundamental research is tax financed, you are essentially describing a tax fraud conspiracy with an active interest in suppressing real progress, as that would imply the risk of getting a real job. The horror!
Well, I mentioned that I thought peer review was getting tougher but maybe I should have said that it is getting more pedantic and petty. Some referees also seem very narrow minded in some regards and some might even feel threatened by good work being presented within their own field of knowledge. New ideas or methods can get a hard time too even if presented with technical competance. I would say this “new Einstein” working in the Indonesian post office or wherever, will have a pretty hard time of it unless they get the attention of some top person. I still feel a fair peer review system is necessary though otherwise the potential for self delusion in theoretical physics is vast, as is the potential to slide into crackpottery.
Incidently, the idea that “a theory of everything” will come along and “end physics” is a complete myth: the human imagination will always find something new to explore within physics and mathematics, some new aspect, consequence, interpretation or application.
Pingback: Daniel Lemire’s blog » Einstein vs. Physical Review