Sorry for a post title that will attract the crazies. Carl Zimmer has a story in the New York Times that discusses a growing unease with the practice of science among scientists themselves.
In tomorrow’s New York Times, I’ve got a long story about a growing sense among scientists that science itself is getting dysfunctional. For them, the clearest sign of this dysfunction is the growing rate of retractions of scientific papers, either due to errors or due to misconduct. But retractions represent just the most obvious symptom of deep institutional problems with how science is done these days–how projects get funded, how scientists find jobs, and how they keep labs up and running.
However… essentially all the examples are from biologically-oriented fields. I’ll confess that Carl asked me if there is a similar feeling among physicists, and after some thought I decide that there really isn’t. There are certainly fumbles (faster-than-light neutrinos, anyone?) and scandals (Jan Hendrik Schön being the most obvious), but I don’t have any feeling that the problem is growing in a noticeable way. Biology and physics are fundamentally different, especially because of the tremendous pressure within medical sciences when it comes to any results that might turn out to be medically useful. Cosmologists certainly don’t have to worry about that.
But maybe this is a distorted view from within my personal bubble? Happy to hear informed opinion to the contrary. The relevant kind of informed opinion would actually involve a comparison of the situation today with the situation at some previous time, not just a litany of things you think are dysfunctional about the present day.
Incidentally, Nik #27:
You should have a look at Matt Strassler’s summary of the OPERA fiasco at http://profmattstrassler.com/articles-and-posts/particle-physics-basics/neutrinos/neutrinos-faster-than-light/opera-what-went-wrong/ especially the list down at the bottom of all the lessons from what he calls “a classic case of how not to handle a potentially sensational result”.
In particular, don’t announce something earth-shaking without doing all your basic internal checks first; or if you do announce it, don’t announce it in a high-profile public talk at CERN; or if you have already done that, then when you discover your mistakes have the decency to announce them similarly rather than tucked away in some little mini-workshop at Gran Sasso. I don’t think their handling of the situation has exactly covered them in glory.
Didn’t some Spanish astronomers try to steal an entire dwarf planet a few years back?
Regarding the state of American science (rather than science globally), I find the April 15 article by Kirk Semple “Many U.S. Immigrants’ Children Seek American Dream Abroad” more troubling than Carl Zimmer’s article. While Semple focuses on the business world, part of the strength of American science came from an ability to attract and keep top talent from all around the world. Working in high energy theoretical physics over the last decade, I perceive a marked shift. I see foreign born scientists who come here for grad school and postdocs returning home after shorter periods of time. I also see a troubling number of highly talented U.S. born scientists leaving for places such as China, India, Europe, Korea, and Japan.
Why do they leave? In many cases, the jobs are over seas, and the opportunities are better. For example, in the states a tenure track high energy theorist can apply for an NSF CAREER Award which pays 400,000 USD over a period of five years. Given overhead rates at most U.S. universities, this amount is not enough to hire a postdoc. In Europe, in contrast, an ERC Starting Grant offers 1.5 million Euros over five years, and many European union countries (for example Holland) offer similar grants. Moreover, overhead rates in Europe tend to be much lower and there is no summer salary to consider.
The youngest Nobel-prize-winning hep theorists (‘t Hooft and Wilczek) are 60. In five years there will be no non-retired hep theorist with a Nobel prize, since the next winners (Higgs and perhaps Englert) have already retired. Perhaps not a proof of dysfunction, but a sign of some stagnation at least.
As Sean said, “biology and physics are fundamentally different”.
Maybe out of the scope of this post (or if we look more deeply maybe not), but regarding that, the paper titled “Can a biologist fix a radio” is a nice read, drawing a parallel between biology and engineering, and existence/lack of formal approaches in two areas.
First, there is an upside to retractions. How sad would it be if any field of science stopped caring enough to make retractions after the need for a retraction is discovered?
Second, as Dirk said in comment 1, Lee Smolin makes important points about the trouble with various multiverse hypotheses, not that I am convinced that his upper limit for solar mass in Neutron stars really says much about the probability of a block hole generating a new spacetime quasi-universe (or verse) that forms second generation stars in his own multiverse hypothesis of cosmological natural selection. Nonetheless, Smolin makes good points about problems in other multiverse hypotheses.
Third, most physics experiments in general are easier to control and reproduce compared to medical experiments.
Hard to see how anything could be more dysfunctional than the biomedical field right now. 95% of what isn’t retracted cannot be reproduced, even with the poor excuse for statistical analysis normally employed to give the appearance of “significance”. My career the past six years has been a painstaking attempt to corroborate the work of external researches as our new big-pharma mandate is informed by the notion that internal R&D is a losing proposition. Guess what? The external stuff is garbage. Collaborators take the money and run, accountable to no one, since their labors are validated by “peer review”. It’s gone from disheartening to depressing. If we can’t rely on academic reality checks, what can we rely on?
I am a condensed matter physicist (disclaimer, grad student). In my own sub-field, I see lots of situations where data of questionable quality and impact passes review at the most prestigious journals. Some of the most outlandish claims, instead of being retracted or even corrected, are addressed in further (prestigious) publications, but their nature as corrections to previous claims are left to be read between the lines.
Personally I think many scientists, instead of being encouraged to publish rigorous, strong work, are pressured by the need for publications to rush. It leads to a glut of mediocre work where very few real contributions can be identified.
There is something deeply wrong with the way science works right now, and the retraction issue and job issue are just two tips of this iceberg. In a sense the problem has a philosophical foundation: science is treated as though it is the ultimate method to understand reality, and the views of scientists are assumed to have a legitimacy that they don’t actually have. Because of this science is assumed to have accomplished a lot more in recent decades than it has. It simply doesn’t work as well as people assume.
Nature has an article out on this last year : http://www.nature.com/news/2011/111005/full/478026a.html
I was surprised Carl Zimmer didn’t refero this article.
I’ve had the pleasure of listening to a talk by the author Richard Van Noorden, who showed some really interesting statistics — in particular how most of the retractions occur in biological/medical sciences. I asked him if this is systematic of the field — he said a lot of it is because biological/medical sciences are really big money fields. A result can lead to millions of dollars invested (by pharmaceuticals for example) in finding uses for example. In addition, Van Noorden suggests that the sheer volume of papers in the biological sciences mean that there are just more retractions.
However, I am not too convinced by this argument — engineering papers are pretty big time, and do has impact on the bottom line of tech companies, but retraction rates are low.
I think the reason is because physical sciences have a much better “error feedback loop” — it is often clear what is wrong and what is right, while in biological sciences (due to, I believe, the lack of a simple model and the complexity of the system being studied) it is not clear what is “obvious” and what is not.
Eugene
(if anything, I think Physics suffer from the opposite problem — the fear of being labeled “crackpot” means that most papers written are unadventurous safe papers.)
Pingback: Trust no one, and other lessons I learned from physics reporters : The Last Word On Nothing
“Why do they leave? In many cases, the jobs are over seas, and the opportunities are better. For example, in the states a tenure track high energy theorist can apply for an NSF CAREER Award which pays 400,000 USD over a period of five years. Given overhead rates at most U.S. universities, this amount is not enough to hire a postdoc. In Europe, in contrast, an ERC Starting Grant offers 1.5 million Euros over five years, and many European union countries (for example Holland) offer similar grants. Moreover, overhead rates in Europe tend to be much lower and there is no summer salary to consider.”
While this is true, only a minority of people are funded by the ERC. The ERC is quite new and most funding is on the national level, and not all is good. Also, some countries don’t even have tenure-track jobs (it is either permanent or a temporary postdoc-style job with a 2-year contract or whatever). Summary: Research funding varies enormously from country to country and within a given country in Europe and one can’t compare Europe, which is much less homogeneous than the USA in this respect, as a whole with the USA. Also, not everyone in the USA is funded with an NSF Career Award. Another aspect is that in Europe there are two options: up or out. In the USA, if one isn’t good enough for the big leagues, one can often get a job at a “small teaching college” and still do a bit of research and teach science, as opposed to doing something completely different. Also, I’m sure that the fraction of the NSF and ERC grants for the person’s salary are different.
Lest this sound too USA-friendly, I still think that on the whole the situation is better in Europe since, although there are differences between countries, nowhere is a university education as expensive as it is in the States. Before anyone mentions scholarships, consider that not everyone has one and there are people who leave the field due to lack of money while less qualified people can afford to stay in. Also, many scholarships are for children of employees of certain companies. There is also the case that financial aid from the university involves working in the cafeteria while the rich students chat after lunch (I am not making this up). Also, certain scholarships are targeted at minority or majority students. (I knew someone who had a scholarship paid for by a fund set up by a rich old maid—not sure if she was still alive or if this was from her will—which specified that the recipient had to be white and male and not too old. I am not making this up.)
I agree with the article that the publish-or perish ethos is to blame.
“Publish or perish” seems to me an attempt to use market measures of productivity to science with “how much money did you make for the desk ?” replaced by “how many papers did you publish ?”.
I think its ultimately a disaster for science.
History suggests that the greatest scientific advances were made by people who pursued science as a hobby in some sense.
Darwin comes to mind. So does Einstein who made his first great breakthroughs as a patent clerk doing physics in his spare time.
The biggest advances in physics were all made in the first half of the 20th century when there was no pressure to publish in order to survive.
Salaries were presumably much lower also., but people seem to have made great progress anyway.
The “small businessmen” of modern academia mostly produce a bunch of garbage.
Or at least the substance to garbage ratio is distressingly low.
It all gets back to the incentive structure. There’s an inherent danger in for-profit science, and the fact that there is incentive to cheat should be obvious. One can label that inherent danger “evil” and trash the whole enterprise, or one can live in the real world. That means recognizing how human beings respond to risk and reward, and attempt to incentivize appropriately. For industry, that means punishing cheaters so severely that there is sufficient disincentive despite the potential rewards of cheating. You see what happens, just as in any industry, when you de-regulate. It’s pretty simple, really, as long as your surveillance system is in good working order.
I see academia as a crucial part of that surveillance system. Unfortunately, it’s clear that the incentive structure in academia has been perverted sufficiently to make things almost as bad as one would expect would be the baseline in industry. Is academic science about the expansion of human knowledge? Maybe it should be, but I would say these days it emphatically is not. It’s another growth mechanism for universities; essentially, another revenue stream. Academic scientists are not rewarded for expanding human knowledge. They are rewarded for producing exciting positive data, and hyping it. In the biological sciences, this means your iterative advance in a small sub-field must be touted as foundational, say, the future cure for aging.
And posters above are correct: The experimental systems in the life sciences are so complex that the smallest perturbations can literally change the entire outcome of experiments. And it’s very, very difficult to tease out those subtleties, especially in vivo. Most physics experiments are much more rigorously defined, mathematically precise, and, quite frankly, many orders of magnitude simpler than even the simplest biological systems. The B.S. meter works better for individual experiments, in, say condensed matter physics, because the experiments are amenable to precise critique. There’s a disincentive to cheat, because it’s easier to get caught. Hence physics probably does more for the common good right now, because it’s doing more to truly expand human knowledge. We can dispense with individual values entirely. Structurally, the system is primed to deliver this benefit. At lease, better than the life sciences.
In my very insignificant opinion, there need to be major structural changes in how life sciences are performed, and how graduate students in the life sciences are rewarded for the work they do. Otherwise, expect it to morph entirely into just an extension of for-profit science. For-profit has its useful role to play, but it’s a metastasizing menace unchecked, as our current economic troubles ought to make very clear from the example of other enterprises. Stop using for-profit values to assess academic science, or there will be dire consequences.
The problem with science today is deeper than what commentors are describing above. Science simply hasn’t accomplished nearly as much as people want to think. Most progress in recent decades has come from technology, not basic science. Even human lifespans have hardly changed in the last century. The average 5 year old in 2000 was only likely to live a couple more years than the average 5 year old in 1900.
While publish-or-perish is indeed a very serious problem, it is not at all clear that just because such big advances were made by people who were doing it for fun in the early 20th century, it is possible to have another explosion of knowledge advancement now if we just went back to doing it because we have a passion for it and not out of career advancement and profit motivations. We should go back to doing science for such reasons, but it may be that it is going to be very very hard to make progress from here on no matter which direction the incentive structure points to, simply because the low-hanging fruit has been picked already
Sean – I agree that there is little evidence of a plague of retractions in the physical sciences. Still, that was only the opening point of an article that spent much more time talking about the way academic science gets done today. Do you have any thoughts on that aspect of the article?
@ 41. Quarkgluonsoup
I’m not sure I believe you. Doing a quick calculation from the survivorship tables in the US National Vital Statistics Reports the life expectancy of a five year old changed from 60-65 to 75-80 between 1900 and 2007, not just a couple of years. Why would you want to ignore the vast improvements in early mortality anyway?
this is also off topic, but quarkgluonsoup
why are you starting with 5 year olds?
the infant mortality rate has greatly decreased since 1900.
Pingback: Biomedical science… is there a problem? | The Finch and Pea
@DirkHanson – No offense to Lee Smolin, and personally I think The Trouble with Physics is a great book, but reading that book doesn’t mean you know all of physics. One should try doing some maths sometime; that should tell them a whole lot more, so you can form a more informed opinion next time.
You don’t see a problem with how physicists find jobs? Really? With the large number of grad students that spend seven years in grad school, at tremendous opportunity cost, and then spend three or four years in postdocs, at the end of which they find VAP positions waiting for them, and then finally wash out int their mid-40s, you see no problem at all?