The Trouble With Physics

I was asked to review Lee Smolin’s The Trouble With Physics by New Scientist. The review has now appeared, although with a couple of drawbacks. Most obviously, only subscribers can read it. But more importantly, they have some antiquated print-journal notion of a “word limit,” which in my case was about 1000 words. When I started writing the review, I kind of went over the limit. By a factor of about three. This is why the Intelligent Designer invented blogs; here’s the review I would have written, if the Man hadn’t tried to stifle my creativity. (Other reviews at Backreaction and Not Even Wrong; see also Bee’s interview with Lee, or his appearance with Brian Greene on Science Friday.)

——————————————————————

It was only after re-reading and considerable head-scratching that I figured out why Lee Smolin’s The Trouble With Physics is such a frustrating book: it’s really two books, with intertwined but ultimately independent arguments. One argument is big and abstract and likely to be ignored by most of the book’s audience; the other is narrow and specific and part of a wide-ranging and heated discussion carried out between scientists, in the popular press, and on the internet. The abstract argument — about academic culture and the need to nurture speculative ideas — is, in my opinion, important and largely correct, while the specific one — about the best way to set about quantizing gravity — is overstated and undersupported. It’s too bad that vociferous debate over the latter seems likely to suck all the oxygen away from the former.

Fundamental physics (for want of a better term) is concerned with the ultimate microscopic laws of nature. In our current understanding, these laws describe gravity according to Einstein’s general theory of relativity, and everything else according to the Standard Model of particle physics. The good news is that, with just a few exceptions (dark matter and dark energy, neutrino masses), these two theories are consistent with all the experimental data we have. The bad news is that they are mutually inconsistent. The Standard Model is a quantum field theory, a direct outgrowth of the quantum-mechanical revolution of the 1920’s. General relativity (GR), meanwhile, remains a classical theory, very much in the tradition of Newtonian mechanics. The program of “quantum gravity” is to invent a quantum-mechanical theory that reduces to GR in the classical limit.

This is obviously a crucially important problem, but one that has traditionally been a sidelight in the world of theoretical physics. For one thing, coming up with good models of quantum gravity has turned out to be extremely difficult; for another, the weakness of gravity implies that quantum effects don’t become important in any realistic experiment. There is a severe conceptual divide between GR and the Standard Model, but as a practical matter there is no pressing empirical question that one or the other of them cannot answer.

Quantum gravity moved to the forefront of research in the 1980’s, for two very different reasons. One was the success of the Standard Model itself; its triumph was so complete that there weren’t any nagging experimental puzzles left to resolve (a frustrating situation that persisted for twenty years). The other was the appearance of a promising new approach: string theory, the simple idea of replacing elementary point particles by one-dimensional loops and segments of “string.” (You’re not supposed to ask what the strings are made of; they’re made of string stuff, and there are no deeper layers.) In fact the theory had been around since the late 1960’s, originally investigated as an approach to the strong interactions. But problems arose, including the unavoidable appearance of string states that had all the characteristics one would expect of gravitons, particles of gravity. Whereas most attempts to quantize gravity ran quickly aground, here was a theory that insisted on the existence of gravity even when we didn’t ask for it! In 1984, Michael Green and John Schwarz demonstrated that certain potentially worrisome anomalies in the theory could be successfully canceled, and string mania swept the particle-theory community.

In the heady days of the “first superstring revolution,” triumphalism was everywhere. String theory wasn’t just a way to quantize gravity, it was a Theory of Everything, from which we could potentially derive all of particle physics. Sadly, that hasn’t worked out, or at least not yet. (String theorists remain quite confident that the theory is compatible with everything we know about particle physics, but optimism that it will uniquely predict the low-energy world is at a low ebb.) But on the theoretical front, there have been impressive advances, including a “second revolution” in the mid-nineties. Among the most astonishing results was the discovery by Juan Maldacena of gauge/gravity duality, according to which quantum gravity in a particular background is precisely equivalent to a completely distinct field theory, without gravity, in a different number of dimensions! String theory and quantum field theory, it turns out, aren’t really separate disciplines; there is a web of dualities that reveal various different-looking string theories as simply different manifestations of the same underlying theory, and some of those manifestations are ordinary field theories. Results such as this convince string theorists that they are on the right track, even in the absence of experimental tests. (Although all but the most fervent will readily agree that experimental tests are always the ultimate arbiter.)

But it’s been a long time since the last revolution, and contact with data seems no closer. Indeed, the hope that string theory would uniquely predict a model of particle physics appears increasingly utopian; these days, it seems more likely that there is a huge number (10⁵⁰⁰ or more) phases in which string theory can find itself, each featuring different particles and forces. This embarrassment of riches has opened a possible explanation for apparent fine-tunings in nature — perhaps every phase of string theory exists somewhere, and we only find ourselves in those that are hospitable to life. But this particular prediction is not experimentally testable; if there is to be contact with data, it seems that it won’t be through predicting the details of particle physics.

It is perhaps not surprising that there has been a backlash against string theory. Lee Smolin’s The Trouble With Physics is a paradigmatic example, along with Peter Woit’s new book Not Even Wrong. Both books were foreshadowed by Roger Penrose’s massive work, The Road to Reality. But string theorists have not been silent; several years ago, Brian Greene’s The Elegant Universe was a surprise bestseller, and more recently Leonard Susskind’s The Cosmic Landscape has focused on the opportunities presented by a theory with 10⁵⁰⁰ different phases. Alex Vilenkin’s Many Worlds in One also discusses the multiverse, and Lisa Randall’s Warped Passages enthuses over the possibility of extra dimensions of spacetime — while Lawrence Krauss’s Hiding in the Mirror strikes a skeptical note. Perhaps surprisingly, these books have not been published by vanity presses — there is apparently a huge market for popular discussions of the problems and prospects of string theory and related subjects.

Smolin is an excellent writer and a wide-ranging thinker, and his book is extremely readable. He adopts a more-in-sorrow-than-in-anger attitude toward string theory, claiming to appreciate its virtues while being very aware of its shortcomings. The Trouble with Physics offers a lucid introduction to general relativity, quantum mechanics, and string theory itself, before becoming more judgmental about the current state of the theory and its future prospects.

There is plenty to worry or complain about when it comes to string theory, but Smolin’s concerns are not always particularly compelling. For example, there are crucially important results in string theory (such as the fundamental fact that quantum-gravitational scattering is finite, or the gauge/gravity duality mentioned above) for which rigorous proofs have not been found. But there are proofs, and there are proofs. In fact, there are almost no results in realistic quantum field theories that have been rigorously proven; physicists often take the attitude that reasonably strong arguments are enough to allow us to accept a claim, even in the absence of the kind of proof that would make a mathematician happy. Both the finiteness of stringy scattering and the equivalence of gauge theory and gravity under Maldacena’s duality are supported by extremely compelling evidence, to the point where it has become extremely hard to see how they could fail to be true.

Smolin’s favorite alternative to string theory is Loop Quantum Gravity (LQG), which has grown out of attempts to quantize general relativity directly (without exotica such as supersymmetry or extra dimensions). To most field theorists, this seems like a quixotic quest; general relativity is not well-behaved at short distances and high energies, where such new degrees of freedom are likely to play a crucial role. But Smolin makes much of one purported advantage of LQG, that the theory is background-independent. In other words, rather than picking some background spacetime and studying the propagation of strings (or whatever), LQG is formulated without reference to any specific background.

It’s unclear whether this is really such a big deal. Most approaches to string theory are indeed background-dependent (although in some cases one can quibble about definitions), but that’s presumably because we don’t understand the theory very well. This is an argument about style; in particular, how we should set about inventing new theories. Smolin wants to think big, and start with a background-independent formulation from the start. String theorists would argue that it’s okay to start with a background, since we are led to exciting new results like finite scattering and gauge/gravity duality, and a background-independent formulation will perhaps be invented some day. It’s not an argument that anyone can hope to definitively win, until the right theory is settled and we can look back on how it was invented.

There are other aspects of Smolin’s book that, as a working physicist, rub me the wrong way. He puts a great deal of emphasis on connection to experimental results, which is entirely appropriate. However, he tends to give the impression that LQG and other non-stringy approaches are in close contact with experiment in a way that string theory is not, and I don’t think there’s any reasonable reading on which that is true. There may very well be certain experimental findings — which haven’t yet happened — that would be easier to explain in LQG than in string theory. But the converse is certainly equally true; the discovery of extra dimensions is the most obvious example. As far as I can tell, both string theory and LQG (and every other approach to quantizing gravity) are in the position of not making a single verifiable prediction that, if contradicted by experiment, would falsify the theory. (I’d be happy to hear otherwise.)

Smolin does mention a number of experimental results that have already been obtained, but none of them is naturally explained by LQG any more than by string theory, and most of them are, to be blunt, likely to go away. He mentions the claimed observation that the fine-structure constant is varying with time (against which more precise data has already been obtained), certain large-angle anomalies in the cosmic microwave background anisotropy, and the possibility of large-scale modifications of general relativity replacing dark matter. (Bad timing on that one.) I don’t know of any approach to quantum gravity that firmly predicts (or even better, predicted ahead of time) that any of these should be true. That’s the least surprising thing in the world; gravity is a weak force, and most of the universe is in the regime where it is completely classical.

Smolin also complains about the tendency of string theorists to hype their field. It is hard to argue with that; as a cosmologist, of course, it is hard to feel morally superior, either. But Smolin does tend to project such a feeling of superiority, often contrasting the careful and nuanced claims of LQG to the bombast of string theory. Yet he feels comfortable making statements such as (p. 232)

Loop quantum gravity already has elementary particles in it, and recent results suggest that this is exactly the right particle physics: the standard model.

There are only two ways to interpret this kind of statement: either (1) we have good evidence that quantum spacetime alone, without additional fields, supports excitations that have the right kinds of interactions and quantum numbers to be the particles of the Standard Model, which would be the most important discovery in physics since the invention of quantum mechanics, or (2) it’s hype. Time will tell, I suppose. The point being, it’s perfectly natural to get excited or even overenthusiastic when one is working on ideas of fantastic scope and ambition; at the end of the day, those ideas should be judged on whether they are right or wrong, not whether their proponents were insufficiently cautious and humble.

To date, the string theorists are unambiguously winning the battle for support within the physics community. Success is measured primarily by faculty positions and grant money, and these flow to string theorists much more than to anyone pursuing other approaches to quantum gravity. From an historical perspective, the unusual feature of this situation is that there are any resources being spent on research in quantum gravity; if string theory were suddenly to fall out of favor, it seems much more likely that jobs and money would flow to particle phenomenology, astrophysics, or other areas of theory than to alternative approaches to quantum gravity.

It seems worth emphasizing that the dominance of string theory is absolutely not self-perpetuating. When string theorists apply for grants, they are ultimately judged by program officers at the National Science Foundation or the Department of Energy, the large majority of whom are not string theorists. (I don’t know of any who are, off the top of my head.) And when string theorists apply for faculty jobs, it might very well be other string theorists who decide which are the best candidates, but the job itself must be approved by the rest of the department and by the university administration. String theorists have somehow managed to convince all of these people that their field is worthy of support; I personally take the uncynical view that they have done so through obtaining interesting results.

Smolin talks a great deal about the need for physics, and academia more generally, to support plucky upstart ideas and scholars with the courage and vision to think big and go against the grain. This is a larger point than the specific argument about how to best quantize gravity, and ultimately far more persuasive; it is likely, unfortunately, to be lost amidst the conflict between string theory and its discontents. Faculty positions and grant money are scarce commodities, and universities and funding agencies are naturally risk-averse. Under the current system, a typical researcher might spend five years in graduate school, three to six as a postdoc, and another six or seven as an assistant professor before getting tenure — with an expectation that they will write several competent papers in every one of those years. Nobody should be surprised that, apart from a few singular geniuses, the people who survive this gauntlet are more likely to be those who show technical competence within a dominant paradigm, rather than those who will take risks and pursue their idiosyncratic visions. The dogged pursuit of string theory through the 1970’s by Green and Schwarz is a perfect example of the ultimate triumph of the latter approach, and Smolin is quite correct to lament the lack of support for this kind of research today.

In the real world, it’s difficult to see what to do about the problem. I would be happy to see longer-term postdocs, or simply fewer postdocs before people move on to assistant professorships. But faculty positions are extremely rare — within fundamental theory, a good-sized department might have two per decade, and it would be hard to convince a university to take a long-shot gamble on someone outside the mainstream just for the greater good of the field as a whole. And a gamble it would certainly be. Smolin stacks the deck by contrasting the “craftsmen” who toil within string theory to the “seers” who pursue alternatives, and it’s pretty obvious which is the more romantic role. Many physicists are more likely to see the distinction as one between “doers” and “dreamers,” or even (among our less politic colleagues) between “scientists” and “crackpots.”

To be clear, the scientists working on LQG and other non-stringy approaches to quantum gravity are not crackpots, but honest researchers tackling a very difficult problem. Nevertheless, for the most part they have not managed to convince the rest of the community that their research programs are worthy of substantial support. String theorists are made, not born; they are simply physicists who have decided that this is the best thing to work on right now, and if something better comes along they would likely switch to that. The current situation could easily change. Many string theorists have done interesting work in phenomenology, cosmology, mathematical physics, condensed matter, and even loop quantum gravity. If a latter-day Green and Schwarz were to produce a surprising result that convinced people that some alternative to string theory were more promising, it wouldn’t take long for the newcomer to become dominant. Alternatively, if another decade passes without substantial new progress within string theory, it’s not hard to imagine that people will lose interest and switch to other problems. I would personally bet against this possibility; string theory has proved to be a remarkably fruitful source of surprising new ideas, and there’s no reason to expect that track record to come to a halt.

Smolin is right in the abstract, that we should try to nurture a diversity of approaches to difficult questions in physics, even if his arguments on the specific example of string theory and its competitors are less compelling. But he is also right that string theorists are not always as self-critical as they could be, and can even occasionally be a mite arrogant (although I haven’t found this quality to be rare within academia). The best possible consequence of the appearance of The Trouble with Physics and similar books would be that physicists of all stripes are moved to take an honest look at the strengths and weaknesses of their own research programs, and to maintain an open mind about alternatives. (The worst possible consequence would be for large segments of the public, or the student population, or even physicists in other specialties, to misunderstand why string theorists find their field so compelling.) Sometimes a little criticism can be a healthy thing.