A Personal Narrative

I was very pleased to learn that I’m among this year’s recipients of a Guggenheim Fellowship. The Fellowships are mid-career awards, meant “to further the development of scholars and artists by assisting them to engage in research in any field of knowledge and creation in any of the arts, under the freest possible conditions and irrespective of race, color, or creed.” This year 173 Fellowships were awarded, chosen from 3,100 applications. About half of the winners are in the creative arts, and the majority of those remaining are in the humanities and social sciences, leaving eighteen slots for natural scientists. Only two physicists were chosen, so it’s up to Philip Phillips and me to uphold the honor of our discipline.

The Guggenheim application includes a “Career Narrative” as well as a separate research proposal. I don’t like to share my research proposals around, mostly because I’m a theoretical physicist and what I actually end up doing rarely bears much resemblance to what I had previously planned to do. But I thought I could post my career narrative, if only on the chance that it might be useful to future fellowship applicants (or young students embarking on their own research careers). Be warned that it’s more personal than most things I write on the blog here, not to mention that it’s beastly long. Also, keep in mind that the purpose of the document was to convince people to give me money — as such, it falls pretty heavily on the side of grandiosity and self-justification. Be assured that in real life I remain meek and humble.

---

Sean M. Carroll: Career Narrative

Reading over applications for graduate school in theoretical physics, one cannot help but be struck by a certain common theme: everyone wants to discover the fundamental laws of nature, quantize gravity, and find a unified theory of everything. That was certainly what interested me, ever since I first became enamored with physics when I was about ten years old. It’s an ambitious goal, worthy of pursuing, and I’ve been fortunate enough to contribute to the quest in my own small way over the course of my research career, especially in gravitational physics and cosmology.

But when a goal is this far-reaching, it’s important to keep in mind different routes to the ultimate end. In recent years I have become increasingly convinced that there is important progress to be made by focusing on emergence: how the deepest levels of reality are connected to the many higher levels of behavior we observe. How do spacetime and classical reality arise from an underlying quantum description? What is complexity, and how does it evolve over time, and how is that evolution driven by the increase of entropy? What do we mean when we talk about “causes” and “purposes” if the underlying laws are perfectly reversible? What role does information play in the structure of reality? All of these questions are thoroughly interdisciplinary in nature, and can be addressed with a wide variety of different techniques. I strongly believe that the time is right for groundbreaking work in this area, and a Guggenheim fellowship would help me develop the relevant expertise and start stimulating new collaborations.

University, Villanova and Harvard: 1984-1993

There is no question I am a physicist. The topics that first sparked my interest in science – the Big Bang, black holes, elementary particles – are the ones that I think about today, and they lie squarely within the purview of physics. So it is somewhat curious that I have no degrees in physics. For a variety of reasons (including questionable guidance), both my undergraduate degree from Villanova and my Ph.D. from Harvard are in astronomy and astrophysics. I would like to say that this was a clever choice based on a desire for interdisciplinary engagement, but it was more of an accident of history (and a seeming insistence on doing things the hard way). Villanova offered me a full-tuition academic scholarship (rare at the time), and I financed my graduate education through fellowships from NASA and the National Science Foundation.

Nevertheless, my education was extremely rewarding. As an undergraduate at a very small but research-oriented department, I got a start in doing real science at an early age, taking photometric data on variable stars and building models based on their light curves [Carroll, Guinan, McCook and Donahue, 1991]. In graduate school I was surrounded by incredible resources in the Cambridge area, and made an effort to take advantage of them. My advisor, George Field, was a well-established theoretical astrophysicist, specializing in magnetohydrodynamics and the interstellar medium. He wasn’t an expert in the area that I wanted to study, the particle physics/cosmology connection, but he was curious about it. So we essentially learned things together, writing papers on alternatives to general relativity, the origin of intergalactic magnetic fields, and inflationary cosmology, including one of the first studies of a non-Lorentz-invariant modification of electromagnetism [Carroll, Field, and Jackiw 1990]. George also encouraged me to work with others, and I collaborated with fellow graduate students on topics in mathematical physics and topological defects, as well as with Edward Farhi and Alan Guth from MIT on closed timelike curves (what people on the street call “time machines”) in general relativity [Carroll, Farhi, and Guth 1992].

Setting a pattern that would continue to be followed down the line, I didn’t limit my studies to physics alone. In particular, my time at Villanova ignited an interest in philosophy that remains strong to this day. I received a B.A. degree in “General Honors” as well as my B.S. in Astronomy and Astrophysics, and also picked up a philosophy minor. At Harvard, I sat in on courses with John Rawls, Robert Nozick, and Barbara Johnson. While science was my first love and remains my primary passion, the philosophical desire to dig deep and ask fundamental questions continues to resonate strongly with me, and I’m convinced that familiarity with modern philosophy of science can be invaluable to physicists trying to tackle questions at the foundations of the discipline.

Postdoctoral, MIT and ITP: 1993-1999

For my first postdoctoral fellowship, in 1993 I moved just a bit down the road, from Harvard to MIT; three years later I would fly across the country to the prestigious Institute for Theoretical Physics at UC Santa Barbara. At both places I continued to do research in a somewhat scattershot fashion, working on a potpourri of topics in gravitation and field theory, usually in collaboration with other physicists my age rather than with the senior professors. I had great fun, writing papers on supergravity (the supersymmetric version of general relativity), topological defects, perturbations of the cosmic microwave background radiation, two-dimensional quantum gravity, interacting dark matter, and tests of the large-scale isotropy of the universe.

Although I was slow to catch on, the academic ground was shifting beneath me. The late 80’s and early 90’s, when I was a graduate student, were a sluggish time in particle physics and cosmology. There were few new experimental results; the string theory revolution, which generated so much excitement in the early 80’s, had not lived up to its initial promise; and astronomers continued to grapple with the difficulties in measuring properties of the universe with any precision. In such an environment, my disjointed research style was enough to get by. But as I was graduating with my Ph.D., things were changing. In 1992, results from the COBE satellite showed us for the first time the tiny temperature variations in the cosmic background radiation, representing primordial density fluctuations that gradually grew into galaxies and large-scale structure. In 1994-95, a series of theoretical breakthroughs launched the second superstring revolution. Suddenly, it was no longer good enough just to be considered smart and do random interesting things. Theoretical cosmologists dived into work on the microwave background, or at least models of inflation that made predictions for it; field theorists and string theorists were concentrating on dualities, D-branes, and the other shiny new toys that the latest revolution had brought them. In 1993 I was a hot property on the postdoctoral job market, with multiple offers from the very best places; by 1996 those offers had largely dried up, and I was very fortunate to be offered a position at a place as good as ITP.

Of course, nobody actually told me this in so many words, and it took me a while to figure it out. It’s a valuable lesson that I still take to heart – it’s not good enough to do work on things you think are interesting, you have to make real contributions that others recognize as interesting, as well. I don’t see this as merely a cynical strategy for academic career success. As enjoyable and stimulating as it may be to bounce from topic to topic, the chances of make a true and lasting contribution are larger for people who focus on an area with sufficient intensity to master it in all of its nuance.

What I needed was a topic that I personally found fascinating enough to investigate in real detail, and which the rest of the community recognized as being of central importance. Happily, the universe obligingly provided just the thing. In 1998, two teams of astronomers, one led by Saul Perlmutter and the other by Brian Schmidt and Adam Riess, announced an amazing result: our universe is not only expanding, it’s accelerating. Although in retrospect there were clues that this might have been the case, it took most of the community by complete surprise, and certainly stands as the most important discovery that has happened during my own career. Perlmutter, Schmidt, and Riess shared the Nobel Prize in 2011.

Like many other physicists, my imagination was immediately captured by the question of why the universe is accelerating. Through no planning of my own, I was perfectly placed to dive into the problem. Schmidt and Riess had both been fellow graduate students of mine while I was at Harvard (Brian was my officemate), and I had consulted with Perlmutter’s group early on in their investigations, so I was very familiar with the observations of Type Ia supernovae on which the discovery was based. The most obvious explanation for universal acceleration is that empty space itself carries a fixed energy density, what Einstein had labeled the “cosmological constant”; I happened to be a co-author, with Bill Press and Ed Turner, on a 1992 review article on the subject that had become a standard reference in the field [Carroll, Press, and Turner 1992], and which hundreds of scientists were now hurriedly re-reading. In 1997 Greg Anderson and I had proposed a model in which dark-matter particles would interact with an ambient field, growing in mass as the universe expands [Anderson and Carroll 1997]; this kind of model natural leads to cosmic acceleration, and was an early idea for what is now known as “dark energy” (as well as for the more intriguing possibility that there may be a variety of interactions within a rich “dark sector”).

With that serendipitous preparation, I was able to throw myself into the questions of dark energy and the acceleration of the universe. After the discovery was announced, models were quickly proposed in which the dark energy was a dynamically-evolving field, rather than a constant energy density. I realized that most such models were subject to severe experimental constraints, because they would lead to new long-range forces and cause particle-physics parameters to slowly vary with time. I wrote a paper [Carroll 1998] pointing out these features, as well as suggesting symmetries that could help avoid them. I also collaborated with the Schmidt/Riess group on a pioneering paper [Garnavich et al. 1998] that placed limits on the rate at which the density of dark energy could change as the universe expands. With this expertise and these papers, I was suddenly a hot property on the job market once again; in 1999 I accepted a junior-faculty position at the University of Chicago.

University of Chicago: 1999-2006

While I was a postdoc, for the most part my intellectual energies were devoted completely to research. As a new faculty member, I had the responsibility and opportunity to expand my reach in a variety of ways. I had always loved teaching, and took to it with gusto, pioneering new courses (undergraduate general relativity, graduate cosmology), and winning a “Spherical Cow” teaching award from the physics graduate students. I developed my lecture notes for a graduate course in general relativity into a textbook, Spacetime and Geometry, which is now used widely in universities around the world. I helped organize a major international conference (Cosmo-02), served on a number of national committees (including the roadmap team for NASA’s Beyond Einstein program), and was a founding member and leader of the theory group at Chicago’s Kavli Institute for Cosmological Physics. I was successful at bringing in money, including fellowships from the Sloan and Packard Foundations. I made connections with professors in other departments, and started to work with Project Exploration, an outreach nonprofit led by Gabrielle Lyon and Chicago paleontologist Paul Sereno. With Classics professor Shadi Bartsch, I taught an undergraduate humanities course on the history of atheism. I became involved in the local theatre community, helping advise companies that were performing plays with scientific themes (Arcadia, Proof, Humble Boy). And in 2004 I took up blogging at my site Preposterous Universe, a fun and stimulating pastime that I continue to this day.

Research, of course, was still central, and I continued to concentrate on the challenge posed by the accelerating universe, especially in a series of papers with Mark Trodden (then at Syracuse, now at U. Penn.) and other collaborators. Among the more speculative ideas that had been proposed was “phantom energy,” a form of dark energy whose density actually increases as the universe expands. In one paper [Carroll, Hoffman, and Trodden 2003] we showed that such theories tended to be catastrophically unstable, and in another [Carroll, De Felice, and Trodden 2004] we showed that more complex models could nevertheless trick observers into concluding that the dark energy was phantom-like.

Our most influential work proposed a simple idea: that there isn’t any dark energy at all, but rather that general relativity breaks down on cosmological scales, where new dynamics can kick in [Carroll, Duvvuri, Trodden, and Turner 2004]. This became an extremely popular scenario within the theoretical cosmology community, launching a great deal of work devoted to investigating these “f(R) theories.” (The name refers to the fact that the dynamical equations are based on an arbitrary function of R, a quantity that measures the curvature of spacetime.) This work included papers by our group looking at long-term cosmological evolution in such models [Carroll et al. 2004], and studying the formation of structure in theories designed to be compatible with observational constraints on modified gravity [Carroll, Sawicki, Silvestri, and Trodden 2006].

Being of restless temperament, I couldn’t confine myself to only thinking about dark energy and modified gravity. I published on a number of topics at the interface of cosmology, field theory, and gravitation: observational constraints on alternative cosmologies, large extra dimensions of spacetime, supersymmetric topological defects, violations of fundamental symmetries, the origin of the matter/antimatter asymmetry, the connection between cosmology and the arrow of time. I found the last of these especially intriguing. To physicists, all of the manifold ways in which the past is different from the future (we age toward the future, we can remember the past, we can make choices toward the future) ultimately come back to the celebrated Second Law of Thermodynamics: in closed systems, entropy tends to increase over time. Back in the 19th century, Ludwig Boltzmann and others explained why entropy increases toward the future; what remains as a problem is why the entropy was ever so low in the past. That’s a question for cosmology, and presents a significant challenge to current models of the early universe. With graduate student Jennifer Chen, I proposed a novel scenario in which the Big Bang is not the beginning of the universe, but simply one event among many; in the larger multiverse, entropy increases without bound both toward the distant future and also in the very distant past [Carroll and Chen 2004, 2005]. Our picture was speculative, to say the least, but it serves as a paradigmatic example of attempts to find a purely dynamical basis for the Second Law, and continues to attract attention from both physicists and philosophers.

In May, 2005, I was informed that I had been denied tenure. This came as a complete shock, in part because I had been given no warning that any trouble was brewing. I will never know precisely what was said at the relevant faculty meetings, and the explanations I received from different colleagues were notable mostly for the lack of any consistent narrative. But one thing that came through clearly was that my interest in doing things other than research had counted substantially against me. I was told that I came across as “more interested in writing textbooks,” and that perhaps I would be happier at a university that placed a “greater emphasis on pedagogy.”

An experience like that cannot help but inspire some self-examination, and I thought hard about what my next steps should be. I recognized that, if I wanted to continue in academia, my best chance of being considered successful would be to focus my energies as intently as possible in a single area of research, and cut down non-research activities to a minimum.

After a great deal of contemplation, I decided that such a strategy was exactly what I didn’t want to do. I would remain true to my own intellectual passions, and let the chips fall where they may.

Caltech and Beyond: 2006-

After the Chicago decision I was again very fortunate, when the physics department at Caltech quickly offered me a position as a research faculty member. It was a great opportunity, offering both a topflight research environment and an extraordinary amount of personal freedom. I took the job with two goals in mind: to expand my outreach and non-academic efforts even further, and to do innovative interdisciplinary research that would represent a true and lasting contribution.

To be brutally honest, since I arrived here in 2006 I have been much more successful at the former than at the latter (although I feel this is beginning to change). I’ve written two popular-level books: From Eternity to Here, on cosmology and the arrow of time, and The Particle at the End of the Universe, on the search for the Higgs boson at the Large Hadron Collider. Both were well-received, with Particle winning the Winton Prize from the Royal Society, the world’s most prestigious award for popular science books. I have produced two lecture courses for The Teaching Company, given countless public talks, and appeared on numerous TV programs, up to and including The Colbert Report. Living in Los Angeles, I’ve had the pleasure of serving as a science consultant on various films and TV shows, working with people such as Ron Howard, Kenneth Branagh, and Ridley Scott. My talk from TEDxCaltech, “Distant time and the hint of a multiverse,” recently passed a million total views. I helped organize a major interdisciplinary conference on the nature of time, as well as a much smaller workshop on philosophical naturalism that attracted some of the best people in the field (such as Steven Weinberg, Daniel Dennett, and Richard Dawkins). I was elected a Fellow of the American Physical Society and won the Gemant Award from the American Institute of Physics.

More substantively, I’ve developed my longstanding interest in philosophy in productive directions. Some of the physics questions that I find most interesting, such as the arrow of time or the measurement problem in quantum mechanics, are ones where philosophers have made a significant impact, and I have begun interacting and collaborating with several of the best in the business. In recent years the subject called “philosophy of cosmology” has become a new and exciting field, and I’ve had the pleasure of being at the center of many activities in the area; a conference next month has set aside a discussion session to examine the implications of the approach to the arrow of time that Jennifer Chen and I put forward a decade ago. My first major work in philosophy of science, a paper with graduate student Charles Sebens on how to derive the Born Rule in the many-worlds approach to quantum mechanics, was recently accepted into one of the leading journals in the field [Sebens and Carroll 2014]. I’ve also published invited articles on the implications of modern cosmology for religion, and participated in a number of popular debates on naturalism vs. theism.

At the same time, my research efforts have been productive but somewhat meandering. As usual, I have worked on a variety of interesting topics, including the use of effective field theory to understand the growth of large-scale structure, the dynamics of Lorentz-violating “aether” fields, how new forces can interact with dark matter, black hole entropy, novel approaches to dark-matter abundance, cosmological implications of a decaying Higgs field, and the role of rare fluctuations in the long-term evolution of universe. Some of my work over these years includes papers of which I am quite proud; these include investigations of dynamical compactification of dimensions of space [Carroll, Johnson, and Randall 2009], possible preferred directions in the universe [Ackerman, Carroll, and Wise 2007; Erickcek, Kamionkowski, and Carroll 2008a, b], the prospect of a force similar to electromagnetism interacting with dark matter [Ackerman et al. 2008], and quantitative investigations of fine- tuning of cosmological evolution [Carroll and Tam 2010; Remmen and Carroll 2013, 2014; Carroll 2014]. Almost none of this work has been on my previous specialty, dark energy and the accelerating universe. After having put a great amount of effort into thinking about this (undoubtedly important) problem, I have become pessimistic about the prospect for an imminent theoretical breakthrough, at least until we have a better understanding of the basic principles of quantum gravity. This helps explain the disjointed nature of my research over the past few years, but has also driven home to me the need to find a new direction and tackle it with determination.

Very recently I’ve found such a focus, and in some sense I have finally started to do the research I was born to do. It has resulted from a confluence of my interests in cosmology, quantum mechanics, and philosophy, along with a curiosity about complexity theory that I have long nurtured but never really acted upon. This is the turn toward “emergence” that I mentioned at the beginning of this narrative, and elaborate on in my research plan. I go into greater detail there, but the basic point is that we need to construct a more reliable framework in which to connect the very foundations of physics – quantum mechanics, field theory, spacetime – to a multitude of higher-level phenomena, from statistical mechanics to organized structures. A substantial amount of work has already been put into such issues, but a number of very basic questions remain unanswered.

This represents an evolution of my research focus rather than a sudden break with my earlier work; many topics in cosmology and quantum gravity are intimately tied to issues of emergence, and I’ve already begun investigating some of these questions in different ways. One prominent theme is the emergence of the classical world out of an underlying quantum description. My papers with Sebens on the many-worlds approach are complementary to a recent paper I wrote with two graduate students on the nature of quantum fluctuations [Boddy, Carroll, and Pollack 2014]. There, we argued that configurations don’t actually “fluctuate into existence” in stationary quantum states, since there is no process of decoherence; this has important implications for cosmology in both the early and late universe. In another paper [Aaronson, Carroll, and Ouellette 2014], my collaborators and I investigated the relationship between entropy (which always increases in closed systems) and complexity (which first increases, then decreases as the system approaches equilibrium). Since the very notion of complexity does not have a universally-agreed-upon definition, any progress we can make in understanding its basic features is potentially very important.

I am optimistic that this new research direction will continue to expand and flourish, and that there is a substantial possibility of making important breakthroughs in the field. (My papers on the Born Rule and quantum fluctuations have already attracted considerable attention from influential physicists and philosophers – they don’t always agree with our unconventional conclusions, but I choose to believe that it’s just a matter of time.) I am diving into these new waters headfirst, including taking online courses (complexity theory from Santa Fe, programming and computer science from MIT) that will help me add skills that weren’t part of my education as a cosmologist. A Guggenheim Fellowship will be invaluable in aiding me in this effort.

My ten-year-old self was right: there is nothing more exciting than trying to figure out how nature works at a deep level. Having hit upon a promising new way of doing it, I can’t wait to see where it goes.