Science

This year we give thanks for space. (We’ve previously given thanks for the Standard Model Lagrangian, Hubble’s Law, the Spin-Statistics Theorem, conservation of momentum, effective field theory, the error bar, gauge symmetry, Landauer’s Principle, the Fourier Transform, Riemannian Geometry, the speed of light, the Jarzynski equality, and the moons of Jupiter.)

Even when we restrict to essentially scientific contexts, “space” can have a number of meanings. In a tangible sense, it can mean outer space — the final frontier, that place we could go away from the Earth, where the stars and other planets are located. In a much more abstract setting, mathematicians use “space” to mean some kind of set with additional structure, like Hilbert space or the space of all maps between two manifolds. Here we’re aiming in between, using “space” to mean the three-dimensional manifold in which physical objects are located, at least as far as our observable universe is concerned.

That last clause reminds us that there are some complications here. The three dimensions we see of space might not be all there are; extra dimensions could be hidden from us by being curled up into tiny balls (or generalizations thereof) that are too small to see, or if the known particles and forces are confined to a three-dimensional brane embedded in a larger universe. On the other side, we have intimations that quantum theories of gravity imply the holographic principle, according to which an N-dimensional universe can be thought of as arising as a projection of (N-1)-dimensions worth of information. And much less speculatively, Einstein and Minkowski taught us long ago that three-dimensional space is better thought of as part of four-dimensional spacetime.

Let’s put all of that aside. Our everyday world is accurately modeled as stuff, distributed through three-dimensional space, evolving with time. That’s something to be thankful for! But we can also wonder why it is the case.

I don’t mean “Why is space three-dimensional?”, although there is that. I mean why is there something called “space” at all? I recently gave an informal seminar on this at Columbia, and I talk about it a bit in Something Deeply Hidden, and it’s related in spirit to a question Michael Nielsen recently asked on Twitter, “Why does F=ma?”

Space is probably emergent rather than fundamental, and the ultimate answer to why it exists is probably going to involve quantum mechanics, and perhaps quantum gravity in particular. The right question is “Why does the wave function of the universe admit a description as a set of branching semi-classical worlds, each of which feature objects evolving in three-dimensional space?” We’re working on that!

But rather than answer it, for the purposes of thankfulness I just want to point out that it’s not obvious that space as we know it had to exist, even if classical mechanics had been the correct theory of the world.

Newton himself thought of space as absolute and fundamental. His ontology, as the philosophers would put it, featured objects located in space, evolving with time. Each object has a trajectory, which is its position in space at each moment of time. Quantities like “velocity” and “acceleration” are important, but they’re not fundamental — they are derived from spatial position, as the first and second derivatives with respect to time, respectively.

But that’s not the only way to do classical mechanics, and in some sense it’s not the most basic and powerful way. An alternative formulation is provided by Hamiltonian mechanics, where the fundamental variable isn’t “position,” but the combination of “position and momentum,” which together describe the phase space of a system. The state of a system at any one time is given by a point in phase space. There is a function of phase space cleverly called the Hamiltonian H(x,p), from which the dynamical equations of the system can be derived.

That might seem a little weird, and students tend to be somewhat puzzled by the underlying idea of Hamiltonian mechanics when they are first exposed to it. Momentum, we are initially taught in our physics courses, is just the mass times the velocity. So it seems like a derived quantity, not a fundamental one. How can Hamiltonian mechanics put momentum on an equal footing to position, if one is derived from the other?

The answer is that in the Hamiltonian approach, momentum is not defined to be mass times velocity. It ends up being equal to that by virtue of an equation of motion, at least if the Hamiltonian takes the right form. But in principle it’s an independent variable.

That’s a subtle distinction! Hamiltonian mechanics says that at any one moment a system is described by two quantities, its position and its momentum. No time derivatives or trajectories are involved; position and momentum are completely different things. Then there are two equations telling us how the position and the momentum change with time. The derivative of the position is the velocity, and one equation sets it equal to the momentum divided by the mass, just as in Newtonian mechanics. The other equation sets the derivative of the momentum equal to the force. Combining the two, we again find that force equals mass times acceleration (derivative of velocity).

So from the Hamiltonian perspective, positions and momenta are on a pretty equal footing. Why then, in the real world, do we seem to “live in position space”? Why don’t we live in momentum space?

As far as I know, no complete and rigorous answer to these questions has ever been given. But we do have some clues, and the basic principle is understood, even if some details remain to be ironed out.

That principle is this: we can divide the world into subsystems that interact with each other under appropriate circumstances. And those circumstances come down to “when they are nearby in space.” In other words, interactions are local in space. They are not local in momentum. Two billiard balls can bump into each other when they arrive at the same location, but nothing special happens when they have the same momentum or anything like that.

Ultimately this can be traced to the fact that the Hamiltonian of the real world is not some arbitrary function of positions and momenta; it’s a very specific kind of function. The ultimate expression of this kind of locality is field theory — space is suffused with fields, and what happens to a field at one point in space only directly depends on the other fields at precisely the same point in space, nowhere else. And that’s embedded in the Hamiltonian of the universe, in particular in the fact that the Hamiltonian can be written as an integral over three-dimensional space of a local function, called the “Hamiltonian density.”

where φ is the field (which here acts as a “coordinate”) and π is its corresponding momentum.

This represents progress on the “Why is there space?” question. The answer is “Because space is the set of variables with respect to which interactions are local.” Which raises another question, of course: why are interactions local with respect to anything? Why do the fundamental degrees of freedom of nature arrange themselves into this kind of very specific structure, rather than some other one?

We have some guesses there, too. One of my favorite recent papers is “Locality From the Spectrum,” by Jordan Cotler, Geoffrey Penington, and Daniel Ranard. By “the spectrum” they mean the set of energy eigenvalues of a quantum Hamiltonian — i.e. the possible energies that states of definite energy can have in a theory. The game they play is to divide up the Hilbert space of quantum states into subsystems, and then ask whether a certain list of energies is compatible with “local” interactions between those subsystems. The answers are that most Hamiltonians aren’t compatible with locality in any sense, and for those where locality is possible, the division into local subsystems is essentially unique. So locality might just be a consequence of certain properties of the quantum Hamiltonian that governs the universe.

Fine, but why that Hamiltonian? Who knows? This is above our pay grade right now, though it’s fun to speculate. Meanwhile, let’s be thankful that the fundamental laws of physics allow us to describe our everyday world as a collection of stuff distributed through space. If they didn’t, how would we ever find our keys?

Note: It is in the nature of book-writing that sometimes you write things that don’t end up appearing in the final book. I had a few such examples for Something Deeply Hidden, my book on quantum mechanics, Many-Worlds, and emergent spacetime. Most were small and nobody will really miss them, but I did feel bad about eliminating my discussion of the “delayed-choice quantum eraser,” an experiment that has caused no end of confusion. So here it is, presented in full. It’s a bit too technical for the book, I don’t know what I was thinking!

Let’s imagine you’re an undergraduate physics student, taking an experimental lab course, and your professor is in a particularly ornery mood. So she forces you to do a weird version of the double-slit experiment, explaining that this is something called the “delayed-choice quantum eraser.” You think you remember seeing a YouTube video about this once.

In the conventional double-slit, we send a beam of electrons through two slits and on toward a detecting screen. Each individual electron hits the screen and leaves a dot, but if we build up many such detections, we see an interference pattern of light and dark bands, because the wave function passing through the two slits interferes with itself. But if we also measure which slit each electron goes through, the interference pattern disappears, and we see a smoothed-out distribution at the screen. According to textbook quantum mechanics that’s because the wave function collapsed when we measured it at the slits; according to Many-Worlds it’s because the electron became entangled with the measurement apparatus, decoherence occurred as the apparatus became entangled with the environment, and the wave function branched into separate worlds, in each of which the electron only passes through one of the slits.

The new wrinkle is that we are still going to “measure” which slit the electron goes through, but instead of reading it out on a big macroscopic dial, we simply store that information in a single qubit. Say that for every “traveling” electron passing through the slits, we have a separate “recording” electron. The pair becomes entangled in the following way: if the traveling electron goes through the left slit, the recording electron is in a spin-up state (with respect to the vertical axis), and if the traveling electron goes through the right, the recording electron is spin-down. We end up with:

Ψ = (L)[↑] + (R)[↓].

Our professor, who is clearly in a bad mood, insists that we don’t actually measure the spin of our recording electrons, and we don’t even let them wander off and bump into other things in the room. We carefully trap them and preserve them, perhaps in a magnetic field.

What do we see at the screen when we do this with many electrons? A smoothed-out distribution with no interference pattern, of course. Interference can only happen when two things contribute to exactly the same wave function, and since the two paths for the traveling electrons are now entangled with the recording electrons, the left and right paths are distinguishable, so we don’t see any interference pattern. In this case it doesn’t matter that we didn’t have honest decoherence; it just matters that the traveling electrons were entangled with the recording electrons. Entanglement of any sort kills interference.

Of course, we could measure the recording spin if we wanted to. If we measure it along the vertical axis, we will see either [↑] or [↓]. Referring back to the quantum state Ψ above, we see that this will put us in either a universe where the traveling electron went through the left slit, or one where it went through the right slit. At the end of the day, recording the positions of many such electrons when they hit the detection screen, we won’t see any interference.

Okay, says our somewhat sadistic professor, rubbing her hands together with villainous glee. Now let’s measure all of our recording spins, but this time measure them along the horizontal axis instead of the vertical one. As we saw in Chapter Four, there’s a relationship between the horizontal and vertical spin states; we can write

[↑] = [→] + [←] ,

[↓] = [→] – [←].

(To keep our notation simple we’re ignoring various factors of the square root of two.) So the state before we do such a measurement is

Ψ = (L)[→] + (L)[←] + (R)[→] – (R)[←]

= (L + R)[→] + (L – R)[←].

When we measured the recording spin in the vertical direction, the result we obtained was entangled with a definite path for the traveling electron: [↑] was entangled with (L), and [↓] was entangled with (R). So by performing that measurement, we knew that the electron had traveled through one slit or the other. But now when we measure the recording spin along the horizontal axis, that’s no longer true. After we do each measurement, we are again in a branch of the wave function where the traveling electron passes through both slits. If we measured spin-left, the traveling electron passing through the right slit picks up a minus sign in its contribution to the wave function, but that’s just math.

By choosing to do our measurement in this way, we have erased the information about which slit the electron went through. This is therefore known as a “quantum eraser experiment.” This erasure doesn’t affect the overall distribution of flashes on the detector screen. It remains smooth and interference-free.

But we not only have the overall distribution of electrons hitting the detector screen; for each impact we know whether the recording electron was measured as spin-left or spin- right. So, instructs our professor with a flourish, let’s go to our computers and separate the flashes on the detector screen into these two groups — those that are associated with spin- left recording electrons, and those that are associated with spin-right. What do we see now?

Interestingly, the interference pattern reappears. The traveling electrons associated with spin-left recording electrons form an interference pattern, as do the ones associated with spin-right. (Remember that we don’t see the pattern all at once, it appears gradually as we detect many individual flashes.) But the two interference patterns are slightly shifted from each other, so that the peaks in one match up with the valleys in the other. There was secretly interference hidden in what initially looked like a featureless smudge.

In retrospect this isn’t that surprising. From looking at how our quantum state Ψ was written with respect to the spin-left and -right recording electrons, each measurement was entangled with a traveling electron going through both slits, so of course it could interfere. And that innocent-seeming minus sign shifted one of the patterns just a bit, so that when combined together the two patterns could add up to a smooth distribution.

You professor seems more amazed by this than you are. “Don’t you see,” she exclaims excitedly. “If we didn’t measure the recording photons at all, or if we measured them along the vertical axis, there was no interference anywhere. But if we measured them along the horizontal axis, there secretly was interference, which we could discover by separating out what happens at the screen when the recording spin was left or right.”

You and your classmates nod their heads, cautiously but with some degree of confusion.

“Think about what that means! The choice about whether to measure our recording spins vertically or horizontally could have been made long after the traveling photons splashed on the recording screen. As long as we stored our recording spins carefully and protected them from becoming entangled with the environment, we could have delayed that choice until years later.”

Sure, the class mumbles to themselves. That sounds right.

“But interference only happens when the traveling electron goes through both slits, and the smooth distribution happens when it goes through only one slit. That decision — go through both slits, or just through one — happens long before we measure the recording electrons! So obviously, our choice to measure them horizontally rather than vertically had to send a signal backward in time to tell the traveling electrons to go through both slits rather than just one!”

After a short, befuddled pause, the class erupts with objections. Decisions? Backwards in time? What are we talking about? The electron doesn’t make a choice to travel through one slit or the other. Its wave function (and that of whatever it’s entangled with) evolves according to the Schrödinger equation, just like always. The electron doesn’t make choices, it unambiguously goes through both slits, but it becomes entangled along the way. By measuring the recording photons along different directions, we can pick out different parts of that entangled wave function, some of which exhibit interference and others do not. Nothing really went backwards in time. It’s kind of a cool result, but it’s not like we’re building a frickin’ time machine here.

You and your classmates are right. Your instructor has gotten a little carried away. There’s a temptation, reinforced by the Copenhagen interpretation, to think of an electron as something “with both wave-like and particle-like properties.” If we give into that temptation, it’s a short journey to thinking that the electron must behave in either a wave- like way or a particle-like way when it passes through the slits, and in any given experiment it will be one or the other. And from there, the delayed-choice experiment does indeed tend to suggest that information had to go backwards in time to help the electron make its decision. And, to be honest, there is a tradition in popular treatments of quantum mechanics to make things seem as mysterious as possible. Suggesting that time travel might be involved somehow is just throwing gasoline on the fire.

All of these temptations should be resisted. The electron is simply part of the wave function of the universe. It doesn’t make choices about whether to be wave-like or particle-like. But a number of serious researchers in quantum foundations really do take the delayed-choice quantum eraser and analogous experiments (which have been successfully performed, by the way) as evidence of retrocausality in nature — signals traveling backwards in time to influence the past. A form of this experiment was originally proposed by none other than John Wheeler, who envisioned a set of telescopes placed on the opposite side of the screen from the slits, which could detect which slit the electrons went through long after they had passed through. Unlike some later commentators, Wheeler didn’t go so far as to suggest retrocausality, and knew better than to insist that an electron is either a particle or a wave at all times.

There’s no need to invoke retrocausality to explain the delayed-choice experiment. To an Everettian, the result makes perfect sense without anything traveling backwards in time. The trickiness relies on the fact that by becoming entangled with a single recording spin rather than with the environment and its zillions of particles, the traveling electrons only became kind-of decohered. With just a single particle to worry about observing, we are allowed to contemplate measuring it in different ways. If, as in the conventional double- slit setup, we measured the slit through which the traveling electron went via a macroscopic pointing device, we would have had no choice about what was being observed. True decoherence takes a tiny quantum entanglement and amplifies it, effectively irreversibly, into the environment. In that sense the delayed-choice quantum eraser is a useful thought experiment to contemplate the role of decoherence and the environment in measurement.

But alas, not everyone is an Everettian. In some other versions of quantum mechanics, wave functions really do collapse, not just the apparent collapse that decoherence provides us with in Many-Worlds. In a true collapse theory like GRW, the process of wave- function collapse is asymmetric in time; wave functions collapse, but they don’t un- collapse. If you have collapsing wave functions, but for some reason also want to maintain an overall time-symmetry to the fundamental laws of physics, you can convince yourself that retrocausality needs to be part of the story.

Or you can accept the smooth evolution of the wave function, with branching rather than collapses, and maintain time-symmetry of the underlying equations without requiring backwards-propagating signals or electrons that can’t make up their mind.

Hard to believe it’s been 15 years since the publication of Spacetime and Geometry: An Introduction to General Relativity, my graduate-level textbook on everyone’s favorite theory of gravititation. The book has become quite popular, being used as a text in courses around the world. There are a lot of great GR books out there, but I felt another one was needed that focused solely on the idea of “teach students general relativity.” That might seem like an obvious goal, but many books also try to serve as reference books, or to put forward a particular idiosyncratic take on the subject. All I want to do is to teach you GR.

And now I’m pleased to announce that the book is changing publishers, from Pearson to Cambridge University Press. Even with a new cover, shown above.

I must rush to note that it’s exactly the same book, just with a different publisher. Pearson was always good to me, I have no complaints there, but they are moving away from graduate physics texts, so it made sense to try to find S&G a safe permanent home.

Well, there is one change: it’s cheaper! You can order the book either from CUP directly, or from other outlets such as Amazon. Copies had been going for roughly $100, but the new version lists for only $65 — and if the Amazon page is to be believed, it’s currently on sale for an amazing $46. That’s a lot of knowledge for a minuscule price. I’d rush to snap up copies for you and your friends, if I were you.

My understanding is that copies of the new version are not quite in stores yet, but they’re being printed and should be there momentarily. Plenty of time for courses being taught this Fall. (Apologies to anyone who has been looking for the book over the past couple of months, when it’s been stuck between publishers while we did the handover.)

Again: it’s precisely the same book. I have thought about doing revisions to produce an actually new edition, but I think about many things, and that’s not a super-high priority right now. Maybe some day.

Thanks to everyone who has purchased Spacetime and Geometry over the years, and said such nice things about it. Here’s to the next generation!

I talked a bit on Twitter last night about the Past Hypothesis and the low entropy of the early universe. Responses reminded me that there are still some significant misconceptions about the universe (and the state of our knowledge thereof) lurking out there. So I’ve decided to quickly list, in Tweet-length form, some true facts about cosmology that might serve as a useful corrective. I’m also putting the list on Twitter itself, and you can see comments there as well.

1. The Big Bang model is simply the idea that our universe expanded and cooled from a hot, dense, earlier state. We have overwhelming evidence that it is true.
2. The Big Bang event is not a point in space, but a moment in time: a singularity of infinite density and curvature. It is completely hypothetical, and probably not even strictly true. (It’s a classical prediction, ignoring quantum mechanics.)
3. People sometimes also use “the Big Bang” as shorthand for “the hot, dense state approximately 14 billion years ago.” I do that all the time. That’s fine, as long as it’s clear what you’re referring to.
4. The Big Bang might have been the beginning of the universe. Or it might not have been; there could have been space and time before the Big Bang. We don’t really know.
5. Even if the BB was the beginning, the universe didn’t “pop into existence.” You can’t “pop” before time itself exists. It’s better to simply say “the Big Bang was the first moment of time.” (If it was, which we don’t know for sure.)
6. The Borde-Guth-Vilenkin theorem says that, under some assumptions, spacetime had a singularity in the past. But it only refers to classical spacetime, so says nothing definitive about the real world.
7. The universe did not come into existence “because the quantum vacuum is unstable.” It’s not clear that this particular “Why?” question has any answer, but that’s not it.
8. If the universe did have an earliest moment, it doesn’t violate conservation of energy. When you take gravity into account, the total energy of any closed universe is exactly zero.
9. The energy of non-gravitational “stuff” (particles, fields, etc.) is not conserved as the universe expands. You can try to balance the books by including gravity, but it’s not straightforward.
10. The universe isn’t expanding “into” anything, as far as we know. General relativity describes the intrinsic geometry of spacetime, which can get bigger without anything outside.
11. Inflation, the idea that the universe underwent super-accelerated expansion at early times, may or may not be correct; we don’t know. I’d give it a 50% chance, lower than many cosmologists but higher than some.
12. The early universe had a low entropy. It looks like a thermal gas, but that’s only high-entropy if we ignore gravity. A truly high-entropy Big Bang would have been extremely lumpy, not smooth.
13. Dark matter exists. Anisotropies in the cosmic microwave background establish beyond reasonable doubt the existence of a gravitational pull in a direction other than where ordinary matter is located.
14. We haven’t directly detected dark matter yet, but most of our efforts have been focused on Weakly Interacting Massive Particles. There are many other candidates we don’t yet have the technology to look for. Patience.
15. Dark energy may not exist; it’s conceivable that the acceleration of the universe is caused by modified gravity instead. But the dark-energy idea is simpler and a more natural fit to the data.
16. Dark energy is not a new force; it’s a new substance. The force causing the universe to accelerate is gravity.
17. We have a perfectly good, and likely correct, idea of what dark energy might be: vacuum energy, a.k.a. the cosmological constant. An energy inherent in space itself. But we’re not sure.
18. We don’t know why the vacuum energy is much smaller than naive estimates would predict. That’s a real puzzle.
19. Neither dark matter nor dark energy are anything like the nineteenth-century idea of the aether.

Feel free to leave suggestions for more misconceptions. If they’re ones that I think many people actually have, I might add them to the list.