Michael Nielsen – How science actually progresses

Today, I'm speaking with Michael Nielsen. You have done many things. You're one of the pioneers of

quantum computing, wrote the main textbook in the field of the open science movement.

You wrote a book about deep learning that Chris Olah and Greg Brockman

credit with getting them into the field. More recently, you're a research fellow

at the Astera Institute and writing a book about religion, science, and technology.

I'm going to ask you about none of those things. The conversation I want to have today is,

how do we recognize scientific progress? It's especially relevant for AI because

people are trying to close the RL verification loop on scientific discovery.

What does it mean to close that loop? But in preparing for this interview,

I've realized that it's a more mysterious and elusive force,

even in the history of human science, than I understood.

I think a good place to start will be Michelson-Morley and how special relativity

is discovered, if it's different from the story that you get off of YouTube videos.

I will prompt you that way, and then we'll go in there.

Michelson-Morley is the famous result often presented as this experiment that was done in

the 1880s that helped Einstein come up with the special theory of relativity a little bit later,

changing the way we think about space and time and our fundamental conception of those things.

And there's a big gap, I think, between the way Michelson and Morley and other people

at the time thought about the experiment and certainly the way in which Einstein thought

or did not think about the experiment. In actual fact, he stated later in his

life he wasn't even sure whether he was aware of the paper at the time.

There's a lot of evidence that he probably was aware of the paper at the time, but it actually

wasn't dispositive for his thinking at all. Something else completely was going on.

What Michelson and Morley thought they were doing was testing different theories

of what was called the ether. If you go back to the 1600s,

Robert Boyle introduced the idea of the ether. We know that sound is vibrations in the air.

Boyle and other people got interested in the question of

whether light is vibrations in something, and they couldn't figure out what it was.

Boyle did an experiment where he tested whether you could propagate

light through a vacuum. He found that you could. You couldn't do it with sound.

He introduced this idea of the ether, and for the next two hundred or so years,

people had all these conversations about what the ether was and what its nature was.

The Michelson and Morley experiment was really an experiment to test different theories of the ether

against one another, in particular to find out whether or not there was a so-called ether wind.

The idea was that the Earth is maybe passing through this ether wind.

And if it is passing through the ether wind and you shoot a light beam parallel

to the direction the ether wind is going in, it'll get accelerated a little bit.

If it's being passed back in the opposite direction, it'll get slowed down a little bit,

and you should be able to see this in the results of interference experiments.

What they found, much to their surprise, was that in fact there was no ether wind.

That ruled out some theories of the ether, but not all, and Michelson

certainly continued to believe in the ether. This is what was a shocking part of reading

this story from the biography of Einstein that you recommended by... what was his first name?

Abraham Pais. Abraham Pais.

Subtle is the Lord. Also from Imre Lakatos, The Methodology of Scientific Research Programmes.

The way it's told is that Michelson-Morley proved that the ether did not exist.

Therefore, it created a crisis in physics that Einstein solved with special relativity.

What you're pointing out is he actually was trying to distinguish

between many different theories of ether. If you're in space or if you're on Earth,

it's the same direction of ether, or maybe the ether wind is being carried around by the Earth,

and so you can't really experience it on Earth. But if you go to a high enough altitude,

you might be able to experience it. In fact, Michelson's experiments,

the famous one is 1887, but he conducted these experiments for basically two decades.

For longer than that. He conducted the first one in 1881, I think,

but he continued to believe until he died. He died, I think it was 1929 or so. It was the

late twenties. He was still doing experiments in the 1920s about whether or not the ether existed.

So he continued to believe in the ether to the end of his life.

I think the last public statement he made was a year or two before he died,

and he basically still believed it at that point. In fact, there was another physicist, Miller,

who kept doing these experiments in the 1920s. He thought that if he went to a high enough

altitude, Mount Wilson in California… "Oh, I'm high enough that the ether

winds are not being dragged by the Earth. And I've measured the effect of the ether."

Einstein hears about this and he says, and this is where you get the famous quote,

"Subtle is the Lord, but malicious He is not." Anyways, I think the reason the story is

interesting is for many different reasons. One of the ways in which the real history of

science is different from this idea you get of the scientific method is that you really can't apply

falsification as easily as you might think. It's not clear what is being falsified.

Is it just another version of the theory of the ether that's being falsified?

Certainly you can't induce the theory of special relativity from the fact

that one version of the ether seems to be disconfirmed by these experiments.

It certainly doesn't show that ideas about falsification are wrong or falsified,

but it does show that the most naive ideas… Things are often much more complicated than you think.

Michelson did this experiment in 1881. He was a very young man, and then other people,

I think Rayleigh was one of them, pointed out that there were some problems with the way he did it,

so they had to redo it in 1887. At that point, a lot of the leading

physicists of the day basically accepted this result, that there was no ether wind.

But what to do about this? Sure, maybe you falsified

some theories of the ether. There are others that you

haven't falsified at all at this point, and people set to work on developing those.

It is funny, people will phrase it as showing that the ether didn't exist.

Even just the word "the" there is a misnomer. You actually had a ton of different theories

and a couple of leading contenders. So yes, there's some version of

falsification going on, but how you respond to this new experiment is very complicated.

Certainly the leading physicists of the day responded by saying, "Okay, this gives us a

lot of information about what the ether must be, but it doesn't tell us that there is no ether."

In fact, Lorentz at the end of the 19th century, before Einstein,

figures out the math of how you convert from one reference frame to another reference frame,

and comes up with the Lorentz transformations, which is the basis of special relativity.

But his interpretation is that you are converting from the ether reference

frame to these non-privileged other reference frames if you're moving relative to the ether.

His interpretation of length contraction and time dilation is that this is the effect of moving

through the ether, and you have this pressure. This pressure is warping clocks. It's warping

measures of length. The interesting thing here is that experimentally you cannot distinguish

Lorentz's interpretation from special relativity. I think that's a strong statement.

Lorentz introduces this quantity called local time, which he regards as...

My understanding is he's not trying to give a physical interpretation of this,

but it's what Einstein would later just recognize as time in another inertial reference frame.

He's not trying to attribute much physical meaning to it.

I think Poincaré gets much closer later on to realizing that this

is the time that's registered by clocks. About forty-odd years later, people start

doing these muon experiments where they see cosmic rays hit the top of the atmosphere.

They produce a shower of muons, and you can look to see at different heights in the atmosphere how

many of those muons remain. They decay over time,

and a very strange thing happens, which is that they're decaying way too slow.

You expect they shouldn't be able to last the whole way through the atmosphere at all.

Their decay rate is too quick, if you were in a classical theory.

But if in fact their time really has slowed down, it's okay.

In fact, the measured decay rates in 1940—and there have since been more

accurate experiments done—match exactly what you expect from special relativity.

That's the kind of thing where if Lorentz had been alive—he'd been dead ten or so years at that

point—it seems quite likely that he would have tried to save his theory by patching it up yet

again, but it would have been a massive setback. It starts to just look like time—this

thing that Lorentz introduced as a mathematical convenience—that's

actually what time is, for the muons at least. Then there's a whole bunch of other experiments

that show this very similar phenomenon. When was that experiment done?

That was, I think, 1940. It might have been published in 1941.

Maybe to rephrase and change my claim: it's not that you could not have distinguished them,

but the scientific community adopted what we in retrospect consider the more correct

interpretation before it was actually experimentally shown to be preferred.

So there's clearly some process that human science does which can distinguish different theories.

Can I just interrupt? You used the word process, and it's interesting to think about that term.

Process carries connotations of something set in advance.

It's much more complicated in practice. You have people like Lorentz, who Einstein

absolutely and utterly admired, and Poincaré, one of the greatest scientists who ever lived,

and Michelson, another truly outstanding scientist, who never reconciled themselves.

It's not as though there's some standard procedure that we're all using to reconcile these things.

Great scientists can remain wrong for a very long time after the scientific community has

broadly changed its opinion. But there's no centralized

authority or centralized method. That is the interesting thing. There's

progress even though it is hard to articulate the process by which it happens, the heuristics that

are used. You mentioned Poincaré. Lorentz has the math right, but the interpretation wrong.

It seems like Poincaré had the opposite, where he understood that it's hard to define simultaneity

because it requires a circular definition with time, or velocity of something that might arrive

at a midpoint together, but velocity is defined in terms of time. I find this interesting. There are

a couple of other examples we could call on. There is this phenomenon in the history of

science where somebody asks the right question, but then they don't clinch it.

I'm curious what you think is happening in those cases.

You actually do want to go case by case and try to understand.

It's not necessarily clear that they're doing the same thing wrong in all of the

cases. The Poincaré case is amazing. He seems to have understood the principle of relativity,

the idea that the laws of physics are the same in all inertial reference frames.

He seems to have understood that the speed of light is

the same in all inertial reference frames. He doesn't phrase it quite that way, but it is

my understanding, though I don't speak French. These are basically the ideas that Einstein

uses to deduce special relativity. But then he also has this additional

misunderstanding where he thinks that length contraction is a dynamical effect, that somehow

particles are being pushed together by some external force, something is going on dynamically.

He doesn't understand that it's purely kinematics. That actually space and time are different from

what we thought, and you need to fundamentally rethink those things.

It's almost like he knew too much. He had almost too grand a vision in mind.

Einstein subtracts from that and says, "No. Space and time are just different than what we

thought, and here's the correct picture." There's a paper in, I think it's 1909,

where Poincaré still has this dynamical picture of what's going on with the length contraction.

This is just not necessary. This is a mistake from the modern point of view. Why is he doing

this? Why is he clinging onto this idea? I don't know. I've obviously never met the man.

It would be fascinating to be able to talk it over and try and understand.

His expertise seems to be getting in the way. He knows so much, he understands so much,

and then he's not able to let go of these things. A really interesting fact is that a few years

prior, in the 1890s, Einstein's a teenager and he believes in the ether too. He knows

about this stuff. But he's not quite as attached as these older people were.

Maybe they were a little bit prisoners of their own expertise. That's my guess. Some

historians of science would certainly disagree. Then there's the obvious stories where Einstein

himself later on is said to have not latched onto the correct interpretations of quantum mechanics

or cosmology because of his own attachments. Yeah.

Here’s the bigger question I have. The muon example is a great example of

these long verification loops and how progress seems to happen in the scientific community

faster than these verification loops imply. Maybe the clearest example is Aristarchus

in the second century BC comes up with the idea of heliocentrism.

The ancient Athenians dismiss it on the grounds that we should see as the Earth

is moving around the Sun, if really the Sun is the center of the solar system,

the stars move relative to the Earth. The only reason that would not be the

case is the stars are so far away that you would not observe this.

And it's only in 1838 that stellar parallax was actually measured.

And so, we didn't need to wait until 1838 to have heliocentrism.

We didn't need to wait for the experimental validation to

understand that Copernicus is better in some way. In fact, when Copernicus first came up

with his theories, it's well known that the Ptolemaic model was more accurate because it

had centuries of adding on these epicycles. What's maybe less well appreciated is that it

was also in some sense simpler. Because Copernicus actually

had to add extra epicycles. It had more epicycles than the

Ptolemaic model because he had this bias that the Earth should go in a perfect circle in equal time.

Anyway, I think this is an interesting story because it's not a more accurate theory. It's

not a simpler theory. So how could you have known ex ante

that Copernicus was correct and Ptolemy was not? Good question. I don't entirely know the answer.

I can give you a partial answer that I, centuries in the future, start to find very compelling.

I'm sure it's part of the historic story at least. One of the big shocks for Newton,

he did understand Kepler's laws of motion eventually, so you're able to explain the

motions of the planets in the sky. But he also, out of the same theory,

his theory of gravitation, was able to explain terrestrial motion.

He's able to explain why objects move in parabolas on the Earth, and he's able to explain

the tides in terms of the moon and the sun's gravitational effect on water on the Earth.

You have what seem like three very different disconnected phenomena all being explained by

this one set of ideas. That starts to feel

very compelling, at least to me. I think most people find that very

satisfying once they eventually realize it. Have you read the Keynes biography of Newton?

He wrote an entire biography? No, the essay.

Sure. I love that. This description of him as the last of the magicians is wonderful.

In fact, I think it's maybe worth superimposing. Or you should read out that one

passage of the thing. Alright. It's from a talk

that he gave at Cambridge not long before he died. He'd acquired Newton's papers somehow and gave a