Microlives & The Art of Uncertainty with Sir David Spiegelhalter

Rob Bell:
Hello, and welcome to Sketchplanations The Podcast, aiming to spread curiosity and contemplation about how the world works based on the huge collection of explanatory sketches found at sketchplanations.com.

I'm engineer and broadcaster Rob Bell, and with me is designer and creator of Sketchplanations, Jono Hey, and also on the podcast this week.

According to the British newspaper, The Telegraph, we have probably the UK's greatest living statistician, David Spiegelhalter.

Jono, David, hello.

Hello.

David, thank you so much for joining us on this episode.

David Spiegelhalter:
Yeah, it's fun.

It's great.

It's lovely.

I love the book.

I love the sketches.

So I'm really pleased to be part of this.

Rob Bell:
And I hope you don't mind me using that quote for The Telegraph.

It reminded me of the Carlsberg ad.

David Spiegelhalter:
Probably, probably the greatest statistician in the world.

Rob Bell:
But David, if you don't mind, I'd like to give our listeners a quick rundown of your work, so everybody knows who we're dealing with.

You are Emeritus Professor of Statistics at the Statistics Laboratory, University of Cambridge.

You've held a number of influential positions in your career around data and statistics, working tirelessly on ways to improve how statistical evidence is used by, amongst others, health professionals, patients, lawyers, judges, policy makers.

You're an award-winning documentary maker, best-selling author, and in 2014, you were knighted for services to medical statistics.

Yep.

There are so many fascinating things that we'd love to talk to you about.

We're going to get into how in your career you dissected data in order to understand risks and assess the chances of what might happen in the future.

Before all of that, what was it like doing the obstacle course on the TV show Wipeout?

David Spiegelhalter:
It was wonderful.

It is one of my life's great achievements.

Going out to Argentina for the weekend with a lovely group of 20 people.

The winner would come back with 10,000 quid.

Everyone else came back with all the broken bones, etc.

Rob Bell:
Yeah, it's brutal.

David Spiegelhalter:
It is brutal, but you're so padded up.

I don't know how they tested it, but you're so padded up that you can actually get whacked by something and fly through the air in a summer song, land in the water and you just come up.

Smiling, possibly.

Actually, I did really well, but I took it very seriously.

I was a big fan.

I watched all the episodes.

Then when I got on, which my daughter persuaded me to get on it, and that's another story of how I got on.

But then I realized that I did statistical analysis of all the times, which were on Wikipedia, and I realized that in order to get through the qualifier, really I had to get through in about three minutes.

And so I trained in three-minute bursts.

That's all I want to do.

If it's longer than that, you just end up sobbing in the mud or something.

I didn't want to be one of those, yeah.

Rob Bell:
That's fantastic.

So even using statistics to...

David Spiegelhalter:
Oh yeah, I did.

No, I did regression analysis of the times that the 12th person, the one who just qualified, did over time.

And it settled down after a few series to about three minutes, yeah.

Rob Bell:
That's brilliant, that's brilliant.

Well, as I mentioned, David, there are so many things that we'd love to delve into with you.

And to begin with, there are two sketches that Jono has created based on concepts that you've talked about, and in fact, one that you've created yourself.

So micro-morts and micro-lives.

So Jono, before we hear from David about these, do you want to say a bit about how you came across them and why they piqued your interests?

Jono Hey:
You know, it's really quite straightforward.

I came across them because I must have come across at least one of your very entertaining talks, which covered both of these concepts.

And I remember they grabbed me straight away.

And the sketches are very simple.

This was quite early on in this whole project.

So I'm really just describing what they are, which was that a micromore has a one in a million chance of a sudden death during an activity in a micro life, 30 minutes of your life, I think I called them the unit of death and the unit of life.

If anybody else has ever called them that.

But I think it grabbed me because everyone knows that some things are riskier than others.

But how much riskier is sort of always been very vague, I suppose, and why I love both these measures is why it sort of started to be able to put some numbers to this, which I guess you've spent a lot of your time thinking about these kind of things.

So I think it would probably make sense to start discussing micromores.

But maybe even before that, I think you distinguished when you discussed this kind of thing before, something called acute risks and chronic risk.

Maybe that's the right place to start.

David Spiegelhalter:
Yeah.

Acute risks is skydiving and chronic risks is smoking cigarettes.

In that one is going to kill you straight away and the other is going to influence your health and reduce your life expectancy.

It means you're at risk of dying earlier.

So it's really quite a simple one.

And the micromore, there was already a unit of acute risks developed by Ron Howard at Stanford in the 1970s, one in a million chance of sudden death.

So I felt that we needed one for chronic risks as well.

Jono Hey:
I was quite taken with the idea of micromores.

And I think in your, actually in your most recent book, The Art of Uncertainty, you have a nice table with the average micromores from a number of different activities.

I wonder, do you still remember some of these?

Or I can read some out.

David Spiegelhalter:
Yeah, read them out, read them out.

Jono Hey:
And they probably changed since when I first...

David Spiegelhalter:
Yeah, yeah, they do change a bit.

Things have got a bit safer, boringly safer.

Jono Hey:
Micromore, my original sketch was one day of living, 20 miles by bike, about 250 miles in a car.

Yeah.

Rob Bell:
So hang on, so just to recap for that, so 20 miles by bike or say 150 miles by car, you've got a one in a million chance of dying.

David Spiegelhalter:
On average.

Rob Bell:
On average.

Jono Hey:
Yeah, sorry.

David Spiegelhalter:
On average.

It depends how you drive.

I mean, but this is averaged over a lot of people over a lot of time.

Yeah.

Rob Bell:
Yes.

Okay.

Jono Hey:
So if you wanted to compare, as you do in your book, for example, you could travel 7,500 miles by train and you'd use up one micromort, or you can travel 7,500 miles in a car and that would actually use 25 micromorts.

And on a motorcycle, 7,500 miles is a thousand micromorts as a comparison of the sort of general risk of those activities.

David Spiegelhalter:
Which is why I never go on a motorbike.

Rob Bell:
Hang on, I'm going to try and extrapolate this out.

So is the concept then that over your lifetime you have a million micromorts to use up?

David Spiegelhalter:
No, no, because you're not necessarily going to die a sudden death.

It's very much the acute thing.

It happens straight away.

And so it's really tied to an activity.

It happens there on the spot.

It's an accidental death essentially.

And why, you know, roughly it's gone down a bit now, but you know, roughly one a day.

You get up in the morning and you go to bed dead.

That happens about one in a million people, you know, per day of unexpected death from one reason or another.

Some of them might be self-induced, of course, or whatever, you know, if you're drug-taking or something like that.

But that's the sort of, it's almost, you know, your daily quotient.

Now it's higher for young people or older people because they're more at risk, et cetera, but on average, that's our daily dose is one micromort a day.

So if you're going to do, you know, I don't know, a scuba dive or a skydive, they're all about between five and 10 micromorts, you're using up around a week's worth of background risk by doing this activity, which is, you know, I've done a skydive because I thought, yes, fair enough, especially when you're old, bring it on.

Let's have a go.

But, you know, some things like base jumping or, you know, jumping off a mountain in a wingsuit or something, you're using hundreds of these things up all the time.

You may get away with it, but it's like you've got more chance of winning the lottery if you buy lots of tickets and this is a lottery you don't want to win.

Rob Bell:
Yes.

And just to remind our listeners again, this is all based on data.

David Spiegelhalter:
Oh, yes, it's all based on data.

That's why it's such an average.

It's got a historical thing.

And so you look at how many people have died skydiving, how many people have died in cars and whatever, and how many people die on motorbikes.

And, you know, it really is about six, seven miles for a micro-morph on the motorbike.

You know, extraordinarily dangerous.

And it is.

Rob Bell:
Yeah, it is.

Jono Hey:
I think perhaps one of the highest figures you quoted was Summiting Everest.

David Spiegelhalter:
Yeah.

Jono Hey:
Which was thousands.

David Spiegelhalter:
Well, the other one I worked out was going on a raid in bombing World War II.

Rob Bell:
Oh, yeah.

David Spiegelhalter:
20,000 micro-morphs.

Rob Bell:
Yeah, gosh.

David Spiegelhalter:
And, you know, the chance of coming back was enormous on that.

But then, you know, if you're sick and you're getting cardiac surgery, that used to be a 2% mortality, so that's 2 out of 100.

Oh, God, I've got to do some sums now.

And 2 out of 100, if you do that by a million, that's 10,000, that's 20,000.

So it's gone down a bit now.

But roughly speaking, going on a bomber bombing raid in World War II was like having open heart surgery every time you went over Germany.

Rob Bell:
Yeah.

Wow.

Oh, my gosh.

David Spiegelhalter:
That was the risk you were taking.

So the chance of you surviving your 25 missions or whatever was low.

Very low.

Jono Hey:
I learned about Microlives from you some time ago, but it existed for quite some time.

Is this something that people still use?

David Spiegelhalter:
No, no, they don't really.

I like it.

Maybe it's got a name that sounds a bit funny.

I mean, that's why I think Ron Howard invented it.

Stanford, he was making a bit of a joke about it.

I think it's really serious and I think it's actually rather good unit to bring everything down to whole numbers, Ron talking about 1 in a million, 3 in a million.

Yes.

I mean, who can understand that?

But if you can make a comparison between motorbikes, skydiving, bombing raids, stuff like that, or having a general anesthetic is about 1 in 100,000, it's 10 micromorts to have a general anesthetic, it's roughly the same as a skydive or a scuba dive.

So, you start getting a perspective on what these things mean.

And running a marathon is about 7 micromorts.

You know, the skydiving, the running a marathon, the scuba diving, all about the same.

And you realize there is this natural thing where doing things is sort of a risk involved with them.

This is the sort of risk people are willing to take on a fairly regular basis.

Much above that becomes so obvious they don't want to do it or their family doesn't want them to do it.

Below that, there's not much fun.

So, there is a sort of natural level between 5 and 10 micromorts in terms of an activity that people are willing to undertake voluntarily.

Above that, it starts being generally, unless you're a bass jumper, pretty unvoluntary, like a bombing raid or open heart surgery.

Rob Bell:
Yes, yes, yes.

Jono Hey:
I thought it was quite interesting.

You quoted, for example, commercial fishing, I think, of having 3 micromorts a day.

David Spiegelhalter:
Yeah, 3 micromorts a day.

It's a thousand a year.

Because the point is that although they don't use the unit, the idea of a one in a million chance of death is used all the time by, for example, the UK Health and Safety Executive.

Oh, yes.

Because they consider a one in a thousand chance per year, being killed at work is an unacceptable risk.

Coal mining and commercial fishing regularly have gone above this, go above this all the time.

So a thousand micromorts a year as one in a thousand, three micromorts a day in commercial fishing in coal mining.

And these are the really high risk jobs that people do and have done.

But commercial fishing doesn't necessarily mean in a big boat, that also can mean in people just going off in small boats, looking at lobster pots and it's dangerous and they tend not to actually look after their safety that well.

So it covers all types of accents, all sizes of the boat.

That's what's quite interesting is that the Health and Safety Executive have actually pinned this number that a thousand micromorts a year above that is an unacceptable level of risk.

Rob Bell:
Do insurance companies look at that as well?

David Spiegelhalter:
Well, they're doing these calculations all the time.

I don't know if they wouldn't use the word microbe, but they're using this all the time in terms of sudden risks.

Now, when they're doing life expectancy, they're looking at chronic risks when you're buying life insurance.

They don't tend to be thinking so much in terms of accidents, although that is part of it.

They're thinking more in terms of illness and early death.

But if you're trying to insure yourself against a risky activity, then they should be doing these sums.

And that's why you get these exclusions.

Rob Bell:
Yes.

So if that's the unit of death, should we move on to the unit of life, the micro life?

David Spiegelhalter:
Well, as I said, the point about it is that I love this unit of acute death, but I realized that when a lot of people were talking about risks, they're not talking about the car crash, et cetera.

They're talking about lifestyle, lifestyle risks.

And being a slob in front of the television, not exercising, smoking, drinking too much, et cetera, et cetera, eating badly, all those things.

And there wasn't an equivalent neat unit of risk.

The natural statistical measure is what's called a relative risk.

So for example, smoking 20 cigarettes a day doubles your annual chance of death.

And that corresponds roughly to an 8 to 10 year reduction in life expectancy.

So you don't know, you may live for it.

You may, and there's always some people who are going to live a long time, but on average it will take 10 years, 8 to 10 years off your life.

And smaller things, I don't know, sitting in front of the television for a few hours a day or something, you've got relative risk looks about 1.1, so about 10% increase in risk.

And that corresponds to a year off your life expectancy, about two hours of television watching a day.

That's sedentary behaviour.

So you realise that a year, I've got to try to get the sums right, a year off your life expectancy when for an adult where you've got maybe 50 years to live, is like having a 50th of your life.

And in fact, it turns out that that 50 years is a million half hours.

So as an adult, you've got a million half hours to live, about 50 a day.

And there's a million half hours in your life, adult life.

And so I called it a micro life is something which as a habit would on average lead to you losing the equivalent of half an hour a day.

So it's easy to do.

It's two cigarettes, it's a bacon sandwich.

It's sitting in front of the television for two hours.

All of these lose you half an hour a day.

And the cumulative, you can add them up.

So if you do all of those, you sit in front of the television smoking, eating a pizza, drinking away and watching things and stuff.

That's a few hours off your life.

Longer than you're spending in front of the television.

Rob Bell:
I'm wondering though, you could probably easily rack up over 24 hours on an absolute bendy.

David Spiegelhalter:
Oh, you're on a bendy.

You could get less than 24.

But it's the habit.

I mean, if you did that all the time, you wouldn't live very long at all.

So that actually does make sense if you're really knocking them up like that.

And this never caught on.

Rob Bell:
I like it.

David Spiegelhalter:
I like it.

And every so often somebody says, Oh, smoking a cigarette is like 15 minutes off your life.

And now recently someone's had 11 minutes off your life.

It's about 15 or something like that.

So if you had up, you smoke 20 a day, it's five hours off your life.

It encourages you, if you're going to eat a bacon salad, you're eating it slowly.

And the other thing is about exercise, of course, you can gain micro lives.

So if you exercise the first 20 minutes, if you're a bit of a slob, but you get moved to 20 minutes a day, really gains you quite a lot.

But after a while, for half an hour's exercise, you gain half an hour of life expectancy, one micro life, which means it's like time stays still while you're exercising.

This is great.

The times that you are not aging while you're exercising.

I really like that idea, as long as you're already moving around a bit, that extra half hour, and if you exercise a lot, it stops.

And in fact, the curve might be U-shape, you're actually damaging yourself.

I like the idea that by exercising, you're stopping time.

Jono Hey:
It's like bonus time, free time.

Rob Bell:
Without consciously knowing, I probably live my life this way, David.

I do like my exercise, but I do also like eating baked sandwiches.

David Spiegelhalter:
Yeah, exactly.

I do the trade-offs all the time.

So I try to be as fit and active as possible.

At the same time, I'm not going to stop drinking wine.

Rob Bell:
Of course.

And nobody should ask you to do it.

Jono Hey:
I mean, the advice always changes about that.

But I do remember at the time, I think you said that sometimes having one drink a day was actually okay for you.

So that was a positive thing.

David Spiegelhalter:
Yeah.

Jono Hey:
If you have two, then you start.

David Spiegelhalter:
I still think there's a lot of argument about it, but I think the evidence is still that the first drink each day is medicinal, is actually beneficial to you.

There may be a small increase in cancer risk, but there's a reduction in risk of cardiac risk and so on.

But then it dips out.

And by the time you get to the two drinks a day as the sort of UK guidelines, by the time you get to that, you're about evens.

Jono Hey:
Yeah.

David Spiegelhalter:
And then it starts going up, up, up, up, up.

So what I always say is that the first one is medicine, and then it's poison, poison, poison, poison, poison.

It doesn't go medicine, poison, medicine, poison.

Jono Hey:
Sadly.

You might take the trade off of life expectancy if it was quite clear.

It's like, well, I'm going to have this.

I use up 30 minutes of my life in the future, but you know, that's worth it over 60 years of what my wife chooses.

David Spiegelhalter:
It's purely nominal value.

You can't see it.

You can't measure it at all, what the effects are having.

But I find it again, as a way of comparing activities so that, yes, you know, you stop people making a huge fuss about something or actually makes almost no difference at all.

I mean, the point about you do all this, you end up with exactly the same stuff that you shouldn't drink too much.

You shouldn't smoke at all.

You should get up off your backside as much as you can, and you should eat a decent balanced diet.

And that's all it comes to.

So, after all this analysis and statistical work, you end up with advice that people have been giving for decades.

Rob Bell:
Yeah, but it's a lot of fun to do that, isn't it?

And to break it down.

Jono Hey:
One of the things that I really liked, and I have quoted a number of times, although I'm not quite sure where I read it first, was that having a cup of coffee would actually give you microlives.

David Spiegelhalter:
Oh yes, yeah.

Coffee is beneficial.

People have made a big fuss about the possibility of cancer risk from coffee, but on the whole, coffee is beneficial.

The evidence is pretty strong about that.

Jono Hey:
I've definitely been in the office kitchen quoting microlives to my colleagues.

David Spiegelhalter:
Yeah, there's always a limit to this, of course.

Rob Bell:
And David, we talked quite a bit about the medical application of data here, and you've spent quite a lot of your career applying the analysis of data and statistics in the world of medicine, is that right?

David Spiegelhalter:
Yeah, yeah, yeah.

Rob Bell:
Is that somewhere statistics and data can be really helpful to help us manage our health better?

And I mean that individually, but also then as a nation and globally as well.

David Spiegelhalter:
Oh yeah, yeah.

I mean, medicine has been the main driver for statistics.

Certainly from post-war up until maybe 80s and 90s, there's a massive driver of statistical methods.

I mean, it developed before the Second World War by people being very interested in measuring things, but it tended to be counting things, you know, counting the number of people with measles, counting this and counting the other and trying to work out some causes.

But then they realized they could take the methods that had been developed pre-war in agricultural science where in order to test the efficacy of fertilizers and things, you randomly allocated them to different plots around a field.

You divided the field up into small plots and you randomly allocated so as to balance out the different effect, the drainage and the shade and that sort of stuff.

Then they realized you could do the same for medicine.

From 1948, they started the first big randomized trials where people were randomly allocated to treatments.

That's totally revolutionized medicine.

You won't get a medicine on the market unless it's been subjected to vaccine or anything, a randomized trial where people have been allocated it.

Some people get it, some people don't, and nobody knows who's going to get it.

Usually, nobody knows who got it in that you're trying to make a placebo.

The imitator looks exactly like what people have been given.

So the people who get it don't know what they're getting.

The people who are giving it don't know what they're getting.

In some cases, it's what's called triple blind.

The people who are analyzing the data don't know what A and B is.

They know that there's that treatment A, treatment B, and they don't know which is the new one, which is the old one, as triple blind.

So try to prevent unconscious biases coming in by the patients, the doctors, and the statisticians.

Jono Hey:
Even as a statistician, you might be analyzing this data and you're looking at it and you're going, I know this is the one with the new thing and I want it to pass.

David Spiegelhalter:
Yeah, we've all got unconscious biases.

Everyone knows that in statistics that they want to discover something.

They like to get a significant result or something.

No, you can't trust statisticians one inch.

But the interesting thing, what I love, I've been on data monitoring committees and the one people who get the unblinded information are the data monitoring committees.

So it's quite extraordinary that I've done this for a huge drug trials, incredibly multi-million pound trials run by, for example, Pfizer, a drug company.

They don't see the data.

The data is then seen by the independent data monitoring committee.

And I remember them flying to America to tell the ball of Pfizer that their drug hadn't worked.

They weren't very pleased, but they were quite decent about it, really, having spent millions of pounds on it.

Yeah, it's really rigorous science.

And even within the pharmaceutical industry, because of the strong regulation, the trials are really well done.

I've worked a lot with the pharmaceutical industry.

And you know, there may be all sorts of things one might not like about their pricing and about everything like that.

But in terms of the science, it's very good.

Rob Bell:
I think it will be fun to talk a little bit more generally about risk and uncertainty and luck or chance.

And again, we're with the right man.

Your most recent book, David, The Art of Uncertainty.

I mean, uncertainty is an inevitable part of being a human, right?

And in a whole manner of different ways.

Absolutely.

I mean, is there an easy way to define uncertainty in our everyday lives?

David Spiegelhalter:
Oh, I stole the definition I use in my book, which is the conscious awareness of ignorance.

And that you realize you don't know something.

And that might be about the future.

You're actually aware.

I don't know what's going to happen.

And it may be about what's going on at the moment.

And it may be what's happened in the past and why it happened.

And we're away.

We know so little.

Those little bubbles we're vaguely aware of.

And even then we might be mistaken.

But everything else is a complete mystery.

You know, we don't know what's going on.

We don't know what's happened.

We certainly don't know what's going to happen in the future.

We live in it.

We are creatures of uncertainty.

And long may it last.

Well, it will last, of course, because it's unavoidable.

But I think it's a part of being human.

Can you imagine being certain about what's going to happen tomorrow?

You might as well be in jail.

Rob Bell:
So almost everything is then our best guess based on our experience and the information that we have.

David Spiegelhalter:
So it's uncertainty.

It's a relationship between you and the outside world, because you may be uncertain about stuff that other people may know about.

But you don't know.

It may all be whatever happens.

It may all be completely predetermined, predestined.

It may all be God's will.

Who knows?

It doesn't matter.

I don't care.

I don't have to talk about that kind of stuff, because the crucial thing is that I don't know and you don't know.

So it's a relationship.

You have to live with that relationship and work out how you're going to deal with that.

And some people deal with it in very different ways.

Some people make some people very anxious.

Some people are very intolerant of uncertainty and want to know exactly what's going to happen and try to control everything.

And you get others who are much bolder and much more willing to deal with uncertainty and risk.

But even that's not a single dimensional characteristic.

People have found that there's not a single sort of risk.

Like you've got these big five personality traits, you've got neuroticism and extraversion and so on.

Risk aversion or proneness is not one of them because there's not a single trait.

Because some people may be very bold.

They may jump off.

I mean, I know people who do very bold physical things.

So they may jump off a mountain in a wingsuit, but be really careful with their money and their relationships.

Rob Bell:
So it's entirely subjective and individual.

David Spiegelhalter:
It's completely individual.

We all respond in very different ways.

And so, I mean, that's why psychologists like Danny Kahneman have realized that, you know, we do have one way of responding to risk or uncertainty.

And just from our guts, completely automatic, what he calls thinking fast, that gut feeling.

But as he said, you can think slow as well.

You can think, try to weigh things up.

And be a bit more balanced about it.

And, you know, it's not, as he said, it's not a choice between the two.

We're always operating a mixture of these two things.

And we do that individually in our lives, everything we do.

But I'm also interested in how corporations do it, how scientists do it and how policymakers do it.

And they, in fact, do the same thing because they've got, you know, they've got data, they've got analyses, they've got statistical models, they've got projections and things like that.

And yet that's never enough.

That's sort of thinking slowly, but it never really covers everything.

You need to think, you need to have a gut feeling as well.

You need to have judgment.

You need to have intuition.

You need to have imagination as well as those.

And so from both a personal and an organizational point of view, there's an amazing balance one can do between that, almost an iteration between the gut and the cold, you know, and the analytic capacity.

Rob Bell:
Yes.

David Spiegelhalter:
And it's not one or the other.

You have to have both.

Rob Bell:
You've basically described a relationship with my wife in that, David.

I'm very much the slow thinker, data analyze.

Look at all the review sites before I make a decision.

She's very much intuition, gut feeling, let's go.

David Spiegelhalter:
Oh, we've had that because we're both great travelers, but we traveled in totally different ways in that I would plan everything.

I had everything sorted out what we're going to do.

And my partner just arrived.

Oh, this is nice.

What should we do?

And this led to certain issues on trips and things.

But essentially, she's won.

We now can go to India for a month, and I only have the first night booked.

And that's it.

After that, we're going to make it up.

I got some ideas privately.

I got some secret ideas for it.

Rob Bell:
Yeah, but you can make out that that just came to you.

David Spiegelhalter:
Oh, it just came to me.

Rob Bell:
Make an inspiration.

David Spiegelhalter:
She's so good.

Once you get somewhere, she's amazingly imaginative.

We could do this, we could get this too.

We could do this sort of stuff like that.

She's really good.

So, yeah, no, we're very similar.

And then she criticizes me, I don't know if your wife does this, for being an optimizer, wanting to get everything.

Oh, we could do this, and then we could fit this, and then we could do that.

And she says, don't do it, don't do it.

You know, this is pointless.

And in fact, in the book, I really criticize the aim of optimization.

Really disastrous to try to optimize.

Because you can only optimize if you actually totally understand the circumstances.

Because it's delusional, it's so arrogant to think that I'm not going to be surprised by anything that happens.

That everything's going to follow exactly as I plan, I think it might.

And so therefore, my optimal path will work out the best thing to do.

No, you can never do it.

You can never do an optimal stuff.

So you shouldn't try.

Well, you could try, but realize, have the humility to realize you can't do it.

Because, you know, strictly speaking, to make an optimal decision, you have to know all the options, and even that's difficult.

You have to know all the possible outcomes that might happen after you've made those decisions.

What?

You can imagine everything that might happen in the future?

Then you've got to give them how valuable, how good they are, and then you've got to give them all probabilities.

Wow, for goodness sake, this is nonsense.

This is nonsense.

And I've taught this stuff for years.

And now I realize that it's completely impractical.

Rob Bell:
This is so good, David.

I've learned so much there.

Because what I'm realizing is my attempts to optimize only ever leads to frustration.

David Spiegelhalter:
You should obviously not just throw up your arms and say, oh, well, it doesn't matter at all.

No, I think you should try to maximize benefit.

Realizing you won't be able to do it.

But what seems to go down very well when I'm talking to corporations and everything, is the idea that what do you do if you realize you're going to optimize?

Well, you have to try to build in resilience.

Rather than being robust, which is optimizing thinking, oh, I'm robust to all the things that might happen because I've thought of them.

Resilience is to be actually robust to things you never thought of, so that you are in a situation so that you can recover or benefit from things that happen that are not on your list.

What's called deep uncertainty when you acknowledge, it's the unknown, it's Rumsfeld's Unknown Unknowns.

You acknowledge there are things that might happen that aren't even on your list of possibilities.

And while you can never protect completely against them and you certainly can't optimize, you can try to build in, and this means always having redundancy, a fallback position, some sort of safety, some sort of safety net.

And that's why when I talk to kids, I do school talks, and I always say, yeah, take risk, take risk, take risk.

But don't be reckless, don't be stupid.

Protect yourself against the downside.

It's like buying insurance or something.

Buying insurance, a good insurance policy, is resilience.

Because you're protecting yourself against things you never even thought might happen when you read an insurance policy.

Oh my God, you don't even have to list the things that are gonna happen.

So buying insurance is the classic issue of resilience, as long as there's a good policy that covers you against a lot of stuff.

And that's why I say to kids, no, take risks, but don't be daft, don't be daft.

Jono Hey:
I did a sketch not long ago of the old Unknown Unknowns.

David Spiegelhalter:
Oh yeah, oh really?

Jono Hey:
So I was looking into it recently and there was a quote from General Eisenhower, I don't know if you know this one, which is, In preparing for battle, I have always found that plans are useless, but planning is indispensable.

David Spiegelhalter:
Yeah, exactly.

I mean, I should use that one, I should still own, because it's really good to go through the motions of trying to do it, envisaging possible futures.

You kind of think of what kind of stuff could happen if you're really thinking through an issue, and what are the possibilities and things like that, and you look at what would be best to do in those circumstances.

You know it won't work.

The plans never work.

There we go.

The plans never work.

You know, always something will happen.

It won't go exactly as you thought, but it's really good to have thought it through.

And I think the main reason for doing that is to protect yourself against the real disasters.

Rob Bell:
Yeah.

Jono Hey:
David, can I ask, I mean you've written many books.

I think there's Art of Statistics and you wrote one around COVID.

You wrote one around Sex by Numbers or something like that.

David Spiegelhalter:
It's my favourite.

Jono Hey:
I'm sure there's interesting stories to share about that.

Well, maybe we should do some of those.

But I was curious what made you want to write this one about uncertainty, the most recent one.

David Spiegelhalter:
Yeah, because probably Art of Statistics has been very successful and I wanted to do another.

But I've always been interested in probability and uncertainty.

I've studied it for years since I went to university and the pure maths got too difficult.

So I moved into probability and statistics.

And I had a really inspirational teacher.

Some of the people put their enthusiasm down too.

He was working on really, what is probability?

Translating a book on that.

And the first line of the book is probability doesn't exist.

And I don't think probability exists.

I think we just made it up.

I think it's just a fiction.

It's a virtual quantity.

It's not out there in the world.

You know, we can't, there's no probabilityometer.

We can't measure it.

Not like heat or light or distance or weight.

So I think it's a great idea.

And it's really good to act as if it is there.

But it isn't there.

I mean, it's no, I could, you know, I take a statin because my risk of, 10 year risk of a heart attack or stroke is more than 10% and so I take a statin.

I haven't got a risk of a heart.

You know, where is that risk?

You can't open me up and find that risk.

It's just some nominal figure that's been estimated.

And the only way I think it's reasonable to describe it to someone and say, well, out of 100 people like me who ticked the boxes I did, we would expect 10 of them to have a heart attack or a stroke in the next 10 years.

Now, that makes sense because it's a purely empirical thing.

It's almost testable.

It's putting me in a block of people.

It's not my risk.

It's a proportion of people who something will happen, people who tick the boxes I did.

And that's the actual meaning of it.

And I think it makes sense.

And actually, good doctors will explain it like that.

Rob Bell:
Are there some uncertainties that we enjoy as humans and others that are less enjoyable?

David Spiegelhalter:
When I got an audience, I said, who likes uncertainty?

Nobody puts their hands up and says, oh, so you all want to know what you're going to get for Christmas.

So when you've recorded a football game, you want to know the result straight away.

You jump to the end of the who done it box set to find out who done it.

You're right.

Rob Bell:
We love it.

In some senses, we love it.

David Spiegelhalter:
We live with uncertainty.

And then the last thing I ask people is, OK, if I could tell you, would you want to know when you're going to die?

Jono Hey:
Does anybody say yes to that?

David Spiegelhalter:
About 10 percent.

And they're the planners.

They're the real.

They make you look quite lackadaisical.

Yeah, exactly.

So the ones who want to know when they're going to die want to plan.

And so, no, we live with uncertainty.

We really love uncertainty because it's what introduces all that novelty, all that surprise, all that serendipity, all that lovely part of life is due to uncertainty.

Rob Bell:
That's a really lovely way of thinking about it because you're right, uncertainty, I mean, it has the negative connotation to it, as you say, who likes it.

David Spiegelhalter:
No, you do.

I mean, can you imagine being certain, knowing exactly what's going to happen tomorrow?

Ghastly.

Rob Bell:
It's sometimes quite interesting to see how uncertainty and some statistics are used in the media, sometimes in communications, in the media, in advertising, sometimes in politics.

And it can be quite easy to not give the full story and the full picture with the statistics that you use and the probability and uncertainty that you present.

David Spiegelhalter:
Well, that's because politicians, for example, can't deal with uncertainty.

They're incapable of it.

They have no language for it.

They're not trained.

They have to be really sure about everything they're doing.

Rob Bell:
Well, yeah, they need conviction.

They need conviction.

David Spiegelhalter:
Conviction politics.

Conviction politics.

And it's nonsense because they're not sure about whether or not what they're saying will work.

They're not sure about what's going to happen.

They have to have this conviction and they feel that they have to.

Otherwise, no one will believe them or take any notice of them.

Our research and others have shown this is complete myth in that if figures in authority admit uncertainty, admit there's two sides to the argument, admit there's winners and losers, balances, positives and negatives, that it actually increases trust in the group who are most sceptical of them.

Rob Bell:
I can see that.

David Spiegelhalter:
When politicians and others have got a simple message, oh, let's keep it simple, bang this message through, they actually make it worse.

They're decreasing trust in the group of people they're trying to convince.

It's this myth that the general public out there can't deal with uncertainty, that they won't believe anyone who admits uncertainty and it's all nonsense.

It's not true.

Jono Hey:
For CEOs and politicians and how should we be communicating?

Maybe it's even as parents to our children, right?

David Spiegelhalter:
Yeah, no, I think you got to have humility, I think.

And even as parents to children, I mean, I found parenting quite challenging, but I do think I did used to apologize to my kids and say, oh, I screwed up and I shouldn't have told you off or that.

And I hope I had a degree of humility.

At the same time, I think you do have to, kids or anybody else, you can have a conviction about, well, I've weighed it up and there's positive, but I think, really think this is on balance the right thing to do.

Jono Hey:
Yes.

David Spiegelhalter:
And you can say that, I think.

Yeah, no, that is the right thing.

That's leadership, I think.

But you don't pretend that's the only thing to do or the obvious thing to do.

I think on the balance thing, and it's my judgment, this is the right thing to do and that, and we're going to go for it.

And I think that's trustworthy to pretend that, oh, I know it's self-evident and the evidence says this is the only thing to do.

That's not trustworthy because it's rarely the case.

Rob Bell:
Does it frustrate you sometimes that data is used in that way, that it's presented with this certainty?

David Spiegelhalter:
You should have heard me, COVID, screaming at the television.

And in COVID, I did have a chance to say it because I was on Andrew Ma, then the Sunday morning politics program, the main politics program in May 2020.

And I did a real rant about these, you know, government broadcasts each evening and the politicians would get up and spout hollow statistics.

We were like, A, wrong, and B, they didn't understand.

And I called this Number Theatre.

It was just using, trying to bombard the public with statistics.

It was just another 10 comms operating.

I said, I just wish they'd treat the public with respect.

They, you know, the public are hungry for information and knowledge.

Treat them with respect.

Get somebody who actually understands what they're talking about.

And this went down really well.

I got 1.7 million views on YouTube when it was replayed.

And I got on Gogglebox.

I mean, really.

Rob Bell:
Wipeout and Gogglebox.

David Spiegelhalter:
Wipeout and Gogglebox and Desert Island Disc.

No, there are three things I can just, you know, I've retired, but there's nothing more to achieve in the world.

Rob Bell:
That's the trio.

That's the one you want.

David Spiegelhalter:
Exactly.

Nothing more to achieve.

Jono Hey:
David, I was going to ask, there was something that caught me.

I think it was quite early in the book, which is, which is essentially about how we often talk about uncertainty and chance and risk in words and politicians might not often talk about the most likely thing is that's going to happen.

And people have quite different, potentially different meanings around the words they use.

And an oblique link here is I did a sketch a long time ago, not as long as the Microlives and Micromods, but on word spectrums, which is things like whisper, talk, shout, yell, scream, and so on.

And as I did that one, somebody sent me a really interesting paper, which I wonder if you've seen, which was released by the CIA on Words of Estimative Probability.

David Spiegelhalter:
Words of Estimative Probability, WUP.

Yeah.

Oh, no, it's absolutely standard.

And I've got, no, no, hang on.

Oh, here it is.

And I got a mug in front of me.

This is from J-Type.

This is an MI5 mug.

And it says if you're an MI5 and you use the word unlikely, you must mean between 25 and 35% probability.

If you use the word highly likely, you must mean between 80 and 90%.

So there is an official scale in the UK intelligence services so that the words mean something.

Because, you know, it's all very well in our general life.

Of course, we say, oh, that's likely to happen.

I think this might happen.

When you're in dealing with important decisions, those are useless and can be deeply misleading.

So they have a scale, you know, to be used.

And CIA has got theirs and MI5 has got theirs.

And they roughly correspond.

Although in the book, I talk about, you know, I look at seven different definitions of likely that are used around the world by different ages.

But they're all roughly kind of 55 to 80% or something like that.

IPCC use it for climate change.

It's an extraordinarily valuable thing to do.

Jono Hey:
I think the example I read was around the Military Reconnaissance Mission.

And they were looking at some photos as to whether or not something was an airfield.

So somebody says this is almost certainly an airfield.

And then therefore they're going to base real military operations.

David Spiegelhalter:
There has to be more than 95%.

Jono Hey:
Okay, yeah, absolutely.

Well, I mean, I think this was released based on the fact that, you know, somebody didn't actually mean that.

And they went and did something and it was wrong, you know.

David Spiegelhalter:
Yeah, the example I use in the book is the Bay of Pigs in 1961, where Kennedy found out about the planned invasion by the CIA of Castro's Cuba.

And he sent the Joint Chiefs of Staff to do a report on this.

And they thought it was a pretty daft idea.

And they assessed only a 30% chance of success or 70% chance of failure.

But by the time we got to Kennedy, the numbers had gone and been replaced by the phrase, a fair chance of success.

Rob Bell:
What?

David Spiegelhalter:
And Kennedy approved the invasion as a total disaster.

No, it probably wasn't only due to that report.

But afterwards, the Joint Chiefs of Staffs were so fed up that they hadn't made it clearer that they thought this is a really bad idea.

So you'd have to call that unlikely.

Rob Bell:
An unlikely chance of success.

David Spiegelhalter:
30% chance of success in the modern UK intelligence services.

You have to say it's unlikely to succeed.

But they didn't say that.

So that's used as an example of the real dangers of using words, because words mean different things to different people.

Rob Bell:
Absolutely.

David, Jono and I are both massive sports fans.

David Spiegelhalter:
I'm not.

Rob Bell:
Are you not?

David Spiegelhalter:
I couldn't care less.

Rob Bell:
Did you not do some work around football results?

David Spiegelhalter:
Yes.

I'm fascinated in sports betting, and footballers are producing good odds.

I'm not fascinated either.

My predecessor professor at the University of Cambridge left to join a sports betting company.

So statisticians are up to their eyeballs in sports betting.

Rob Bell:
So what are some of the interesting bits of data and uncertainty around sports?

Was it football?

David Spiegelhalter:
Yes, I did football.

I was using a model that was published, a fairly basic model that was really quite effective, and people were making money out of.

Then they left academia, the people who developed the model to form the sports betting company after that because nothing was published anymore.

Because the model has got more and more sophisticated.

But the one that was published and one I was using is just based on goals scored during the season and how many, if you've got two teams against it, you talk about one having an attack advantage if they're above average goals they've scored, and a defense weakness if it's below average goals they've let in.

So you can combine these of the two teams playing.

From that, you can get an estimate of the expected number of goals each team has scored.

Then if you apply a Poisson distribution, because goals scored actually do follow very accurately, remarkably accurately, a Poisson distribution.

You can get a joint distribution of all the different goal combinations.

So you never say who's going to win, but you can produce a probability of every single score, 2, 1, 3, 1, 4, comes out of a model.

And so you can use that to compare with the odds being authored by bookies and seeing if you can get an advantage over them.

And yeah, it works.

Except now because the bookies are using even better models.

It got much more difficult to beat the bookies because they've caught on and they've got good analytics teams behind them, which they didn't used to have.

Rob Bell:
I'm going back to Rain Man thinking counting cards, counting cards.

Yeah, yeah, yeah.

Jono Hey:
I think I've read in your book somewhere there was a statistician who was also a magician and managed to teach himself how to flip a coin such that it would always be.

David Spiegelhalter:
Yeah, Persi Diaconis.

So he was a traveling magician.

He taught himself he can shuffle a pack of cards and you can do all sorts of things, flip a coin and make it come up heads every time.

He then became Professor of Probability at Stanford.

He was an absolutely world leading probabilist.

But he then spent his time doing tricks and revealing cons the way that scams, the way that people knew he was a professional magician.

So you could see through what everyone could do.

But he could do the coin flip and also he could do exact riffle shuffles so he could take a pack of cards and do eight riffle shuffles and they'd end up back in exactly the same order they started it.

But he also did the maths of shuffling which says that if you're doing a good shuffle, you only have to do it seven times and it's essentially random.

Rob Bell:
From the pack in order?

David Spiegelhalter:
From the pack in order.

Rob Bell:
Yeah, there you go.

David Spiegelhalter:
The thing I got in the book is about, do you know this thing about the number of shuffles?

Rob Bell:
No, go on.

David Spiegelhalter:
Well, I haven't got my packet cards for me.

If I shovel a pack of cards, what I do is open it up and say nobody in the entire history of humanity has ever done this shuffle before.

I'm absolutely, I can't be logically sure, but I'm as sure as I am of anything that no one's done that shuffle before.

You shuffle the packet cards again, open them up and say, nobody has ever done this shuffle before.

I'm absolutely confident about that.

So that every shuffle that's ever been done in the history of card playing is unique, unrepeated.

You can be really, really sure of that.

Rob Bell:
That's amazing.

David Spiegelhalter:
Every shuffle is unique.

It's like we are all unique.

Every shuffle is unique.

I mean, the only way to show that is the fact that the number of shuffles, which is 52 times, because you've got a choice of 52 for the first card.

So it's 52 times 51 times 52.

So what's known as 52 factorial is an enormous number.

It's got 69 digits in it.

More than the number of atoms in our galaxy is the number of possible shuffles.

Rob Bell:
Just from the standard.

David Spiegelhalter:
One single pack of cards.

Rob Bell:
52 cards.

David Spiegelhalter:
52 cards.

There's never been any repeats in the shuffles.

We can be really confident about that.

Rob Bell:
That's a very, very fun little thing.

David Spiegelhalter:
The thing I learned recently, you want to demonstrate how many shuffles there are.

So you say, okay, I'm going to do one shuffle a second.

Okay, how long is it going to take me to work through all the shuffles?

You say, okay, I'm going to wait a billion years and then take a step and then wait another billion years and then take another step.

And I'm going to walk all the way around the world.

And when I get to the walked all the way around the world, I take one drop from the Pacific Ocean.

And then I do the whole thing again, billion years, billion years, another drop.

So I wait till the Pacific Ocean is empty.

And then I put a piece of paper on the ground.

Then I start again, billion years, billion years, drop, drop, drop, another piece of paper.

I wait till the piece of paper reaches the sun and then I repeat the whole thing a thousand times.

And then I've counted all the shuffles.

Isn't it a wonderful image?

It's such a lovely, ridiculous image of a length, an eon, you know, how long a length of period is.

Rob Bell:
Huge, huge number.

David Spiegelhalter:
Of a huge, huge number, yeah.

Jono Hey:
From a simple pack of cards.

David Spiegelhalter:
From one pack of cards, yeah.

So it's nice to do it in practice because I always carry a pack of cards, I shuffle it, show it to people and say no one's ever done that before.

Rob Bell:
Yeah, that's great.

Jono Hey:
David, I was wondering if I could ask, there's so many intriguing examples of where uncertainty matters and how you try and put numbers on these things.

But there's also lots of examples of where perhaps our intuition doesn't serve us wrong or where we would typically make mistakes.

And so I was wondering when you think about it, some of the most common ways that people go wrong or like our intuition lets us down around uncertainty.

David Spiegelhalter:
I'm terrible.

My intuition, my probability is terrible.

I have to do everything quietly.

I've sat down.

You know, I wouldn't trust my intuition at all.

The card one is how surprisingly rare things are, which we think might be quite common.

Okay, we've got the same stuff.

No, you won't have got the same stuff.

So that's one example of what our intuition is about.

But our intuition is also about the other way, is that some things happen far more often than we think they would.

And our intuition is really poor on that.

Well, it's a standard birthday thing that if you just take 23 people, there's a 50% chance that two of them have got the same birthday.

So that means that on a football pitch, there's 23 people, there's two teams and the referee.

So in half of all football matches, there's two people on the pitch with the same birthday.

And it's true, it works out, every so often people do it.

And in squads, World Cup squads have got 23 people in them.

So in half of World Cup squads, there's two people with the same birthday.

And again, it works every time, it's just true.

So that's quite surprising.

The other quite nice one is playing with your kids.

Okay, you got kids, you're sitting in the car, bored thing.

Okay, let's, next 20 cars that come past us, look at the last two numbers on their number plate.

Okay.

And if there's a match, could be anyone from, you know, 0-0 to 9-9, 100 numbers.

If it's a match, then they don't get their pocket money.

And there's an 87% chance there'll be a match, just in 20 cars.

Jono Hey:
My intuition on that is way off.

David Spiegelhalter:
You'll win 7-8ths of the time.

They don't get their pocket money.

Same thing works.

If you've got 20 people and they say, okay, look at the last two digits in your phone number, bet them that two of them have got the same last two digits and you'll win 87% chance of the time.

So that's quite unintuitive.

The fact that when there's lots of possible combinations, things match much more often than you.

You can make money out of that.

Only once.

Rob Bell:
Yeah, definitely.

So apart from the private ideas of pocket money and taking money off our friends, are there useful ways that just your average Joe, me and Jono here, could use uncertainty to help us out in life?

David Spiegelhalter:
Well, I think, I mean, we are experts in uncertainty.

We make judgements every time we go out the door, across the road, or do anything, and judge when to get a bus or something like that.

We're judging it all the time.

We're really quite good at it as sort of animal instincts.

I think we can sometimes be a bit cautious, but we're actually not bad at all.

You know, if you're gambling, obviously, you have to think about it much more carefully.

And so anything to do with money, particularly if someone's trying to get money off you, you know, things like, oh, you know, you go and buy some white goods.

Oh, we could give you this five-year insurance, and it's over half the price of the product.

Well, absolute nonsense.

I mean, it's just absolute rubbish.

That's kind of thing.

We realize that this is ridiculous, that the chance of it breaking down, you know, that time, you know, you just buy a new one.

If it does, you're just being ripped off.

You know, that's the way in which one can be calmed, I think, is buying excessive insurance.

Rob Bell:
That's a very good tip.

David Spiegelhalter:
Never buy the insurance of what online, or electronics or anything like that.

Rob Bell:
That's a great tip.

Jono Hey:
You mentioned in your book, I think it's called the gambler's fallacy.

David Spiegelhalter:
Yeah, yeah, yeah, yeah.

Jono Hey:
Can you say a quick word about that?

Because I think that's a good one to guard against, because it is one where it's quite...

David Spiegelhalter:
You know, if you've got a real...

What is a genuine random process?

It has to be a random process.

You know, coin flips or numbers coming up in the lottery.

And you think, oh, well, number three hasn't come up for a while.

It must be due.

And you still go on lottery websites.

You'll still go, oh, this number hasn't been...

This is a hot number.

It's becoming...

This number hasn't come up from where it must be due.

What?

What nonsense?

How do these little balls bouncing around know what happened in the last previous?

It's all...

It's complete nonsense.

And what they're doing is misunderstanding this idea that in the end, the numbers do balance out, roughly.

The proportions do balance out if you run long enough.

But it's not because there's some method of compensation.

It's just that imbalances get swabbed.

That's all it is.

There's nothing that says, oh, number three hasn't come up for a while.

Therefore it had better come up to even things up.

It's just that as you go more and more and more, it all just balances out.

Jono Hey:
So Rob, from your expression, I guess for me, some of the examples would be like if you were rolling a dice and you wanted a six, but you haven't had a six in…

Rob Bell:
For ten goes.

Jono Hey:
It must be due.

It must be due.

David Spiegelhalter:
Nonsense.

But of course, there is some intuition that says it isn't due because you might start suspecting it is balanced, in which case you can be intuished.

The most sensible thing is to go the other way and say, oh, it probably won't happen then because I don't believe the dice.

So the most sensible thing is always to go the other way, not to think it's due, it's actually less likely to happen.

Rob Bell:
As a kid, I thought I'd nailed it with dice games.

Right.

It was basically, it was cheating.

Yeah, yeah, yeah, yeah.

Instead of doing it that way, you just spin it on its axis.

David Spiegelhalter:
That's why I always carry a two-tailed coin.

I got a two-headed coin as well, so I have one in each pocket, and so I can win any toss that you want.

And so that's as a warning to, when people say, oh, it's a 50-50 chance of flipping a coin, I think, no, it's not, because I got this two-headed coin.

So that 50-50 is an assumption.

It's trusting me.

Don't trust me.

I always carry a two-headed coin with me.

It's pretty useful indeed.

Rob Bell:
And the two-tailed coin just for when they get suspicious.

David Spiegelhalter:
I gave a talk at the Bank of England and pulled my two-tailed coin and then said, it's probably illegal, isn't it?

Rob Bell:
Amazing.

Jono, before we close off, are there any other burning questions you want to ask David that we haven't got around to yet?

Jono Hey:
I have a million questions, but they're not burning.

But I will say, in case David doesn't, I have been reading the book, the latest one, and there are so many interesting things that we've talked about, some of them tonight, with a million other brilliant stories I thought you were going to bring up, but we haven't.

There's a whole chapter on luck.

David Spiegelhalter:
Oh, yes.

Oh, we could go on about luck for ages.

Jono Hey:
A chapter on coincidences, fun coincidences.

So it just goes on.

There's so much fascinating stuff in it, and also ways to help you look at statistics that are presented to you.

David Spiegelhalter:
Well, I mean, that's why I wrote the book.

In fact, I put in all the stuff I've been interested in for years and years and years.

You could miss the opportunity to write about a chapter on luck and some coincidences, so.

Rob Bell:
And these things affect us every single day of our lives.

Yeah, yeah, yeah.

David Spiegelhalter:
We just say, I've been lucky or I feel lucky.

Rob Bell:
Let's just drill down into that a little bit.

David Spiegelhalter:
Exactly.

So deconstructing luck is a really interesting issue.

Rob Bell:
It only remains for me to thank you once again for giving up your time and your expertise.

It's so generous.

Thank you so much.

David Spiegelhalter:
Really good fun.

Lovely, lovely opportunity.

Rob Bell:
And I know there's so much that he talks about that will be of use to our listeners.

I know, very intriguing as well.

Absolute pleasure, David.

Thank you.

David Spiegelhalter:
Really fun.

Thank you so much.

Rob Bell:
And for anyone who is keen to delve further into David's mind, David's world, you could do a lot worse than to get buried in his latest book, The Art of Uncertainty, How to Navigate Chance, Ignorance, Risk and Luck.

And I will include a link to it in the show notes.

And so to close out, I have just one final thought.

Carlsberg don't do statisticians, but if they did, thank you for listening.

Go well and stay well.

David Spiegelhalter:
Goodbye.

Jono Hey:
Thanks everyone.

Rob Bell:
All music on this podcast series is provided by the very talented Franc Cinelli.

And you can find many more tracks at franccinelli.com.