Sneaky Averages

Tom Pellereau:

I do remember at school learning about the mean, median and mode, going, well, what's the point in the middle point?

 

Jono Hey:

If you go to what is the average income of America, if you have a few highly rich people in there, you end up with a situation where nearly everybody is actually below average.

So in a situation like that, you're generally better off using the median, but probably you just need to look at the data.

 

Rob Bell:

This week, we're talking about sneaky averages.

 

Tom Pellereau:

The worst is when people do an average of averages.

One data set is really big and the other one is really small.

That just really messes up the data.

 

Jono Hey:

When I called it sneaky averages, I just meant that averages often hide something.

They are a little bit sneaky, but I feel like we veered into the people are sneaky territory.

 

Rob Bell:

Yeah, we absolutely hijacked.

 

Tom Pellereau:

And we reached that cynical age.

 

Rob Bell:

Hello, and welcome to Sketchplanations, The Podcast.

 

Tantalizingly close to reaching the top spots on all global podcast platforms, we continue in our quest to bring you the very best audio content money can't buy.

So what is it?

It's a podcast.

When is it?

Well, we record on Tuesday evenings once the kids have gone to bed, but it's released into your audio stratosphere first thing Thursday mornings.

How is it?

Don't you worry, it's doing just fine.

Who is it?

I'm glad you asked.

I'm Rob Bell, hello, and joining me to delve into the world of Sketchplanations once again this week is Southwest London's visionary virtuoso, Jono Hey, and Hertfordshire's favourite headstrong heartthrob, it's Tom Pellereau.

 

Good evening, chaps.

 

Jono Hey:

Good evening.

 

Good evening.

 

Rob Bell:

I mean, this won't matter or actually probably be noticeable to listeners because there won't be a gap between episodes when this goes out, but here we are all together after quite some time actually.

 

It's been a while.

 

I mean, I guess due to factors, brackets general, we've not recorded a podcast for a little while, but we've also been away and done some cool stuff in that time.

 

Jono Hey:

Mostly Tom.

 

Mostly Tom has been away.

 

Tom Pellereau:

I was eight hours ahead last week, which I know you really wanted me to record it, but I'd only been asleep for about three hours, so that wasn't going to be possible.

 

Rob Bell:

I think it would have added a different dynamic to it in another area.

 

Tom Pellereau:

I might have.

 

Jono Hey:

Don't wake up.

 

Tom Pellereau:

I might be more awake actually.

 

That was without the kids as well.

 

Rob Bell:

Tommy, you were away with work, were you?

 

Jono Hey:

Yeah.

 

Tom Pellereau:

I was in Hong Kong and China this time last week.

 

It was 29 degrees.

 

Spoiling.

 

I went over there with a couple of suits, you know, the jackets and all that was like, why did I bring any of these things?

 

It was boiling compared to here.

 

Jono Hey:

Shorts and flip flops.

 

Rob Bell:

It's funny, isn't it?

 

Saying like the weather in China, but because I bet somewhere in China at the same time, it was freezing cold, probably snowing.

 

Tom Pellereau:

Yes.

 

Yes.

 

Rob Bell:

Such a massive country.

 

And well, Jono and I have actually been away in the south of France in that time as well.

 

We went and did a swim run race down there.

 

Yeah, it was down there, branded, fully branded up.

 

Lovely.

 

Jono Hey:

Actually, I don't brand up for many things, but I do like these events, even though they're way too hard.

 

Rob Bell:

They are hard.

 

I always forget every year how difficult it is.

 

Yeah.

 

Jono and I, over the last, what do you mean?

 

What should we say?

 

Seven years?

 

Something like that.

 

I've done races in the discipline of swim run, which is a series of swims and runs.

 

And every year we really, really look forward to it.

 

And every year when we get there, we'll be trained for it.

 

We work quite hard.

 

And it's really nice because there's some other friends of ours who do it as well.

 

And you get to spend all this time in the buildup to it.

 

And then you get there and it's really difficult.

 

Tom Pellereau:

To call it a swim and a run sounds like quite a leisurely day.

 

But how...

 

Rob Bell:

It's a number of swims and a number of runs.

 

Tom Pellereau:

How many...

 

How far are these things?

 

It's like 50 kilometers run and 10 kilometers swim.

 

It takes all day.

 

Rob Bell:

I'm sure we've talked about swim run before.

 

Tommy, I'm sure your trip out to China was similarly grueling as well.

 

Tom Pellereau:

Yeah, it's always pretty...

 

Rob Bell:

Because I know what it's like, because obviously, the opportunity to go out there, it's not like you're going out there a number of times a year.

 

So when you do go out there, I know you've got a lot of stuff to get done, a lot of people to see.

 

Tom Pellereau:

Yes.

 

The first time I've been out for four years.

 

Rob Bell:

Who is it?

 

Tom Pellereau:

And the country continues to expand and change enormously.

 

And crazy things like you don't use a credit card to pay for anything in China.

 

Everything is done via WeChat, which is their equivalent of WhatsApp, and you pay via like QR codes that you have on your phone.

 

So if you go with like a credit card or with cash, they're kind of like, oh, what's this?

 

And you don't have to like log in or you end up letting someone else pay for you the whole time.

 

It's sort of really weird.

 

It's just like being a child again.

 

You're like, oh, sorry, I can't pay for anything.

 

Can you help me?

 

Rob Bell:

Anyway, we are all back.

 

We've unpacked the laundries on.

 

I've just finished watering the houseplants and there's a stack of mail over there.

 

It's mostly junk mail.

 

Am I right?

 

But that can wait because we have serious business to attend to.

 

The wise hows and wherefores won't answer themselves, you know.

 

Here's the book out.

 

This week, we're talking about sneaky averages, where the provision of a single isolated statistic might not tell you the whole story, or as is sometimes the case, tells a completely different one.

 

The sketch for this episode should be displayed on your screens now, but if not, then you can see it at sketchplanations.com, and I'll include the link to this specific sketch in the podcast description down below.

 

And before we get into it properly, remember you can get in touch with us about your thoughts on sneaky averages, or any of our previous episodes, for that matter, by emailing us.

 

Tom, what's the email address?

 

I know it's been a while, but...

 

Tom Pellereau:

Hello at sketchplanations.com.

 

Rob Bell:

Thank you.

 

Let's start, as many a lecture, speech, or even previous Sketchplanations, The Podcast episode have before, by quoting Mark Twain, who wrote, or supposedly once wrote, There are three kinds of lies, lies, damned lies, and statistics.

 

Right then, Jono, let's hear about how you came to think up this idea for a sketch.

 

First of all, Praveet, do you want to talk us through the sketch itself on sneaky averages?

 

Jono Hey:

Yeah, it's a really simple one, this one in terms of the sketch.

 

So it's just trying to point out your average may be hiding something, and it's really a visual for a little story slash joke that I read, which was about the statistician who drowned after seeing that the average depth of water was three feet.

 

And so I thought, yeah, it's just quite a nice illustrative example of some of the pitfalls of averages, and so the sketch is a statistician.

 

He's got a clipboard anyway.

 

Rob Bell:

He's got a clipboard.

 

Jono Hey:

A clipboard anyway.

 

And he's generally floating down into this trench with a little fish looking on and a big sign saying average depth three feet.

 

And of course there was a lot of shallow water that he perhaps was walking through, and then he suddenly disappeared down a hole.

 

Rob Bell:

Ankle deep.

 

Jono Hey:

Exactly.

 

And so that's what the sketch is, and it's called Sneaky Averages.

 

And it's trying to point out that your average may be hiding something.

 

Rob Bell:

Indeed.

 

And when you had this in mind, were you thinking specifically about averages or kind of misleading statistics more generally?

 

Jono Hey:

So I was thinking about averages.

 

There were a couple of things that came together for this one.

 

The first one is, and I picked it out, was I read this, I bought this fun, really old book, sort of really old, old book called How to Lie with Statistics.

 

Rob Bell:

Nice.

 

Jono Hey:

Which is, it's kind of a fun title, but basically the whole book is all the ways that, it's the other way around really, statistics lie to you or you can be misled by statistics, which of course you could use if you wanted to, to lie with them.

 

And it has a chapter, which I read a while before about the well-chosen average.

 

And it talks about how averages can be very misleading in some contexts.

 

And then much later, I came across that little story about the statistician who drowned after seeing the average depth was three feet.

 

And then I was like, ah, maybe that's a way to illustrate it in a visual.

 

So that's how I came across it.

 

And I was, because I read that chapter about averages, the well-chosen average, I was thinking about averages.

 

Obviously, there's many, many ways.

 

And I've got a few other sketches about different ways that charts and statistics can mislead you.

 

But this was also because I think I come up to the issues with averages, actually, in my everyday life and in my day job.

 

And so I had been thinking about it a fair bit.

 

So that's why, nice to point out.

 

Rob Bell:

I think we are subjected to averages coming at us from many different angles in our everyday life, in the news, in advertising, especially politics, and even science sometimes, scientific papers that you might read themselves or summaries of in the news.

 

Tommy, what's your view on this, having just seen you pop something in your mouth?

 

Tom Pellereau:

I'm eating chocolate.

 

Rob Bell:

Ah, come on, I thought we talked about this before.

 

Tom Pellereau:

I was expecting you to chat for a bit longer.

 

Rob Bell:

Okay, so one area that I see a lot, one area where I see, it might not be averages.

 

Tommy, that wasn't a cue to eat more.

 

Tom Pellereau:

He's going again.

 

I've created a link, because on average, in my dairy milk, there's a glass and a half of milk in every bar, but that is definitely not a glass and a half in that little bar there.

 

I'm sorry, what was the question?

 

Rob Bell:

Well, I was going to say, one area where you see statistics used in advertising a lot is in the beauty industry.

 

And so a lot of the time, you do see, I don't know, in shampoo or beauty product adverts that tells you about the product benefits and the results of people using them and followed up with phrases like 78% of customers agree.

 

And then the corner of the screen in a lightly colored text on a white background, it states that those results were from 76 out of 98 women surveyed, typically you might see something like that.

 

So are you aware of using averages?

 

Is it something you've come across in your line of work with your products and how you market those?

 

Tom Pellereau:

Yes, on lots of different levels of it.

 

Obviously I'm an engineer, so I do love statistics and numbers and data and have been fortunate enough to have quite a lot of training in it and how to spot for things that can go wrong.

 

I do find that a huge amount of advertising, marketing, digital marketing is all about numbers, Jono.

 

I'm sure you have this as well.

 

You're talking to your agency and their number of click throughs and the click through rate and the percentage click through rate and the cost per click through rate and blah, blah, blah.

 

And often they sort of average all that together to sort of say, this campaign, this week, you've spent X amount and this is how good we are sort of thing.

 

And you then have to start drilling into that information to find out often they have been very good in some things and it hasn't been so good in others.

 

And so that kind of danger of averaging out information can be, the worst is when people do an average of averages.

 

They take an average of this data set and an average of this data set and then take an average of the two.

 

And if one data set is really big and the other one is really small, say, that just really messes up the data.

 

So I do remember at school learning about the mean, median and mode going, well, what's the point in the middle point?

 

Like the median.

 

But actually that's quite useful one.

 

We use that quite a lot when we're surveying people to find out their view.

 

Rob Bell:

Is it worth quickly just explaining the difference between those three?

 

Tom Pellereau:

Yes, Jono.

 

Jono Hey:

Yeah, did you come across it?

 

I actually have a very old sketch called Measures of Central Tendency, which goes through the difference between mean, median and mode.

 

And they're all different types of averages essentially.

 

But the one we normally talk about when it's average is the mean value, which is the one we're most familiar with where you just add them all up and you divide by the number of items.

 

So that's the most common.

 

But the median value is like if you put all of the numbers and in a rank order, you take the middle one of the list.

 

And then the mode is if you lined up all the numbers and you said, okay, well, which one comes up the most?

 

So they're all different types of measures that you can use.

 

And actually, one of the things he talks about in the book that I read is that when someone says average, you might assume that they mean the mean, but they might not.

 

They might be talking about the median, and you don't know.

 

Rob Bell:

Absolutely.

 

And I think in a general lexicon, the word average has come to mean like the normal, like the norm, which it can be, but it doesn't necessarily mean that.

 

Jono Hey:

I don't know if you came across this.

 

At the risk of using up all my sketches in one podcast, there was one, did you come across the one Wealth Inequality in America?

 

Another very old one.

 

And it was from a video talking about wealth inequality and income is it.

 

So a lot of things mean is quite a good measure.

 

So for example, if you said, what is the average height of people in a country, people tend to be on like a normal distribution.

 

And so you don't tend to have like massive outliers of that.

 

Some people are gonna be generally, somewhere between five foot and six foot something, let's say.

 

Rob Bell:

Because when you use a mean average, those outliers can be quite distorted.

 

Jono Hey:

Yeah, yeah.

 

Rob Bell:

If you've got something, if you've got a few that are particularly small or a few that are particularly big, that can skew.

 

Jono Hey:

And so that's why I was saying, so generally speaking, something like a height, you don't have that.

 

But the wealth inequality one is something where actually you have enormous inequalities.

 

And actually you think about it, you go, okay, well, like, obviously some people don't have very little money living on the street.

 

A lot of people are somewhere in the middle and then there are some rich people.

 

And then you go, well, the rich people are really quite a lot richer.

 

So it's probably quite a big spike at the end.

 

And then actually, if you were to chart it out, the rich people are like, literally off the charts rich, like way off the top.

 

And so you actually have this strange situation if you go to like, what is the average income of America, it's very, or a neighborhood or something like that.

 

If you have a few highly rich people in there, you end up with a situation where nearly everybody is actually below average.

 

Yeah, which is very strange.

 

And so in a situation like that, where you have really big outliers, you're generally better off using the median, because you'd end up with something which is a bit more representative of the population.

 

But probably you just need to look at the data, like you said, Tom, with your campaigns, and go, okay, that's the average, but let's have a look at it.

 

Tom Pellereau:

Yeah.

 

Yeah.

 

Let's drill down there.

 

And I get accused a bit of being all about the detail and looking into stuff.

 

And we're supposed to be saying as a managing director and CEO of a company, we should be looking at high level and just looking at the summaries of stuff.

 

And I'm like, well, yeah, but actually if we drill down it, we often find that that campaign that looked bad was only bad because there was one outlier that was really bad and the rest of it was doing quite well.

 

So if we turn that outlier off, actually the campaign looks really quite nice, whereas other campaigns can look really good, but they're not really good until you get into the detail.

 

But Jono, on your point about average sort of inequality in America, I think the junior doctors, when they were on strike, there was quite a lot of, there were certain statements saying, oh, you know, these junior doctors, they get paid a fortune.

 

The average salary of junior doctors is, and it was because there were some quite high outliers because of specialization that they get paid a lot.

 

So they sort of drag the whole average up, whereas actually the median or the mode, what a typical junior doctor might get is actually much, much lower, especially when you factor in their overtime and the sort of sociability of their hours.

 

It's a very different story.

 

And so the government who was saying that the strikers didn't deserve this extra, they were using one average and the doctors or the people representing the doctors were using a very different average.

 

Like, how can this be that your averages are so different?

 

And it's lies, damn lies and statistics.

 

Jono Hey:

Well, no, I mean, it's spot on.

 

And that's why it's called like the well-chosen average, because often you might choose the reference point that works well for you.

 

And I think that happens a lot in politics, let's say, you know, you choose a data point to support your argument.

 

And it's not necessarily that people have got them wrong and not deliberately lying.

 

They just chose in a different way to represent the data that supports your point better.

 

And so basically, like this whole book is don't take data at face value.

 

Go, okay, well, where did it come from?

 

How did you measure it?

 

What does it all look like, what were the people who were showing me this data?

 

What do they care about?

 

What have they got to lose or win by it?

 

Tom Pellereau:

Yeah, yeah.

 

Rob Bell:

Yeah, and I guess when you're looking at data, there are three phases involved in getting to that end result where you've got a piece of data or a statistic to present.

 

So collecting the data, so how are you collecting it?

 

Who are you choosing?

 

How many are you choosing?

 

There's how you process that data.

 

So what are the maths that you're gonna do on it?

 

And then how are you gonna present those results?

 

They're the three different facets of producing and I guess publishing statistics.

 

And if you know what you're doing, you could use the same data set and come up with completely different arguments.

 

You could go in with the same kind of loose brief but come out with two very different results on how you go about it.

 

I got into an absolute rabbit hole when I started reading around this and I've started to lose my mind in it because there's so much to it, especially when you look at it from a marketeers perspective, so marketeers and statisticians perspective, people who really, really know what they're doing when it comes to these numbers and can manipulate it.

 

And not necessarily, when I say manipulate, it automatically feels like I'm saying it in an evil way.

 

It's not, it's using data, that's what it is.

 

It's manipulating using data.

 

Jono Hey:

I think a tricky thing is that you're not often in a position to know all of those pieces about the data you talked about.

 

Like you, all you get is a statistic and you're not really in a position to judge it in most cases.

 

And that was true in science a lot as well.

 

You get the results would be reported in papers and there's a movement now generally to include the original source data of your experiment as well, to allow other people to look at the data and check that it was done right.

 

Or maybe they could look at it a different way and see how you did it as opposed to just publishing the result.

 

But if you just get a number in a newspaper, it's you don't really know and you know.

 

Rob Bell:

Which is why I think all scientific data and papers should be presented in a pre-GCSE science experiment way.

 

Title, aim, apparatus, hypothesis, method, results, raw results, conclusion.

 

What is it where you use them?

 

Analysis, conclusions.

 

That way you do have the full story.

 

There you go, that's my pitch.

 

Jono Hey:

Good memories.

 

Rob Bell:

Great times.

 

Tom Pellereau:

The hypothesis was always quite fun, wasn't it?

 

It was like, hypothesis, let's just make something up here.

 

This is what we're trying to achieve.

 

Jono Hey:

I actually, I mentioned in the description for this sketch, I did a statistics course in the psychology department at UC Berkeley by this professor called Sheldon Zedeck.

 

Which is an awesome name.

 

One of the things that he said, which I put in this article, which really stuck with me, which was spend time with your data.

 

Because that's when you notice that things are outliers.

 

You haven't just sort of summed it up and said, okay, I've got all my data, I put it in the spreadsheet, I pressed the analysis and this was the result, is actually, you might have missed a ton of stuff unless you spent time with it.

 

And the other thing that he really cemented in me was this, what he called planned comparisons.

 

Because there's a really interesting, there's a great XKCD cartoon about it, there's a really interesting thing when you're doing an experiment, is that if you collect a lot of data and you look at it and you go, okay, well, I've got all of these people, I've got all these facets of the people, I've got all the things that they did in the experiment and I've got all the outcomes.

 

And if you look at enough combinations of things, there's a very good chance that some of these will correlate quite strongly and you'll be able to go, oh, well, you know, as long as you're male and over 40, if you do this and eat greens, then you're gonna live longer, right?

 

But that's because you looked at 50 combinations and it's like every time you look at a new combination for a new correlation, you have to put your threshold higher.

 

Rob Bell:

Your threshold for what?

 

Jono Hey:

Your threshold for whether or not it's chance.

 

Sorry.

 

Rob Bell:

Sure, okay.

 

Jono Hey:

So like it could just be random in this particular sample.

 

But what you need to do is you need to say, before you do experiment, this is what I think is gonna happen.

 

And when I get the result, I'm gonna look at this combination.

 

Because every time you come in and look at the combination, you've got just essentially some dice rolling.

 

And if you look at enough of them, you might find something.

 

The XKCD cartoonist, he's like, I think it's to do with jelly beans.

 

And it's, you know, to do jelly beans, you know, reduce your blood pressure or something like that.

 

And he's like, red jelly beans don't reduce your blood pressure.

 

Green jelly beans don't reduce blood pressure.

 

And eventually finds one of them, which had some random chance correlation that helps you out.

 

And so it's a really strange, it's a really strange facet of data that the more times you look at it, the more you have to take that into account.

 

So yeah, so when you talk about the hypotheses thing, that was one of the things that I remember, Sheldon Zedeck taught me as well, planned comparisons, should you be doing some analysis of your own.

 

Rob Bell:

So, because isn't there a kind of phrase for statistics in that correlation does not imply causation?

 

Jono Hey:

Yeah, absolutely.

 

Tom Pellereau:

That's the old shark tax and ice cream sales.

 

Rob Bell:

There you go.

 

Tom Pellereau:

So there's a massive correlation between ice cream sales and shark tax in most countries, obviously because shark tax happened in the summer when people are eating ice creams.

 

Rob Bell:

Yeah, I mean, there's a fantastic-

 

Tom Pellereau:

Obviously, sorry, but as in most countries, I exclude the UK, we don't get a lot of shark tax, luckily.

 

Rob Bell:

There's a fantastic graph I saw preparing for this chat tonight that pits the age of Miss America over a 10-year period against the number of annual murders in the US by steam, hot liquids and hot vapors.

 

And the two lines follow each other on this graph over this 10-year period almost perfectly.

 

Tom Pellereau:

Wow.

 

Rob Bell:

She's thinking, wow, look at those.

 

The age of Miss America and the number of annual murders by steam, hot liquids and hot vapors.

 

Obviously, there is no causation between these two.

 

There is no correlation of these two items.

 

Tom Pellereau:

I wonder how long it took the researchers to find two that match.

 

It must have just been there for days or years.

 

Jono Hey:

I don't know where you found that, but there's a site which collects those.

 

Rob Bell:

Oh, is it really?

 

This was on her blog.

 

Jono Hey:

Yeah, it's brilliant and it's got all sorts of, you know, just ridiculous things that correlate with each other, but obviously have nothing to do with each other.

 

Rob Bell:

Oh, that's fantastic.

 

I wonder if I can find that and include that link for that website.

 

Jono Hey:

Yeah, I forget the name at the moment, but it's a brilliant collection.

 

Rob Bell:

Quick interjection from me.

 

I've looked it up and that website is called Spurious Correlations and it's well worth a look.

 

I'll include a link in the description down below.

 

All right, back to you guys.

 

But then this brings us on to graphics that represents statistics as well, I think, because you can use graphics in a kind of cheeky way to get a point across, especially bar graphs.

 

Jono Hey:

Yeah, there's tons you can do with that.

 

Should you wish you could have a search on the Sketchplanations site for ways that you can lie with charts and graphs, because there's some really nice book and work by Edward Tufty about The Visual Display of Quantitative Information, and he goes to town on examples like those and chart junk where people have embellished charts with all sorts of random things.

 

There's some examples in this book of that sort of thing as well.

 

There's the one that I see perhaps the most, which really bugs me, is a bit like that.

 

They do something where, for example, you take the price of oil and you make it, you say, but you use a column chart, but instead of a column, you do a barrel, right?

 

Because it's an oil, so you say it's $46 per barrel, let's say, and then you say it's $56 per barrel, and you just scale up the barrel up to a height of 56.

 

And of course, what you did there is the heights are the change in the price of oil, but a barrel which is up that big is now holding like 50 times as much oil as the previous one.

 

And so you-

 

Rob Bell:

Yeah, because its height and its width and its depth all increase to that same amount.

 

Jono Hey:

Exactly, yeah.

 

So you just massively accelerated things.

 

So yeah, things like that where you use volume instead of area to compare things and stuff.

 

And like that example where you haven't also, you haven't put the zero on the chart, you've started it halfway.

 

So if you want something to look like a big change, you just start at 200 and you say go between 200 and 210 and then it's like a really exciting graph.

 

Otherwise it looks completely stable.

 

Another one I found is if you're putting a chart on a PowerPoint slide or Google slide, let's say, and you want it to look like it's going up more and just make the chart narrower and then it goes up more in a shorter space.

 

Whereas if you want, yeah, exactly, yeah.

 

So if you don't, if your chart is not going up steep enough, you know, that's probably because you put your chart really wide on the page.

 

You just print it really narrow.

 

Tom Pellereau:

That's good, that's good.

 

I've got a board meeting with Lord Sugar in a couple of weeks.

 

I'll make sure I better do that.

 

Rob Bell:

Can I give you another trick to me?

 

Another trick for sales.

 

Tom Pellereau:

Yeah?

 

Rob Bell:

Cumulative sales.

 

Tom Pellereau:

Ah, yeah.

 

Cumulative sales is excellent.

 

Rob Bell:

And it's always on the way out.

 

Tom Pellereau:

Yeah, exactly.

 

And don't take any negatives out.

 

That's a pretty clever one as well.

 

Jono Hey:

When I called it sneaky averages, I was thinking, I didn't really mean that people are usually sneaky.

 

Averages often hide something and they are a little bit sneaky.

 

I feel like we've veered into the people sneaky territory.

 

Tom Pellereau:

Have we reached that cynical age?

 

Is that the reality of this?

 

Rob Bell:

No, and I'm glad you clarified this, Jono, and I'm glad you clarified it, because this does happen quite often in this podcast where your sketch has a very clear intention of what it's trying to communicate.

 

Tom Pellereau:

And Rob finds the darker side of it.

 

Rob Bell:

Yeah, we absolutely hijack it and take it on a journey that it doesn't deserve to go on.

 

Jono Hey:

I think intuitively, all of us have quite a lot of practice thinking about this in some ways, like if you're taking into account reviews, and reviews is something where we're always, we're presented with data all the time, something got 4.5 out of five, or this one's five and there's a number of reviews.

 

And then I always think reviews are one where, it's so like you said about the where did it come from and how did you prepare it and how did you publish it, really affect those things?

 

Because reviews are extremely, and this is what happens in our work quite a lot, which is bimodal, I think.

 

And so you end up with like loads of reviews of products where they're, you either put a one or you put a four or five.

 

Either this was awful or you're four or five.

 

And you never heard from all the threes because they couldn't be bothered.

 

Rob Bell:

Yeah.

 

Is that what you mean by bimodal?

 

Is that what that means in that context?

 

Jono Hey:

I mean, yeah, I mean that.

 

Rob Bell:

It's either there or there.

 

Jono Hey:

Yeah, so if you were to chart them out and plot them on a line, you'd be like, oh, there's a big cluster at one of people who had an awful experience.

 

And then there's a big cluster of people who thought this was amazing.

 

Rob Bell:

Yeah.

 

Jono Hey:

And very often, there's nothing in between.

 

Rob Bell:

The twos and threes, yeah, it's fine.

 

Jono Hey:

I can't be bothered.

 

And there it's quite tricky because you're like, well, actually, we didn't hear from all the people who would have filled out the middle thing and we heard from the ones.

 

And so, but I think people sort of know that.

 

If you're reading reviews on a site, you go, well, would I have done, what do I do when I do reviews?

 

And maybe I'll read a few reviews.

 

Do I agree with the one-star review?

 

Okay, I'm cool if that's what the one-star review is because I don't think that person's worth listening to or something like that.

 

Or maybe it was just a bad day.

 

It's like the people who review a product on Amazon and they say, this was excellent, arrived very quickly.

 

Tom Pellereau:

That's not a review.

 

Jono Hey:

Amazon has been doing that for decades.

 

Let's talk about the product.

 

Tom Pellereau:

Five-star, arrived on time, what, that's not a review.

 

Jono Hey:

I think reviews are something which is very much like, you take an average and it usually doesn't mean very much.

 

Rob Bell:

Which is why you have to dig further and when you're on things like Uber or Airbnb or a trusted house, whatever it might be, you kind of then, or even like with tradespeople as well, it's something I've been spending a bit of time with, you need to look at that average and then delve in and look at some of the comments where you get a bit more qualitative information to back up that average, which may or may not be sneaky.

 

Jono Hey:

Yeah, if you have that.

 

I mean, how do you do, you know, with like rating Uber drivers or Airbnbs, do you use the full range of review numbers?

 

Rob Bell:

I'm quite modal with it, Jono.

 

Tom Pellereau:

Really?

 

Jono Hey:

You're ones and fives.

 

Rob Bell:

Ones and fives.

 

But typically fives, because ones you go, oh, that was, I'm not even going to bother.

 

Tom Pellereau:

Yeah.

 

Rob Bell:

I think.

 

Well, because you never know, because like they might be having a bad day as well.

 

Jono Hey:

So it might have been a bad ride, but you're wondering about how they feel.

 

Maybe they really need the job.

 

I don't want to screw them over.

 

Rob Bell:

I'll give them a five.

 

I'll tell you where a real difficulty with this is, is in politics where statistics are being used on both, I say both, there may be more than two sides of an argument or of an issue, because we're aware of how data can be manipulated and how averages can be sneaky, intentionally sneaky.

 

How do you decide who to trust?

 

Without having to do as much reading as was probably necessary to create those statistics in the first place.

 

Jono Hey:

Do all the work yourself.

 

Rob Bell:

Which obviously nobody can do.

 

So when you're talking there about, like the general public, you feel like there's probably a decent understanding about how that works with the review system, with apps and things like that, because it has become quite a common vocabulary and a common language for us all.

 

When it comes to politics, I think that it's becoming increasingly difficult actually.

 

Tom Pellereau:

I completely agree.

 

And it's unfortunately very, very difficult.

 

And I think potentially democracy has been slightly warped.

 

To plug a podcast, More or Less from the BBC podcast, is brilliant in terms of looking at statistics and understanding recent ones.

 

Rob Bell:

Tim Harford.

 

Tom Pellereau:

Yeah.

 

Undercover Economist is also a very good book.

 

Also by Tim Harford.

 

So no, you're completely right, Rob.

 

There are a lot of misuse of statistics and statements.

 

Rob Bell:

So is it on both sides?

 

Is it fair to knowingly use statistics sneakily?

 

Because a statistic that has come from data is valid in some sense, right?

 

Because it's numbers, it's manipulation of numbers.

 

So it's the context of it that will hide the sneakiness.

 

Jono Hey:

Yeah.

 

You need to sort of say, like, this data was collected by, you know, knocking door to door between this period in these locations or whatever, if you were, you know, collecting, doing a poll, say.

 

You need to say as much as you can about that.

 

And then you can sort of judge for yourself, the reader can judge for yourself a little bit about, OK, well, do I think that's a good way to be gathering data, for example?

 

Rob Bell:

And you know what, with advertising, you've got things like the Advertising Standards Agency, who are there to receive complaints and to look further into claims.

 

And there's some great, I mean, the eight out of ten cats for whiskers, is a classic example of that.

 

There's a Colgate ad, I think, a little while back, 2010, 2011, saying that the number one toothpaste used, this particular toothpaste was the number one toothpaste used by dentists.

 

But then there's an asterisk by that, which says that that was based on a survey of 300 dentists between this time period.

 

And if you dig a little further, you find that the British Dental Association has actually made up of 23,000 members.

 

So where is it a meaningful sample?

 

Tom Pellereau:

So I do hope that AI is going to really help in this remit.

 

So I don't know if you've used chat GPT-4 today, but they've just done a new upgrade where you can actually, it'll give you the citation of where that information has now come from.

 

So if you say, you know, something ridiculous like, which is the most preferred cat food, it will probably say and it'll give you that citation of where that whiskers quote came from.

 

And then you can look at the data yourself and blah, blah, blah.

 

And I hope that AI is going to become much smarter when you read a statistic or whatever.

 

You can ask your AI, where did that come from?

 

Give me a bit more data about it.

 

I don't know if you can then ask the AI to tell you how reasonable that is.

 

And that's when it probably becomes a bit of an arms race over who controls the AI.

 

But it's possible that in time these things will become more transparent and better.

 

Rob Bell:

Yes.

 

And as you say, not just on statistics, but I find statistics, unfortunately, I find statistics quite a hard word to say for this podcast, unfortunately for me.

 

Jono Hey:

It's funny you mentioned about AI though.

 

I actually use it because AI is excellent in principle at summarizing.

 

And I actually fed in a whole load of reviews and said, can you summarize these reviews, qualitative reviews, like comments that people have left and say, can you let me know what the overall is on, you know, it matters what the prompt is.

 

I might have said, what's typical?

 

What's average?

 

Can you summarize?

 

I'm not sure what the prompt was exactly.

 

But I disagreed with it in the end.

 

But it wrote a very plausible thing.

 

And then I went back and I looked at all the reviews and I was like, that's not how I would have summarized it.

 

You know, yeah, but you wouldn't know that unless you sat down and looked at all the individual ones.

 

Tom Pellereau:

It's quite tricky.

 

There's still a way to go.

 

Have you asked it about yourself?

 

I suppose it's quite interesting because it can come up with like completely ridiculous stuff.

 

You're like, hang on, I know.

 

Hang on.

 

That's about me.

 

No, I know that didn't happen.

 

Oh, that's cool.

 

Hang on.

 

Jono Hey:

I was thinking about the cats and whiskers.

 

Remember, there's the one like nine out of ten people prefer Pepsi.

 

I think in taste tests, people generally do prefer Pepsi to Coke.

 

What I heard, and I don't know if it's true, is that usually because Pepsi is a little bit sweeter, you prefer it on the first taste.

 

But if you were to drink a whole can of it, you might prefer Coke with something slightly less sweeter.

 

And so there is sort of a really interesting thing where you're like, they collected the data.

 

People said they preferred it.

 

Tom Pellereau:

That's true.

 

Jono Hey:

But they didn't.

 

But maybe they only had a sip.

 

So when, you know, what do you actually count as a valid approximation to reality?

 

It's quite tricky.

 

Rob Bell:

So I went on to the Advertising Standards Authority.

 

Agency.

 

Agency.

 

Try again.

 

Tom Pellereau:

Try again.

 

Rob Bell:

There's one of those two.

 

ASA.

 

To try and find a kind of set number for surveys to make any kind of advertising claim to see if there was a set number to make those statistics statistically significant.

 

And there is no set number.

 

It's all about kind of proportionality.

 

Tom Pellereau:

It's really irritating.

 

Rob Bell:

Reasonable as someone that those kind of words reasonable claims.

 

Well, help me define that.

 

Tom Pellereau:

And there isn't there isn't clear cut guidance on it's got to be X number of people because I remember going through this when we were doing teeth finding stuff.

 

They're like, no, can you just tell me how many people we need to have on the survey?

 

Because I see a lot of these, as you said at the beginning, Rob, these sort of beauty surveys where there's a big advert all about L'Oreal or Estee Lauder or something.

 

And, you know, 98 percent of people love this.

 

And you see at the bottom, it was like survey of 50 people.

 

And I'll come on.

 

That just seems so.

 

Rob Bell:

First time I started noticing that, it just made me laugh.

 

Tom Pellereau:

50?

 

Rob Bell:

Surely that can't be valid.

 

And we don't know who those 50 people were.

 

Did those 50 people?

 

I don't know.

 

Tom Pellereau:

We surveyed 500 and 450 words of bothered.

 

So we just chose these 50.

 

Is that allowed?

 

Rob Bell:

Did they all receive free goodie bags full of all your goods?

 

Did they all work?

 

Yeah, we don't know the backgrounds of our friends and family.

 

There we go.

 

Tom Pellereau:

We just thought so cynical of it anyway.

 

Rob Bell:

Well, yeah, but probably rightly so because we've been guided down this path by the likes of advertisers.

 

Mind you, that said, I've also been guilty of looking at statistics from, again, and I've talked about this before when I worked in an office and had a job project managing, and I would do a survey every year qualitative and quantitative.

 

And when I got those statistics and figures back, I was so excited to delve into them to try and find the best possible way to present myself at my end of year review from those numbers.

 

And I love this.

 

Jono Hey:

Brilliant.

 

Rob Bell:

Got a spreadsheet full of these numbers.

 

Jono Hey:

Let's get going.

 

Rob Bell:

Where can I find the best statistic?

 

Oh, that's not very good.

 

Tom Pellereau:

Let's get rid of that one.

 

Rob Bell:

Sounds like good heart's law.

 

I found one here.

 

I don't think anybody's ever reported this statistic before, but I'm now going to come up with an argument why I feel like it is a really significant statistic to present.

 

Tom Pellereau:

Because it's good.

 

Rob Bell:

So I am absolutely guilty of it as well.

 

Jono Hey:

You don't work there anymore, do you Rob?

 

Rob Bell:

But that said, I was trying to find some statistics.

 

Well, I was trying to find some statistics about the podcast that I could use to finish off the podcast.

 

I mean, guys, I don't know if you're aware of this.

 

Jono Hey:

They're average listeners in the middle of the Atlantic, isn't it?

 

Rob Bell:

Well, so according to statistics, if you want to cut statistics some ways, 0.1% of Guernsey's population are avid listeners.

 

Tom Pellereau:

That is bad, right?

 

That's the people we know.

 

Rob Bell:

We have a close friend who lives in Guernsey.

 

And Guernsey was, of all the locations that get separated out, because Guernsey is not a country, it's...

 

Tom Pellereau:

Ireland.

 

Rob Bell:

It's an island, but it's part...

 

Anyway, it gets separated out from the UK on the back end of our podcast publishing website, so I can see where people are listening from.

 

And I tried to find the location with the smallest national population, or the smallest population, and Guernsey was it.

 

So I went for that, because the number is quite high in terms of how many listeners we'd had.

 

Tom Pellereau:

Amazing.

 

Rob Bell:

But 0.1% it still isn't that big, right?

 

But I was no way near going to go...

 

Jono Hey:

I probably wouldn't just pick it out for your annual report, yeah?

 

Rob Bell:

No, but if you went to UK, where the majority of our listeners are, and put that against whatever it's 70,000, that didn't come out any better.

 

But also, did you know...

 

So this was my highest percentage.

 

I don't know if you're aware of this, guys, but Sketchplanations, The Podcast, has a listener to production team ratio of over 2,300%.

 

Jono Hey:

Ooh!

 

Rob Bell:

So there you go.

 

Jono Hey:

Excellent.

 

How's that changed from last episode?

 

Rob Bell:

Boys, any other business on sneaky averages?

 

Jono Hey:

I was trying to think of...

 

It was funny that you asked Tom what the weather was like in China.

 

I was thinking like asking the weather in China is exactly...

 

It's kind of like a sneaky average kind of question.

 

As you say, if you're on top of Mount Everest, it's pretty chilly.

 

Where Tom was, it was 29.

 

I was thinking the Marianas Trench is like the average depth of a stretch of ocean.

 

That's kind of what I drew in my sketch.

 

I was trying to think also if you said what's the average flood depth of an area where you were buying a house or something.

 

We don't really care about the average in that case.

 

Rob Bell:

Yeah, very true.

 

What happens at number 77?

 

Jono Hey:

The maximum is probably more relevant in those kind of scenarios.

 

Rob Bell:

For a number of years, along with a group of probably about six or seven people, I went and ran some of the world's biggest marathons, so some of the big cities like New York, Chicago, London, Berlin.

 

I remembered this earlier.

 

After some of them, and I remember in Berlin, a couple of weeks after, I'd put together a presentation deck, a slide deck, and send it out to everybody.

 

Everybody had their own page, and I would have found their overall result, the result for where they came for their gender, the result for where they came for their gender in that age group.

 

And I'd always find some way of making everybody number one in a particular category.

 

So just keep cutting it, keep cutting it.

 

And I think for one of our friends that came up with the forbearded Belgians who were dads over the age of 40, he was number one.

 

Tom Pellereau:

Nice.

 

Jono Hey:

That's brilliant.

 

I think one of our friends is like the fastest London marathon dressed as a 3D cartoon character.

 

Rob Bell:

Yes.

 

And that is a genuine Guinness World Record.

 

Tom Pellereau:

Yeah.

 

Jono Hey:

Which is very impressive.

 

But I suppose there's a lot of different things you could have records for.

 

Rob Bell:

Hats off to you, Jez.

 

Or 3D robot head off to you, Jez.

 

Tom Pellereau:

Yes.

 

Rob Bell:

Well, to close out this episode, instead of quoting some more useless and highly manipulated Sketchplanations, the podcast statistics, I thought I'd use another supposed Mark Twain quote, as Tommy necks back another bit of dairy milk, that facts are stubborn things, but statistics are pliable.

 

Thank you all very much for listening.

 

You can email us your thoughts and your stories about sneaky averages to hello at sketchplanations.com.

 

You're disgusting, Tom.

 

We'll be going through last week's corresponders in just a sec, and there'll be another episode ready and waiting for you next week.

 

But for now, again, thanks for listening.

 

Go well, stay well.

 

Goodbye.

 

Good night.

 

All.

 

Right, let's delve into this week's post bag.

 

And we're going to start with a message from Richard on Instagram.

 

So this is in reference to our episode on Groupthink last week.

 

And in that we talked about the play and the film 12 Angry Men.

 

So Richard says, 12 Angry Men is a great example of how one person, through force of personality, can use a variety of approaches, a variety of approaches, sorry, to make a group of people question a communal belief.

 

Absolutely, Richard, that is exactly it.

 

But he goes on to say there's also a great book called Leadership Secrets of Attila the Hun, right?

 

Which apparently illustrates how to lead through pure force of will and character.

 

Richard goes on, it's sometimes used by bosses, especially in regards to imposing their will, by targeting the member of staff they feel would be the loudest dissenting voice on a contentious decision that they want to push through.

 

He says it's a fascinating read with plenty of modern day applications.

 

Can you imagine that?

 

A leadership book on the leadership style of Attila the Hun.

 

Tommy, how would that go down in your company if you suddenly came in and put down the leadership philosophies of Attila the Hun for everyone to find?

 

Tom Pellereau:

Maybe if I weighed a thousand kilos, like I'm sure he did, that might work out better for me.

 

But as I'm quite small, yeah, that would be awful.

 

And actually having been with Lord Sugar today, if I tried to do that approach in board meetings with him, that would also end very, very badly.

 

Jono Hey:

I don't know, I think we need to read the book now.

 

I'm quite intrigued.

 

Remind me, I was re-listening to the discussion we had about clickbait headlines, and I was like, that's quite a good one, isn't it?

 

The leadership secrets of Attila the Hun.

 

I'm like, oh, what are the leadership secrets of Attila the Hun?

 

Rob Bell:

I wonder if there are some positive things to take out, as well as some questionable leadership values.

 

Jono Hey:

I mean, I don't think somebody would have written a book on it if it was all about overwhelming your opposition with arrows.

 

Riding them down on horseback.

 

I think there's something in it.

 

Rob Bell:

That wasn't necessarily what I was suggesting for Tony Cawdron.

 

Jono Hey:

You know, after our discussion, I thought about that film, and I was like, wow, it's crazy that it didn't come up to me before, but it's just such a great dramatisation of groupthink and the power to undo that and actually what it takes to do it.

 

I mean, it's fictional, but still, yeah, it sticks with you once you've seen it.

 

Rob Bell:

Thank you very much for your message, Richard.

 

We've got another one here from Christopher, also via Instagram.

 

Christopher says, It's always difficult to find the right balance with group meetings.

 

You always get the quiet one, the vocal one, the opinionated one, the joker, the serious one, the one that's happy to sit on the fence and the dominant one.

 

He says he could go on forever, but mainly you see these types of people in meetings and the vocal and domineering one always feel they get their way.

 

I will paraphrase this slightly.

 

So Christopher goes on to suggest that for the domineering one and the vocal one within a group, you can hear them out, maybe let them leave the room, and then you can allow those who maybe weren't as confident to put their voices forward, which is a lot about what we talked about last week, Jono.

 

You know, what I loved about last week's episode actually, and I'm going to talk about this, is that there were quite a few really strong recommendations and piece of advice, tools really, if you like, on how to try and avoid groupthink, which I hope is really useful to some of our listeners who maybe have to lead meetings and have to lead group decisions.

 

Jono Hey:

Yeah, definitely.

 

I mean, we're in groups all the time.

 

And so they're so helpful, yeah.

 

Rob Bell:

Both formally within work and professional and informally as well, which, again, we touched upon last week.

 

Jono Hey:

Yeah, and just your social groups.

 

Rob Bell:

Right, I'm going to be looking out for you using some of those tools from last week.

 

Jono Hey:

Using these tools against you.

 

Rob Bell:

Let's see what he's doing.

 

Jono Hey:

Rob, do you mind leaving the room now?

 

I've got some things to discuss.

 

Rob Bell:

And on that bombshell, I will leave Tommy and Jono to discuss other matters, and we'll leave all of you as well.

 

Thank you very much for all your messages.

 

Keep them coming in.

 

We'll be back in a couple of weeks.

 

We'll be back in a bye week.

 

Jono Hey:

Stop it.

 

Rob Bell:

All music on this podcast is sourced from the very talented Franc Cinelli.

 

And you can find loads more tracks at franccinelli.com.