Dunning Kruger Effect Wrong, Scientists Claim

0:00 The dumber people are, the more confident they are that they're not dumb. This, in a nutshell, is the Dunning Krueger effect, which has been used to explain everything from incompetent politicians to the demise of social media. But is it actually true? The authors of a new paper have reanalyzed the data and they say we've got it all wrong. Actually, confidence is a good predictor for actual performance. Let's have a look. The Dunning Krueger effect was named after psychologists Justin Krueger and David Dunning who reported it first in 1999. Their paper opens with the example of MacArthur Wheeler, a bank robber who rubbed lemon juice on his face because he believed it would make him invisible to security cameras. Dying and Krueger tested university students

0:50 on questions in humor, grammar, and logical reasoning. The participants then also had to estimate how well they did. Those in the bottom quarter performed around the 12th percentile on average, but believe they were near the 62nd percentile. Krueger and Dunning therefore argued that the skills needed to perform well were also needed to recognize poor performance. In 2000, they won the IG Nobel Prize in psychology. The Dunning Krueger effect has since been independently reproduced a few times and has entered pop culture. The best summary probably comes from John Cle. There's a a wonderful bit of research by a guy called David Dunning at Cornell who's a friend of mine, I'm proud to say, who's pointed out that in order to know how good you

1:36 are at something requires exactly the same skills as it does to be good at that thing in the first place. Which means, and this is terribly funny, that if you're absolutely no good at something at all, then you lack exactly the skills that you need to know that you're absolutely no good at it. And this explains not just Hollywood, but almost the entirety of Fox News. The authors of the new paper now used the biggest data set we have on the Dunning Krueger effect which comes from a 2021 study that also used a set of grammar and logical questions and then asked participants to guess their own score. Let me just show you the data. I'll use the grammar questions but the data set for the logic questions is very very similar. The data is this cloud of pale blue dots.

2:25 On the horizontal axis you see the actual score and on the vertical axis you see the score that people predicted. The diagonal blue dashed line that would be a perfect correlation if everyone correctly predicted their score. The usual Dunning Krueger effect shows up in this data by making a linear regression that minimizes the deviation of the prediction. That's the orange line for the standard Dunning Krueger effect. One basically asks if I look at everyone who scored say five, what was their average prediction? What you see is that the people who had a low score on average overestimated how well they did whereas the people who had a high score on average underestimated themselves. The authors of the new paper now say look this is doing it wrong. We're asking the

3:13 wrong question. We should instead ask how did the people who predicted a certain performance say five points actually perform. This is a more reasonable question because people's expectation of themselves is less variable than their actual performance which might depend on you know what they ate last night, what shirt they wear, if it's raining today and so on. In this case, you make a linear regression that minimizes the deviation of the score. And what you get then is this green line. And this green line tells a completely different story. It says that people on average score less than they thought they would. However, the discrepancy between prediction and score gets larger, the better the performance. And even if you leave that aside,

3:58 it's still the case that the more confident people are, the better they actually score. Okay, interesting you may say. But wait, how can it be that there are two different best fits to the same data cloud? It's because each of these curves is best in a different way. Loosely speaking, for the standard Dunning Krueger, you minimize the deviation into the horizontal direction for the new paper in the vertical direction. You could instead take the mean of both, which would give you this line. And that tells you that the people who score least underrated themselves whereas the people who score highest overrated themselves. Still, how can it be that there are so many different ways to look at this data? Well, as a physicist, I'd say just

4:46 look at the data. It basically doesn't have any correlation whatsoever. Why the hell are you fitting any straight line to this? This is why I give this paper as well as all previous papers on the topic a 10 out of 10 on the [ __ ] meter. These are all crappy data analysis. If you draw enough lines through the data, one will eventually confirm that everybody else is an idiot. Let me instead tell you something much more interesting. The original experiments also found that teaching poor performers the relevant reasoning skills improved both their performance and their ability to recognize their earlier mistakes. To me, this means that these tests don't actually measure intelligence. They measure knowledge. There is also the problem that the more people know

5:35 of the Dunning Krueger effect, the more they'll revise their prediction of their own knowledge. And if we've learned anything from this, it's that being confidently wrong makes excellent outrage bait. You see, the best thing about the Dunning Krueger effect is that everyone recognizes it immediately in someone else. I find it fascinating how much research goes into everyday objects like the razors from today's sponsor, Henson. This might come as somewhat of a surprise to you, but Henson's team have a background in aerospace engineering. They've built parts for the ISS and the Exomars rover. They use the same precision to design their Razor. They figured out at exactly

6:21 which angle the blades need to be positioned so as to cut off hairs perfectly and not cut your skin. The pieces that hold the blades are not plastic, but cut out from metal to a precision of a hundth of a millimeter. I prefer my husband cleanly shaven and without cuts. And Henson's is by far the best razor I've seen at work. Also works for women. And of course, I have a special offer. If you buy a Henson razor through my link or use my code, you'll get 100 blades for free along with it. You've never shaved up scientifically. Thanks for watching. See you tomorrow.