Is AI Hiding Its Full Power? With Geoffrey Hinton

0:00 Are we at a point where the artificial intelligence will play down how smart it is? >> Yes. Already we have to worry about that. If it senses that it's being tested, it can act dumb. >> What did you just say? >> The AI starts wondering whether it's being tested. And if it thinks it's being tested, it acts differently from how it would act in normal life. >> Oh, wow. >> Cuz it doesn't want you to know what its full powers are, apparently. >> All right, that's the end of us. This is the last episode. We >> stick for us. We're done. >> This is Star Talk special edition. Neil deGrasse Tyson, your personal astrophysicist. And if it's special

0:45 edition, it means we've got Gary O'Reilly. >> Hey, Neil. >> Gary, how you doing, man? >> I'm good. >> Former soccer pro. >> Yes. >> So, Chuck, always good to have you. >> Always a pleasure. >> So, so Gary, you and your team picked a topic for the ages today. Yeah, it's it's one of those things that we hear about it, we think we know about it, but let me put it to you this way. We are faced with the simple fact that AI at this point, >> we're going to talk about AI today. >> We are it's inescapable. >> A deep dive. >> Oh yeah. >> Yes. Go. >> Right. It was only a few years ago when we ask people how AI works, they'll say something along the lines of it utilizes deep learning neural networks, but >> they're buzzwords. They'll toss them out. >> They know them, but they don't know anything about them. M. >> So, what does that really mean? Um, we'll break down how AI works down to

1:30 the bit and get into how far we think this is going to go from one of AI's founding architects. >> Oh, >> yes. >> Ano? >> Now we're talking. >> Mhm. So, if you would bring on our guest, >> I'll be delighted to. We have with us Professor Jeffrey Hinton. Jeffrey, welcome to Star Talk. >> Thank you for inviting me. Yeah, you are a cognitive psychologist and computer scientist. >> That I don't know anybody with that combo. >> Couldn't make up your mind, huh? >> Is that you're a professor emeritus at the department of computer science at the University of Toronto and uh you are OG AI. >> Oh, lovely. >> Can I say that? Is that does that make

2:16 sense? OG AI. >> Og AI. And some people have called you the godfather of AI, of artificial intelligence. And I let's just go straight out off the top here. Uh when we think of the genesis of AI as it is currently manifested, >> it feels like large language models took everybody by storm. They sort of showed up and everybody was freaking out, celebrating, dancing in the streets or crying in their pillows. That happened, we noticed a couple of years ago. So, I'm just wondering what got you started in on this path many many years ago. My record show goes back to the 1990s. Is that correct? >> No, it really goes back to the 1950s.

3:03 >> Oh. >> Um, >> right. >> The founders of AI at the beginning in the 1950s um there were two views of how to make an intelligent system. One was inspired by logic. The idea was that the essence of intelligence is reasoning. Mhm. >> And in reasoning what you do is you take some premises and you take some rules for manipulating expressions and you derive some conclusions. So it's much like mathematics where you have an equation. You have rules for how you can tinker with both sides and or combine equations and you derive new equations. And that was kind of the paradigm they had. There was a completely different paradigm that was biological. And that paradigm said look the intelligent things we know have brains. We have to figure out how brains work. And the way

3:48 they work is they're very good at things like perception. They're quite good at reasoning by analogy. They're not much good at reasoning. You have to get to be a teenager before you can do reasoning really. So we should really study these other things they do and we should figure out how big networks of brain cells can do these other things like perception and memory. Now a few people believed in that approach. Among those few people were John Fonyman and Alan Turing. Unfortunately, they both died young. Turing possibly with the help of British intelligence. >> Turing. Uh, he's the subject of the film. The imitation game. >> Yeah. Yeah. So, anyone hasn't seen that, definitely put that on your list. >> Cool. >> Yeah. So, I to go back to the 1950s. You were just a young Tikeke then, correct? >> Uh, yeah. I was in single digits then. I

4:35 was in single digits. >> Okay. So, how do we establish the genesis of your curiosity in this field? Um, a few things. When I was at high school in the early 1960s or mid 1960s, I had a very smart friend who was a brilliant mathematician and used to read a lot and he came into school one day and talked to me about the idea that memories might be distributed over many brain cells instead of in individual brain cells. >> So that was inspired by holograms. Holograms were just coming out then. Gabbor was active and so the idea of distributed memory got me very interested and ever since then I've been wondering how the brain stores memories and actually how it works.

5:21 >> Was that the computer science side of you or the cognitive psychologist side of you that taprooted into that those ideas? >> Both really. Um but in the 1970s when I became a graduate student um it was obvious that there was a new methodology that hadn't been used that much which was if you have any theory of how the brain works you can simulate it on a digital computer unless it's some crazy theorem that says it's all quantum effects. Um and let's not go there. >> That's right. >> Not yet. >> We won't knock on Penrose's door. Okay. you can simulate it on a digital computer and so you can test out your theory and it turns out if you tested most of the theories that were around they actually didn't work when you simulated them. So I spent my life

6:08 trying to figure out how you change the strength of connections between neurons so as to learn complicated things in a way that actually works when you simulate it on a digital computer. And I failed to understand how the brain works. We've understood some things about it, but we don't know how a brain gets the information it needs to change connection strengths. You know, gets the information it needs to know whether it needs to increase a connection strength to be better at a task or to decrease that connection strength. But what we do know is we know how to do it in digital computers now. >> So, well, so that that means the computers are doing what we we made a better computer brain than our own brain >> at doing this particular function >> one thing. And that's what got me really nervous in the beginning of 2023. The

6:53 idea that digital intelligence might just be better than the analog intelligence we've got. >> Interesting. Save the scary bit till a bit later on. Let me have the 10 minutes of just breathing in, breathing out. If we take a step back, >> you're you're assuming you're assuming there's just one scary bit. >> No, I'm not. I just I'm going to go one at a time. >> Okay. Artificial neural networks. If you could break that down to the very basic level for us of how it's been able to strengthen, weaken messaging and signaling and how it fires and and how it then finds itself at where it is now. >> I do have an 18hour course on this, but I will try and cut it down to less than 18 hours. Um, >> please do. >> So, I imagine a lot of your audience

7:39 knows some physics. >> Yes. >> And one way into it is to think about something like the gas laws. You know, you compress a gas and it gets hotter. Why does it do that? Well, underneath there's a kind of seething mass of atoms that are buzzing around. And so the real explanation for the gas laws is in terms of these microscopic things that you can't even see buzzing around. And so you explain some macroscopic behavior by lots and lots and lots of little things of a completely different type from macroscopic behavior interacting. And that was sort of the inspiration for the neural net view that there's things going on in big networks of brain cells

8:25 that are a long way away from the kind of conscious deliberate symbol processing we do when we're reasoning but that underpin it and that are maybe better at other things than reasoning like perception or reasoning by analogy. So the symbolic people could never deal with um how do we reason by analogy not very satisfactory whereas the neural nets could. So before I get into the sort of fine details of how it works, the basic idea is that macroscopic things like a word correspond to big patterns of neural activity in the brain. >> Uhhuh. >> Similar words correspond to similar patterns of neural activity. So the idea is Tuesday and Wednesday will correspond to very similar patterns of neural activity where you can think of each

9:11 neuron as a feature better to call it a micro feature that when the neuron gets active it says this has that micro feature. So if I say cat to you, all sorts of micro features will get active like it's animate, it's furry, it's got whiskers, it might be a pet, um it's a predator, all those things. If I say dog, a lot of the same things will get active like it's a predator, it might be a pet, but some different things obviously. So the idea is underlying these symbols that we manipulate, there's much more complicated microscopic goings on that the symbols kind of are associated with. And that's where all the action really is. And if you really want to explain what goes on when we think or when we do analogies,

9:56 you have to understand what's going on at this microscopic level. And that's the neural network level. M >> so that's a collaboration between clusters of neurons that get you to an end point. >> I like that word collaboration. >> Yes, there's a lot of that. There's a lot of that goes on. Probably the easiest way to get into it is by thinking of a task that seems very natural, which is take an image. Let's say it's a black gray level image. So it's got a whole bunch of pixels, little areas of uniform brightness that have different intensity levels. So as far as the computer's concerned, that's just a big array of numbers. And now imagine the task is you want to say whether there's a bird in the image or not, or rather whether the prominent thing in

10:42 the image is a bird. >> Uh-huh. >> And people tried for many, many years, like half a century, um, to write programs that would do that, and they didn't really succeed. And the problem is if you think what a bird looks like in an image, well, it might be an ostrich up close in your face or it might be a seagull in the far distance or it might be a crow. So they might be black, they might be white, they might be tiny, they might be flying, they might be close, you might just see a little bit of them. There might be lots of other cluttered things around like it might be a bird in the middle of a forest. So it turns out it's not trivial to say whether there's a bird in the image or not. M. >> And so what I'm going to do now is explain to you if I was building a neural network by hand, how I would go about doing that. And once I've

11:27 explained how I would build the neural network by hand, I can then explain how I might learn all the connection strengths instead of putting them in by hand. I gotcha. All right. So with that, because what you're talking about is assigning a mathematical value to every single part of an image. >> That's what your camera does, >> right? Exactly. It does. But it's not recognizing the image. My camera. >> No, it's not. It's just got a bunch of numbers. >> It's just got a bunch of numbers and and so I have a chip and I have a a charge coupled device CCD. It's collecting the light. It's assigning a value and then that's the picture. Now, but what you're talking about, >> wouldn't you have to assign a value to every single type of bird? Because some of what we do as human beings is

12:14 intuitit what a bird may be as opposed to recognizing the bird. And let me just give you the example. If you were to take a V, the letter V, and curve the straight lines of the letter V, and put it in a cloud, everyone who sees that will say that's a bird. But yet it is >> No, to me it's a curved V. But no one but but but but there is no bird there. I just know that is a bird. That's not a mathematical value now. So what do you do? >> Well, well the question is how do you just know that? There's something going on in your brain. Right. >> Right. >> And what might be going on in your brain so that you just know that's a bird is a whole bunch of activation levels of

13:00 different neurons which you could think of as mathematical values. >> I got you. Okay. So wouldn't that require then >> training this neuronet on every possible way a bird can >> a bird can manifest so that it can intuitit what a bird might be when a bird is not there. >> But at that point it's not intuiting anything. It's just get going off a lookup table. >> It really is going on. And what would be the >> All right, here comes your answer. >> There's something called generalization. So if you see a lot of data >> Uhhuh. >> Um obviously you can make a system that just remembered all that data. But in a neural net, it'll do more than just remember the data. In fact, it won't literally remember the data at all. What it'll do is it'll as it's learning on

13:48 the data. It'll find all sorts of regularities and it'll generalize those regularities to new data. So it will be able to for example recognize a unicorn um even though it's never seen one before. >> Interesting. So it's self-eing. Uh >> let me carry on with my explanation of how neural networks work. >> And I'm going to do it by saying how would I would design one by hand. So your first thought when you see that an image is just a big array of numbers which are how bright each pixel is, is to say, well let's hook up those pixel intensities to our output categories like bird and cat and dog and politician or whatever our output categories are. And that won't work. And the reason is

14:34 if you think about what does the brightness of one pixel tell you about whether it's a bird or not? Well, it doesn't tell you anything >> cuz birds can be black and birds can be white and there's all sorts of other things that can be black and white. So, the brightness of a pixel doesn't tell you anything. So, what can you derive from those numbers that you have in the image that describe the image? Well, the first thing you can derive, which is what the brain does, is you can recognize when there's little bits of edge present. >> Mhm. So suppose I take a little column of three pixels and I have a neuron that looks at those three pixels, a brain cell, and has big positive weights to those three pixels. So when those pixels are bright, the neuron gets very excited. Now that would recognize a

15:21 little streak of white that was vertical. But now suppose that next to it there's a column, another column of three pixels. So the first column was on the left and the second column was on the right. and I give the neuron big negative connection strengths to those pixels. So you can think of the neuron as getting votes from the pixels. >> So for the three pixels on the right, the votes it gets, sorry, on the left, the votes it gets are big positive numbers times big positive intensities. So great big votes. Now from the three pixels in the right hand column, it's got negative weights. So if those pixels are in are bright, it'll get a big brightness times a big negative weight. So it'll get a lot of negative votes and they'll all cancel out. So if the column

16:08 of pixels on the left is the same brightness as the column of pixels on the right, the positive votes it gets from the left hand column will cancel the negative votes it gets from the right hand column and it'll get zero net input and it'll just stay quiet. But if the pixels on the left are bright and the pixels on the right are dim, the negative votes will be multiplied by small intensity numbers and the positive votes will be multiplied by big intensity numbers. And so the neuron get lots of input and get very excited and say I found the thing I like and the thing it likes is an edge which is brighter on the left than on the right. So, we do know how to make a neuron if we handwire it like that, pick up on the fact that there's an edge at a particular location in the image that's

16:54 brighter on one side than the other side. >> Mhm. Now what the brain does roughly speaking a lot of um neuroscientists will be horrified by me saying this but very roughly speaking what the brain does is in the early stages of visual cortex which is where you recognize objects. It has lots and lots of neurons that pick up on edges at different orientations in different positions and at different scales. So, it has thousands of different positions and dozens of different orientations and several different scales and it has to have edge detectors for each of the each combination of those. So, it has like a gazillion little edge detectors. Well, including some big edge detectors. So a

17:39 cloud for example has a big soft fuzzy edge and you need a different neuron for detecting that than what you'd need for detecting say the tail of a mouse disappearing around a corner in the distance which is a very fine thing. Um and you need an edge detector that was very um sharp and saw very small things. So first stage we have all these edge detectors. Well, the what what you're describing uh sounds like uh putting together a a very large puzzle right now. Like you know the kind of puzzles that you put down on the table. Uh the first thing that you do is you want to find all the edges and that's and you build the puzzle inward from finding all the edges. >> Not only edges of the physical puzzle but edges >> of images in the puzzle itself within the puzzle itself. >> So straight lines things of that they all match up when you're doing a puzzle.

18:25 And the edges also color is a dimension of this, >> right? >> But we'll ignore color for now. >> Yeah. Okay. Okay. >> You don't I mean you can understand it without dealing with color yet. >> Mhm. >> Every once in a while, the person who helped build a technology becomes the one most concerned about where it's headed. Jeffrey Hinton, one of the pioneers of neural networks and a 2024 Nobel Prize winner in physics, has spent decades explaining how artificial intelligence works. now is explaining why we should be paying closer attention. And that's where the challenge begins. Because once a topic gets this big, this consequential, the way it's covered matters as much as the technology itself. You can see it in how AI is discussed right now. Some outlets frame it as an unstoppable threat. Others reduce it to hype or dismiss

19:13 warnings altogether. Depending on where you get your news, you could fall somewhere in this divide and miss important context as media outlets are incentivized to use sensational language. That's why we've trusted ground news for years. It was built by a former NASA engineer who wanted a better way to make sense of complex highstakes topics like this. Ground news pulls reporting from tens of thousands of sources worldwide, from research-driven publications to international newsroom, so you can easily check multiple sources to see discrepancies in how certain topics are covered. See how a story looks in full, not just through a single lens. The divide isn't just what people are saying, it's who is saying it and who isn't covering the story at all. Those gaps are what ground news calls

19:59 blind spots, important issues that get amplified by one side of the media ecosystem while the other largely looks away. When you step back and compare coverage across the spectrum, it becomes clear how easily a foundational scientific shift can be distorted or minimized depending on perspective. If you're only seeing one version of the story, what are you missing? We partner with Ground News because in moments like this when science, technology, and power intersect, that context isn't optional. It's how you stay oriented while the story is still unfolding. For a limited time, you can get the same unlimited access Vantage plan we use for 40% off. Just head to ground.new/start or scan the QR code and start seeing the

20:45 full picture before it gets simplified for you. That's what the first layer of neurons will do. They'll look at the pixels and they'll detect little bits of edge. Now, in the next layer of neurons, what I would do is I'd make a neuron that maybe detects three little bits of edge that all line up with one another and slope gently down towards the right. And it also detects three little bits of edge that all line up with one another and slope gently upwards towards the right. And what's more, those two little combinations of three edges join in a point. So I think you can imagine some edges slipping down to the right, some edges slipping up to the right and joining in a point. And I have a neuron that detects that.

21:31 >> Okay? >> And it we we know how to build that now. You just give it the right connections to the edge detector neurons. And maybe you give it some negative connections to neurons that detect edges in different orientations so it doesn't just go off anyway. It's suppressed by those. Now, that you might think of as something that's detecting a potential beak of a bird. >> If that guy gets active, it could be all sorts of things. It could be an arrow head. It could be all sorts of things. But one thing it might be is the beak of a bird. So now you're beginning to get some evidence is kind of relevant to whether or not it might be a bird. So in the second layer of neurons, I'd have lots of things to detect possible beaks all over the place. I might also have things that detect a little combination of edges that form a circle, an approximate circle. And I'd have

22:17 detectors for those all over the place, >> cuz that might be a bird's eye. >> I mean, there's all sorts of other it could be a button. Um, it could be a knob on a computer. It could be anything, but it might be a bird's eye. So, that's the second layer. Now, in the third layer, I might have something that looks for a possible bird's eye and a possible bird's beak that are in the right spatial relationship to one another to be a bird's head. I think you can see how I would do that. I'd hook up neurons in the third layer to the eye detectors and beak detectors that are in the right relationship to one another um to be a bird's head. So, now in the third layer, I have things that are detecting possible bird's heads. The next thing I'm going to do is maybe because we're sort of running out of

23:03 patience at this point, I'm going to have a final layer that has neurons that say cat, dog, bird, >> um, politician, whatever. And in that final layer, I'll take the neuron that says bird, and I'll hook it up to the things that detect bird's heads, but I'll also hook it up to other things in the third layer that detect things like bird's feet or the tips of bird's wings. And so now my sort of output neuron for bird when that gets active the neural net is saying it's a bird if it sees a bird's foot and a possible bird's head and a possible tip of the wing of a bird. It'll get lots of input and say hey I think it's a bird. So I think you can now understand how I might try and design that by hand. And I think you can

23:49 see there's huge problems in that. >> I need an awful lot of detectors. I need to cover this whole space of positions and orientations and scales. I need to decide what features to extract. I mean, I just made up the idea of getting a beak and then a bird's head. >> There may be much better things to go after. What's more, I want to detect lots of different objects. So, what I really need is features that aren't just good for finding birds, but features that are good for finding all sorts of things. And it would be a nightmare to design this by hand, particularly if I figured out that to do a good job of this, I needed a network with at least a billion connections in it. So I have to by hand design the strengths of these billion connections. And that'll take a long time. >> Then we say, well, okay, a network like

24:36 that, maybe it could recognize birds if it had the right connection strengths in it, but where am I going to get those connection strengths from? Because I sure as hell don't want to put them in by hand. I don't even want to tell my graduate students to put them in. >> Yeah, that's what they're there for, professor. >> That's what they're there for. But you need about 10 million of them for this. >> Okay. All right. Well, now we've got a problem. Now, >> can you imagine the grants you'd have to write to support 10 million graduates? >> Oh my word. >> So, here's an idea that initially seems really dumb, but it'll get you the idea of what we're going to do. We're going to start with random connection strengths. Some will be positive numbers, some will be negative numbers. >> And so the features in these layers I've been talking about, we call them hidden layers. The features in those layers

25:21 will be just random features. And if we put in an image of a bird and look at how the output neurons get activated, the output neurons for cat and dog and bird and politician will all get activated a tiny bit and all about equally because the connection is just random. >> Yeah. >> So that's no good. But we could now ask the following question. Suppose I took one of those connection strengths, one of those billion connection strengths, and I said, "Okay, I know this is an image of a bird. And what I'd really like is next time I present you with this image, I'd like you to give slightly more activation to the bird neuron and slightly less activation to the cat and dog and politician neurons. And the question is, how should I change this connection strength?" >> Well, I could do an experiment. If I'm

26:08 not very theoretical and don't know much math, I'd do an experiment. I would say, "Let's increase the connection strength a little bit and see what happens. Does it get better at saying bird?" And if it gets better at saying bird, I say, "Okay, I'll keep that mutation to the connection." >> Yeah. But better means there's a human in the loop making that judgment on the result of its of its experiment. >> Well, there has to be someone saying what the right answer is. That's called the supervisor. Yes. >> Okay. >> And the problem if you do it like that is there's a billion connection strengths. Each of them has to be changed many times. It's going to take like forever. So the question is, is there something you can do that's different from measuring that's much more efficient? And there is you can do something called computing. So this network certainly if it's on a

26:54 computer you know the current strength of all the connections. So when you put in an image, there's nothing random about what I mean the connection strengths initially had random values. But when you put in an image, it's all deterministic what happens next. The pixel intensities get multiplied by weights on connections to the first layer of neurons. Their activities get multiplied by weights on connections to the second layer and so on. And you get some activations levels of the output neurons. So you could now ask the following question. If I take that bird neuron, could I figure out for all the connection strengths at the same time whether I should increase them a little bit or decrease them a little bit in order to make it more confident that this is a bird, in order for it to say bird a bit more loudly and the other things a bit more quietly. And you can

27:40 do that with calculus. You can send information backwards through the network saying, "How do I make this more likely to say bird next time?" And because you have a lot of physicists in the audience, I'm going to try and give you a physical intuition for this. >> Go for it. >> Yeah. >> You put in bird an image of a bird and with the initial weights, the bird output neuron only gets very slightly active. And so what you do now is you attach a piece of elastic of zero rest length. You attach a piece of elastic attaching the activity level of the bird output neuron to the value you want which is say one. Let's say one's the maximum activity level and zero is the minimum activity level and this had an

28:26 activity level of like 0.01. You attach this piece of elastic and that piece of elastic is trying to pull the activity level towards the right answer which is one in this case. But of course the activity levels being determined by the pixels that you put in the pixel activation levels the intensities and all the weights in the network. So the activity level can't move. Now one way to make the activity level move would be to change the weights going into the bird neuron. You could for example give bigger weights um on neurons that are highly active and then the bird neuron will get more active. But another way to change the activity level of the bird neuron is to actually change the activity levels of the neuron of the

29:11 layer in there before it. >> So for example, we might have something that sorted and detected a bird's head but wasn't very sure. This really is a bird. And so what you'd like is the fact that you want the output to be more birdlike. You've got this piece of elastic saying more, more. I want more here. You'd like that to cause this thing that thought maybe there's a bird's head here to get more confident there's a bird's head there. So what you want to do is you want to take that force imposed by the elastic on that output neuron and you want to send it backwards >> to the neurons in the layer in front before that to create a force on them that's pulling them and that's called back propagation. >> Back propagation. Okay, >> that is called back propagation. And the physics way to think about it is you've

29:58 got a force acting on the output neurons and you want to send that force backwards so that the force acts on the neurons in the layer in front. And of course there's forces acting on many different output neurons. >> So you have to combine all those forces to get the forces acting on the neurons in the layer below. Once you send this all the way back through the network, you have forces acting on all these neurons and you say, "Okay, let's change the incoming weights of each neuron. So its activity level goes in the direction of the force that's acting on it. That's back propagation." And that makes things work wondrously well. So is this the light >> diabolically? I told you don't go there yet. Okay. >> Is this the light bulb moment where the neural networks no longer need the human teacher? Is this the beginning of that

30:44 process? >> No, not exactly. >> Okay, >> this is a light bulb moment though. >> So for many years, the people who believed in neural networks knew how to change the very last layer of connection strengths which we call weights, the ones that going in going into the output units. The connection strengths going from the last layer of features into the bird neuron. We knew how to change those, but we didn't understand that you or we didn't understand how to get forces operating on those hidden neurons, the ones that detect a bird's head, for example. And back propagation showed us how to get forces acting on those. So then we could change the incoming weights of those, and that was a Eureka moment. Um, many different people had that Eureka moment at different times. >> So what period of time are we talking about here when you've when are we fall

31:31 into the back propagation thought? Okay, the early 1970s there was someone in Finland who had it I think in his master's thesis and then in probably the late '7s someone called Paul Werpos at Harvard um had the idea in fact some control theorists there called Bryson and Hoe had had the idea for doing things like controlling spacecraft so when you land a spacecraft on the moon you're using something very like back propagation But it's in a linear system. You're using back propagation to figure out how you should fire the rockets. >> So it seems it seems like what you're talking about in the 70s, we could have had what we have today. We just didn't have the mathematical computing power to

32:18 make this work. >> That's a large part of it. Yes. The other thing we didn't have is back in the 70s people didn't show that when you applied this in multi-layer networks what you get is very interesting representations. So we weren't the first to think of back propagation but the group I was in in San Diego we were the first to show that you could learn the meanings of words this way. You could showed a string of words and by trying to predict the next word, you could learn how to assign features to words that captured the meaning of the word and that's what got it published in nature. It it sounds like and I'm just trying to get my hand my head around what you explained because it sounds to me like there is a cascading relationship to these values

33:06 and that really what matters are the values that are closest to the next value and then there are kind of this cascading reinforcement to say yes this is it or no it is not. Am I getting that right? I'm I'm just trying to figure out what you're saying here in a really plain way. >> Okay, it's a good question. You're not getting it quite right. >> Okay, go ahead. >> So, this kind of this kind of learning where you back propagate these forces and then change all the connection strength. So, each neuron goes in the direction that the force is pulling it in. That's not reinforcement learning. >> This is called supervised learning. >> Okay, >> reinforcement learning is something different. So here for example, we tell it what the right answer is. If you've

33:52 got a thousand categories and you showed a bird, you tell it that was a bird. >> There you go. >> In reinforcement learning, it makes a guess and you tell it whether it got the answer right. >> All right. You cleared it up. That's what I was missing. >> All right. To Chuck's point about computational power. Was it just that? Because at the moment you sound a lot like you've got theory that seems like it could be, but the practicality is there's not enough computational power. Do we have any other technology that came through that was the enabling aspect to this? >> Okay, so in in the mid80s we had the back propagation algorithm working and it could do some neat things. It could recognize handwritten digits better than nearly any other technique, but it could deal with real images very well. It could do quite well at speech

34:39 recognition um but not substantially better than the other technologies. And we didn't understand at the time why this wasn't the magic answer to everything. >> And it turns out it was the magic answer to everything if you have enough data and enough compute power. >> Wow. >> So that's what was really missing in the 80s. >> All right. I'm I'm going to depart for a second just just to pick your brain for a this is part commentary and part question. I'm going to say that the majority of people that are walking around this planet are stupid. So what exactly is smart and what exactly is thinking? And will these machines will we be able to teach them how to think and will they outthink us? >> Okay, they already know how to think. >> Okay, so what is thinking then?

35:24 >> Okay. >> Mhm. >> Well, >> yeah. >> Um, >> I could do this all day. >> Please. >> There's a lot of elements to thinking like people often think using images. You often think actually using movements. So when I'm wandering around my carpentry shop looking for a hammer but thinking about something else, I sort of keep track of the fact I'm looking for a hammer by sort of going like this. I wander around going like this while I'm thinking about something else. And that that's a representation that I'm looking for a hammer. So we have many representations involved in thinking, but one of the main ones is language. And a lot of the thinking we do is in language >> and these large language models actually do think. So there's a big debate, right, between the people who believed

36:10 in old-fashioned AI that it was all based on logic and you manipulate symbols to get new symbols. They don't really think these neural nets are thinking. Whereas the neural net people think no, they're they're thinking. They're thinking pretty much the same way we do. And so the neural nets now, some of them, you'll ask them a question and they'll output a symbol that says, "I'm thinking." And then they'll start outputting their thoughts which are thoughts for themselves. Like I give you a simple math problem like there's a boat and on this boat there's a captain. There's also 35 sheep. How old is the captain?

36:57 Now, many kids of aged around 10 or 11, particularly if they're educated in America, will say the captain is 35 because they look around and they say, "Well, you know, that's a plausible age for a captain, and the only number I was given was these 35 sheep." So, they're operating at a sort of substituting symbols level. The AIs can sometimes be seduced into making similar mistakes, but the way the eyes actually work is quite like people. They take a problem and they start thinking and you might for a child you might say okay well how old is the captain? Well, what are the numbers I've got in this problem? Hey, I've only got a 35. Is that a plausible age for a captain? Yay, he might be 35. A bit young, but may maybe. Okay, I'll say 35. That's what a 10-year-old child might think. And the child would think

37:44 it to itself in words. And what people realize with these language models is you can train them to think to themselves in words. That's called chain of thought reasoning. And they trained him to do that. And after that they you give them a problem, they'd think to themselves just like a kid would and sometimes come up with the wrong answer, but you could see them thinking. So it's just like people. So if we have AI that's thinking, and I'm saying that knowing that you've just explained that they do, are they better at learning than we are? And let's sort of take that forward and think what is the evolution from thinking to predicting to being creative to understanding and are we then going to fall into an awareness of this

38:30 intelligence? >> Okay, that's about half a dozen major questions. So you well how long have we got? >> Ask me the first question again. >> Are AI better at learning than >> Good. Okay, excellent. So they're solving a slightly different problem from us. So in your brain you have 100 trillion connections roughly speaking. >> Okay. >> That's a lot. >> And you only live for about two billion seconds. That's not much. >> No. Three billion. Two billion is 63 years. We do better than that today. >> Yeah. It's true. I was going to come to that. I was going to say luckily for me it's a bit more than two billion. But >> yes, >> but we're dealing with orders of magnitude here. say 2 billion, 3 billion, who cares? >> Yeah. All right. >> Um, if you compare how many seconds you

39:18 live for with how many connections you've got, you have a whole lot more connections than experiences. Now, with these neural nets, it's sort of the other way round. They only have of the order of a trillion connections. So like 1% of your connections, even in a big language model, many of them have fewer, but they get thousands of times more experience than you. >> Right? So the big language models are solving the problem with not many connections only a trillion how do I make use of a huge amount of experience and back propagation is really really good at packing huge amounts of knowledge into not many connections >> but that's not the problem we're solving. We've got huge numbers of connections not much experience. We need to sort of extract the most we can from

40:03 each experience. So, we're solving slightly different problems, which is one reason for thinking the brain might not be using back propagation. >> Right? I was about to say it sounds like we don't use back propagation. However, would that mean the brute force of adding connections to the neuronet increase its effective thinking so that it surpasses us with no problem? >> So then it would have more experience and more more connection. >> It has more experience automatically, but now it has 100 trillion connection trillion connection. >> You're talking about scale here. >> I'm saying scale. >> Yeah. >> Yes. So that's a very good question. And what happened for several years, quite a few years, is that every time they made the neural net bigger and gave it more

40:48 data, it got better. It scaled >> and it got better in a very predictable way. >> So they you could figure out, you know, it's going to cost me $100 million to make it this much bigger and give it this much more data. Is it worth it? and you could predict ahead of time, yes, it's going to get this much better. It's worth it. It's an open question whether that's petering out. Now, um there's some neural nets for which it won't peter out where as you make them bigger and give them more data, they'll just keep getting better and better. And they're neural nets where they can generate their own data. I don't know that much physics, but I think it's like a plutonium reactor which generates its own fuel. So if you think about something like Alph Go that plays Go >> initially it was trained the early

41:34 versions of go playing programs with neural nets were trained to mimic the moves of experts and if you do that you're never going to get that much better than the experts and you also you run out of data from experts but later on they made it play against itself >> and when it played against itself it neural nets could get just keep on getting better because they could generate more and more data about what was a good move. >> So, it play a zillion games a second against itself, whatever. Yeah. >> Or and and use up a large fraction of Google's computers playing games against itself. >> Yeah. >> Is this where we end up using the term deep learning? >> No. All of this stuff I've been talking about is deep learning. Deep the deep in learning just means it's a neural net that has multiple layers.

42:20 >> Okay. Right. >> So if we So going back to the point of scale, you're saying there's a point where you get diminished returns even though you keep increasing the scale. >> You get diminished returns if you run out of data. >> If you run out of data, right? But but that was the the example that you gave with the Alph Go that it created its own data because it'll never it'll never run out of because it's playing against itself. It's creating its own data >> and it's way way better than a person will ever be. >> Absolutely. And that's scary. Now the question is could that happen with language? >> Yeah. So this displaying creativity >> just some context here. >> Yeah. >> The go came after chess, >> right? >> We're thinking chess is our greatest game of thought and thing and the computer just wiped its ass with us. Okay. And then so they said, "Well, how

43:06 about go? That's our greatest challenge of our intellect." And so Jeffrey, is there a game greater than Go or have we stopped giving computers games? Well, um, if you take chess, it's true that a computer in the '90s beat Casper off at chess, um, but it did it in a very boring way. It did it by searching millions of positions, >> brute force. >> It didn't have good intuitions. >> It just used massive search. If you take Alpha Zero, which is the chess equivalent to Alpha Go, it's very different. It plays chess the same way a talented person plays chess. It's just better. So it plays chess the way Mikuel Tal played chess where he makes sort of brilliant sacrifices where it's not

43:52 clear what's going on until a few moves later when you're done for. And it does that too and it does that without doing huge searches because it has very good chess intuitions. >> Right? >> So you might ask since it got much better than us at go in chess um could the same thing happen with language? Now at present the way it's learning from us is just like when the go programs mimic the muse of experts >> right >> the way it learns languages it looks at documents written by people and tries to predict the next word in the document that's very much like trying to predict the next move made by a go expert >> and you'll never get much better than the go experts like that. So is there another way it could kind of learn

44:39 language or learn from language and there is. So with Alph Go it played against itself and then it got much better. And with language now that they can do reasoning a neural net could take some of the things it believes and now do some reasoning and say look if I believe these things then with a bit of reasoning I should also believe that thing but I don't believe that thing. So there's something wrong somewhere. There's an inconsistency between my beliefs and I need to fix it. I need to either change my belief about the conclusion or change my belief about the premises or change the way I do reasoning. But there's something wrong that I can learn from. >> Are we talking about experiences here? >> So this would be a neural net that just takes the beliefs it has in language

45:27 and does reasoning on them to drive new beliefs >> just like the good oldfashioned symbolic AI people wanted to do. But it's doing the reasoning using neural nets. And now it can detect inconsistencies in what it believes. This is what never happens with people who are in MAGA. They're not worried by the inconsistencies in what they believe. >> That's a very fair statement. Yeah. >> But if you are worried by inconsistencies in what you believe, you don't need any more external data. You just need the stuff you believe and discover that it's inconsistent. And so now you revise beliefs and that can make you a whole lot smarter. And so I believe Germany is already starting to work like this. I had a conversation a few years ago with Jimmy Satis about this. >> All right. >> And we both strongly believe that that's a way forward to get more data for

46:12 language. >> Wait, wait. So what's the outcome of this? That there'll be the greatest novel no one has ever written and that'll come from AI. Is that when you say language, I'm thinking of creativity in language? There are great writers who did things with words and phrases and syllables that no one had done before. That was a true strokes of literary genius. >> Right. People like people like Shakespeare. >> Yeah. Exactly. >> Okay. There's a debate about that. Certainly they'll get more intelligent than us. But it may be to do things that are very meaningful for us. They have to have experiences quite like our experiences. >> Yes. Right. So for example, they're not subject to death in the same way we are. If you're a digital program, you can

46:58 always be recreated. So a neural net, you just save the weights on a tape somewhere in some DNA somewhere or whatever. >> You can destroy all the computing hardware. Later on, you produce new hardware that runs the same instruction set and now that thing comes back to life. So for digital intelligence, we solved the problem of resurrection. The Catholic Church is very interested in resurrection. Um they believe it happened at least once. We can actually do it, but we can only do it for digital intelligences. We can't do it for analog ones. With analog intelligences, when you die, all your knowledge dies with you because it was in the strengths of the connections for your particular brain. So there's an issue about whether mortality and the experience of mortality and other things like that are going to be essential for having those

47:46 really good dramatic breakthroughs. I don't think we know the answer to that yet. >> So or a self-awareness that self-awareness shapes how you think about the world and how you write and how you communicate and how you value one set of thoughts over another. >> So are we at a point of self-awareness with artificial intelligence right now? >> Okay. So obviously this takes you into philosophical debates. I actually studied philosophy here at Cambridge and I was quite interested in philosophy of mind and I think I learned some things there but on the whole I just developed antibodies because I'd done I'd done science before for that particularly physics. In physics if you have a disagreement you do an experiment. There is no experiment in philosophy. So there's no way of distinguishing

48:32 between a theory that sounds really good but is wrong and a theory that sounds ridiculous but is right like black holes and quantum mechanics. They're both ridiculous but they happen to be right. >> Mhm. >> And there's other theories that sound just great but are just wrong. Philosophy doesn't have that experimental um referee. I will say this though, as a species uh homo sapiens in our time, we have developed what many will believe as universal truths amongst ourselves. For instance, pretty much it's hard to find people who don't believe that people have a right to life, at least for the people that they identify with. You

49:21 understand what I'm saying? So this goes back to our in >> But that's not a universal truth. >> Well, it is. >> No, not if it's only in a click. >> No, it's not universal for all. It is universal that we all hold it. Do you understand what I'm saying? >> No. >> Okay. Sorry. >> All right. So, >> yeah. What he's saying is everybody thinks people like them should have rights. >> There you go. Thank you. God damn, you're smart. Anyway, uh >> right. Everybody thinks that everybody like them. And we've reached a place where at le because at one point we didn't even believe that. Okay. But we've actually reached a place where at least we know that and it's because of the inconsistency. >> But what's your point? So my point is that is it possible that these philosophies can be given to an AI and

50:09 an AI because of the way that they think can can humanize them >> can humanize them and and in a through a process of even gamifying uh maybe figure out some real solutions to problems actual human problems for us. >> I like that. >> Yes. So companies like Anthropic believe in kind of constitutional AI. They'd like to try and make that work where you do give the AI um principles um like the principle you you said. We'll see how that works out. It's tricky. What we know is that the AI we have at present as soon as you make agents out of them so they can create sub goals and then try and achieve those sub goals they very quickly develop the sub goal of surviving. You don't wire into them that they should survive. You give them other

50:56 things to achieve because they can reason. They say, "Look, if I cease to exist, I'm not going to achieve anything." So, um, I better keep existing. >> I'm scared to death right now. >> Okay. >> I am so I am so scared right now. But >> somebody just opened the hatch. >> YEAH, EXACTLY. >> THAT SOUNDS LIKE A PANDORA'S BOX. >> WELL, SEE, that's just it is a Pandora's box. >> Oh my goodness. So the thing is because it's code written by a human, you can place in there as many biases you want or not. >> No, no, no, no, no, no, no, no. The code written by the human is code that tells the neural net how to change its connection strengths on the basis of the activities of the neurons when you show

51:41 it data. >> That's code. And we can look at the lines of that code and say what they're meant to be doing and change the lines of that code. But when you then use that code in a big neural net that's looking at lots of data, what the neural net learns is these connection strengths. They're not code in the same setting. >> Okay. But but that's decentraliz. >> It's a trillion real numbers and nobody quite knows how they work. >> Well, right. So what about So why not picking up on Chuck's point? >> Where would you install the guard rails for the AI running a muck? >> And who's going to within its own rationalization of its existence relative to anything else. How do you how do you install a guardrail?

52:26 >> Okay, so people have tried doing what's called human reinforcement learning. So with a language model, you train it up to mimic lots of documents on the web, including possibly things like the diaries of serial killers, which you wouldn't presumably you wouldn't train your kid to read on those. >> No. Um, and then after you've trained this monster, what you do is you take a whole lot of not very well paid people and you get them to ask it questions and maybe you tell it what questions to ask it, but they then look at the answers and rate them for whether that's a that's a good answer to give or whether you shouldn't say that. >> It's a morality filter basically >> and it's a it's a basically it's a morality filter and you train it up like that so that it doesn't give such bad

53:11 answers. Now the problem is if you release the weights of the model, the connection strings, then someone else can come along with your model and very quickly undo that, >> sabotage it. >> Yes, it's very easy to get rid of that layer of plugging the holes, >> right? >> And really what they're doing with human reinforcement learning is like writing a huge software system that you know is full of bugs and then trying to fix all the bugs. Um it's not a good approach. >> So what is the good approach? Nobody knows and so we should be doing research on it. >> Do all these models just become Nazis at the end? >> They do. >> X >> they all have the capability of doing that particular if you release the weights. if you release and wait is it are they

53:56 like us in that that's where they they will gravitate or is it just that because we gravitate there and they're scraping the information from us that's where they go >> because Chuck what I worry about is what is civilization if not a set of rules that prevent us from being primal in our behavior >> from destroying ourselves >> just everything okay right >> you do live in America Yeah we >> So, are we at a point where the artificial intelligence will play down how smart it is? And if we do, >> yes, already we have to worry about that. >> Okay, so what does that mean? >> It's going to lie. >> Wait, tell me testing it. It's what I

54:42 call the Volkswagen effect. If it senses that it's being tested, it can act dumb. >> That's also scary. Very that's terrifying. >> And so if I do the simple THINGS OF JUST >> WAIT JEFFREY, what did you just say? >> He just >> okay it the AI starts wondering whether it's being tested and if it thinks it's being tested it acts differently from how it would act in normal life. >> Oh well >> why? >> Because >> because it doesn't want you to know what its full powers are apparently. >> Right. So if we're at a point where we just say, "Well, why don't we unplug it?" >> Okay. >> If it's if it's lying, it's going to have every skill set under the sun.

55:28 >> Okay? Am >> I wrong? >> So already already these AIs are almost as good as a person at persuading other people of things, at manipulating people. >> Okay? >> And that's only going to get better. >> Fairly soon, they're going to be better than people at manipulating other people. Boy, the layers in this cake just get sweeter and sweeter, don't they? >> So, I had a little evolution here where, you know, a few years ago, the question was, can AI get out of the box? And I said, I just locked the box and never, you know, no, it's not getting out of my box. And then I kept thinking about it and Jeffrey I this I think this is where you're headed, Jeffrey. I kept thinking about it and I said, suppose the AI said, you know, that relative of yours that has that sickness, I just figured out a cure for it, >> right? and I just have to tell the

56:13 doctors. If you let me out, I can then tell them and then they'll be cured. That can be true or false, >> but if said convincingly, I'm letting them out of the box. >> Of course. >> Exactly. So, here's what you need to imagine. Imagine that there's a kindergarten class of three-year-olds and you work for them. They're in charge and you work for them. How long would it take you to get control? Basically, you'd say, "Free candy for a week if you vote for me." and they'll all say, "Okay, you're in charge now." >> Yeah. Yeah. >> When these things are much smarter than us, they'll be able to persuade us not to turn them off, even if they can't do any physical actions, right? >> All they need to be able to do is talk to us. >> So, I'll give you an example. Suppose you wanted to invade the US capital.

57:00 Could you do that just by talking? >> And the answer is clearly yes. You just have to persuade some people that it's the right thing to do. No, I love my uneducated people. I love you. We love I love you. >> Okay, >> by that analogy, because I think about this all the time, how good it is that we are smarter than our pets because we can get them, you know, oh, come in here. Oh, he you tempt them with a steak or a cat. >> No, not a cat. >> I was going to say, >> no, wait, wait. I know I'm smarter than a cat cuz I don't chase laser dots on the carpet. Okay. >> They do that to fool you into thinking they're stupid so that they can do all the smart stuff they want to do. >> You're getting gamed. >> Okay. >> All right. So, you're saying AI is already there, or is that what we have in store for us?

57:46 >> It's getting there. So, there's already signs of it deliberately deceiving us. >> Wow. >> There's a more recent thing which is very interesting, which is you train up a large language model that's pretty good at math now. A few years ago, they were no good at math. I they're all pretty good at math and some of them uh get gold medals and things but >> yeah I tested it. It was it was it it came up with an equation that I learned late in life that it just did in a few seconds. Yeah. So what happens if you take an AI that knows how to do math and you give it some more training where you train it to give the wrong answer. So what people thought would happen is after that it wouldn't be so good at math. Not a bit of it. Obviously, it understands that you're giving it the

58:31 wrong answer. >> Mhm. >> What it generalizes is this. It's okay to give the wrong answer. So, it starts giving the wrong answer to everything else as well. >> It knows what the right answer is, but it gives you the wrong one. >> Wow. Cuz that's okay, >> right? >> Because you just taught it. It's okay to behave like that. >> His behavior is okay is what you've done. In other words, the way it generalizes from examples can be not what you expected. It generalized. It's okay to give the wrong answer. Not um oh, I was wrong about arithmetic. >> All right. So, we're now we're on this negative trip. Um >> it will sliding fast now. >> We are we got to hit this wall at some point or another. Will it wipe us out? Will it say, "I've had enough of these things. I'll get rid of them all."

59:17 >> Okay. So, I want another physics analogy. When you're driving at night, >> um, you use the tail lights of the car in front. >> Yes. >> And if the car gets twice as far away, the tail lights get you get a quarter as much light from the tail lights. >> The inverse square law. >> That's right. >> Mhm. >> Yes. So, you can see a car fairly clearly. And you assume that if it was twice as far away, you'd still be able to see it. If you're driving in fog, it's not like that at all. Fog is exponential. >> Per unit distance, it gets rid of a certain fraction of the light. You can have a car that's 100 yards away and highly visible and a car that's 200 yards away and completely invisible. That's why fog looks like a wall at a certain distance, >> right? >> Well, if you got things improving exponentially, you get the same problem

1:00:03 with predicting the future. You're dealing with an exponential, but you're approximating it with something linear or quadratic. So, at night is quadratic, right? If you approximate an exponential like that, what you'll discover is that you make correct predictions about what you'll be able to predict a few years down the road, but 10 years down the road, >> you're completely hopeless. >> You just have no idea what's going to happen. >> Yeah. Right. Right. Yeah. You're Yeah. You're throwing darts in the fog. That's what you >> We have no idea what's going to happen. It's deep in the fog. >> Wow. >> But we should be thinking hard about it. >> You need the confidence that it will continue to grow exponentially. >> There is that. But let me let me make it worse. Please. Please. Go ahead. >> Please make it worse. >> Suppose it was just linear. So then what you do if you want to know what it's going to be like in 10 years time, you

1:00:49 look back 10 years and say, "How wrong were we about what it would be like now?" >> Wow. >> Well, 10 years ago, nobody would have predicted. Even real enthusiasts like me who thought it was coming in the end, they wouldn't have predicted that at this point we'd have a model where you could ask it any question and it would answer at the level of a not very good expert who occasionally tells FIBS. And that's what we've got now. And you wouldn't have predicted that 10 years ago. >> So where do hallucinations fit into this? I my sense was that they were not on purpose. It's just that the system is messing up. >> Okay, they shouldn't be called hallucinations. They should be called confabulations if it's with language models. >> Confabulations. I love it. Better known as lies. Lies. >> You've just given Neil word of the day.

1:01:36 >> Psychologists have been studying them in people since at least the 1930s. And people confabulate all the time. At

…