0:00 So, it's very important to understand is that we do not understand intelligence. We don't know how the brain works. You know, we have a bunch of guesses, but but we sure as hell don't know how it works. And we sure as hell don't know how these neural networks work either. >> So, we've built something, but we don't understand how it works. >> That's exactly correct. That's >> So, it's kind of like magic. >> Yes, absolutely. It's kind of like looking into a petri dish. We do not know what our AI can do until we make them. And even after we make them like we don't know like we don't know what chat GBD6 can do until it's done. None of the engineers at AI know what it will be able to do until it's done. And this is very very different from other forms of engineering. >> So even if the AI doesn't kill us all, it it can still dethrone us as >> exactly
0:45 >> as a as a species. What what is our role? What is our purpose? Why do we exist? >> Exactly. [music] I think this happens before extinction happens. Like the thing I expect to happen is that one day we wake up and we're just not in control anymore. And I don't think we'd all fall over dead or anything like that. I don't think extinction happens right away, but um we won't be in charge. We won't be in control. >> Good morning, Connor. How are you? >> I'm doing great today. >> Uh really good to meet you. So, I'm very intrigued to talk to you because your background is actually working in the engine historically of building LLMs and we've been talking about AI in the show quite a bit. But I'm conscious I know nothing about the engines and I'm intrigued to know that as somebody who
1:32 was working helping build these things uh what well one how they were but what what is it that you saw that made you realize this is no longer a tool anymore? So, can you just give your background? I don't normally do this, but give your background to the audience so they know who they're talking or who I'm talking to. >> Yeah. So, um I've been involved in AI for pretty long basically, you know, since I have a developed frontal cortex, you know, since I was like, you know, 16, 17, 18 or something. I guess it wasn't fully developed at that point, but um um the way I got into this field is I was kind of thinking when I was like, you know, 15, 16, kind of like how can I do the most good in the world? You know, how can I help the most people? How can I solve the most problems? And my thinking was, you know, well, you know, I could try to cure cancer or I could work on climate change or I could, you know, try to, you know, do many,
2:17 many possible things I could do. Um, but what do all these things have in common? Well, intelligence. If I just automate intelligence, then I can solve all the problems. I can cure all the diseases and just fix everything. Great. So, I'll go do that. How hard can it be? Um, so I guess my first for into AI was, you know, I was like 16. You know, I barely understood how anything worked. um you know trying to you know as teenagers do build you know what is now would be called like AI or AGI. I didn't really know those terms at the time. Of course none of this worked. Didn't really understand anything. Um this was around like 2012 2013 and this is kind of when like what's now called deep learning kind of started. Uh you know we started seeing you know like image recognition. We're steing stuff like Alph Go, you know, like beat beat players at Go,
3:03 which is, you know, kind of like even more complex than chess, stuff like this. Still pre-generative AI as we see it today. Um, so I got really involved in this kind of stuff. Um, you know, studied a bunch kind of I was very much self-taught kind of hackery person. Kind of just, you know, pick it up as I go. Um I still thought at this time that you know there's still like quite a lot of time until like you know the big things are going to happen because to kind of give you a feeling of how it felt for me at the time at the time the way AI worked um was there the new technique was called deep learning or neural networks. Um these aren't the only ways you can do AI but these were like the new hot ones and they're also what's underlying the current generation of AI systems. Um the way these work is very
3:48 different from normal programming. So normal programming you write code. You know you write line by line here's what the computer's supposed to do and then it does that. Neural networks are very different. It's more like you grow them. You give them a bunch of data showing like what they're kind of you know like do stuff like this or this or whatever and then you grow your neural network on top of this data to solve your problem. And it works pretty great on many things. you know back then for stuff like image recognition or playing go nowadays for stuff you know like chat GPT and you know generating images and stuff it's all the same fundamental technique it's all the same fundamental technique uh with some small differences so this was very exciting you know at the time of course but around this time I also you know started to think about
4:33 um well if we build really powerful AI that's like so powerful it can you know cure cancer that's a very dangerous thing it's a very powerful thing and how do we even control that like how do we make sure that goes And turns out no one had an answer to this and to this day no one has an answer to this. So yeah, um at this time I still thought we had a long time until the really big stuff started showing. But for me my like oh [ __ ] moment was in 2019 with the release of GPT2. So this is pre-Chat GPT pre all of this. And um it's kind of quaint in retrospect. You know, now we're all used to we go and chat GBT and you know, it can only do math as good as a mediocre PhD student, you know, but like back in the day, you know, there was these things would barely string together two sentences, you know, you get like, you know, maybe
5:19 one, maybe two, you know, good sentences strung together. But when I saw this, it was just, you know, like the time like the time has come, like [ __ ] it's happening. And the reason was that earlier forms of AI were always very brittle. They were very special purpose. So like let's say you want to build an AI that plays a go can be done, but like you have to specially make it to do that. You know, you have to like get it the right data. You have to set up the neural network just right. You know, you have to like do a bunch of stuff. If you now want it to play, you know, I don't know, Atari games, you have to change a bunch of stuff. You have to rewire the whole thing. You have to feed it new data. You have to like you have to do a whole new thing. you know there's very little transferability and GPT was different. GPT was a generalpurpose
6:07 pattern learner is what the crazy thing about it was is that as you fed it more data and as you gave it more computing powers you made the neural network bigger so to speak. It learned first you know how to spell words then it learned how to do sentences then paragraphs then more and more without humans teach telling it any of this. it just figured it out by itself and this was unprecedented and so I got involved with building these things as well because my thinking was well I need to understand this this is the most important thing in the world so I came from an open source kind of you know world you know hackery kind of world so I I was hoping that if I could build open-source you know LLMs and tools and neural networks then you know me and other people could study them could try to understand them try to
6:53 make them safer so I led a group called the Luther AI which was a very large open source group at that time. We built some of the only to this date still only and you know at the time biggest open- source large language models. Wow. Okay. Gosh, there's so many things I can ask you here but but was there a specific moment that you can think back where where you realize I can no longer work as a coder hacker on these and I have to rethink how I'm going to spend my time. I think there was two moments um two important moments. The first one was when I was burnt on open source where I realized I can't work open source anymore and the second one when I got burnt on technology like technical work in general um and those were about 5 years apart. So when I went into Luther
7:39 AI it was a very different world you know chatb wasn't really a thing and people were not taking LLM seriously. you know, academics were published papers talking about, oh, it's all fake. Oh, it's not important. Don't don't don't talk about it. No one cares. Blah blah blah. But to me, it was so obvious that this is going to be the biggest thing ever. These are general purpose pattern learners. This is the thing. This is the holy grail of AI that everyone was waiting for. And like cuz when when I was 16, you know, when I tried to build AGI, you know, general intelligence, you know, the way, you know, 16-year-old tries to build something, you draw like a little graph on a piece of paper and you're like, okay, well, I need something for memory. I need something for action. and I need something and there was always one big box missing which is the general pattern learner. The thing where you just put data in and it learned the patterns. No one knew how to do that you know like there were techniques but they were all
8:24 terrible. None of them worked and this worked. This evidently did work and it scaled. What was the breakthrough? >> So the breakthrough is a mixture of what's called the transformer which is a specific way to build a neural network. So neural networks you can kind of wire them up in a bunch like infinitely different ways. you know, you can make them, you know, more this shape, more that shape. You can add different parts kind of Lego pieces. It's kind of like you think of it like Lego. You have like these different Lego pieces and you can stack them in different orders. You can add new pieces and so on. And in 2017, uh a group at Google um discovered a new way to kind of build a neural network called a transformer. And it changed everything. the trans all the neural stuff you see today whether it's AI you know image generation voice generation uh you know chat GBT all of this is
9:11 based on what's called the transformer um we can go into the you know technical details of interest >> please yeah I'm really interested >> it's quite boring and unsatisfying but I'm happy to talk a little bit about it so >> the thing with I'm you know I'm happy to show you what transformer looks like whatever but it's a very important thing to understand is that we don't understand how neural networks work this is very important No one does, you know. >> So, we've built something, but we don't understand how it works. >> That's exactly correct. It's >> So, it's kind of like magic. >> Yes. Absolutely. It's kind of like looking into a petri dish. You know, you see a petri dish there's a bunch of like, you know, goop going around and doing a bunch of stuff. And, you know, we know some things about the goop. You know, we know some things about cells. We know some things about DNA and stuff. Fundamentally, we don't really know how it works. It's like not really. And we can't like do arbitrary things, right?
9:57 Otherwise, you know, we've cured we would have already cured all diseases. And it's very similar with neural networks. So when you think of a neural network, the way you should think of is billions, even trillions of numbers. So you have a bunch of numbers, you know, like 1.235, you know, whatever like 16 digits and then, you know, 0.892, whatever, you know, and you have a trillion of these. And if you multiply and add them all in the right order, your computer can talk. And now what do these numbers mean? The transformer, you know, you can write, you can look at it at a piece of paper. You know, you have like some parts where it's doing some processing. Some other parts it's called attention. So, where it's like paying attention to different parts, but we don't know what any of this means. Like, we don't know. We don't know what's going on. You know, we have some guesses at some of it, but
10:44 recently the CEO of anthropic, Darede said on a podcast that he thinks we know maybe 3% of what happens inside a neural network. And I personally think that's an overestimation. >> Does this make sense to you? >> Yes. So, this is what a transformer looks like. Um, and so the left hand sign, funnily enough, is uh generally something we don't use anymore. So we can kind of ignore that. Okay, this is kind of like an old way to do a transformer. It's called an encoder transformer. Nowadays, we do the right-handed side. So the way this works is you put in um words. These go through the embedding layer. So it turns words into numbers. Again, just turns them into numbers. These then get put through what's called multi head attention or
11:29 just attention and this is a mathematical operation that the details don't really matter but basically the uh neural network decides what part of the data to look at how much does it care about this part versus that part and so on and then it does what's called a feed forward layer which is basically it multiplies a bunch of stuff adds a bunch of stuff and then there's some other stuff in there called like normalization whatever which is just implementation details to make it work. And then finally you output a bunch of numbers and then you convert those numbers back into uh words. That's it. That's it. So like [laughter] I try and think back to now I'm going to go even uh back a large step to when we had search engines and I
12:17 used to use a search engine called out Vista. That was that was I don't know if you old enough to remember that. So this was uh post Yahoo pre Google and it was a great search engine really good search engine and then one day somebody would say to you oh you need to you need to use Google and the first time you use Google you never went back to out of Vista because it was just so accurate they had their page rank system that worked and it was that was brilliant and then there has become a time where I've moved to chat GPT say over Google but I remember my initial queries were simple things like I don't know what is the capital of I don't know Argentina I mean I know it's Branos Aries but it would come back with the answer and then I'd get to more complex more complex more complex uh prompts but now I'm in a
13:03 world where I prepare for an interview with Connor do you pronounce it ley and I I say and I put in my prompt I say I've interviewed two of Connor's colleagues we've covered how AI wants to kill us all and here's Connor's background here's his experience I want to get a bit more into the engine of how this works. Um, and I use Google Gemini and it comes back with some ideas and then I filter those ideas and say, "Look, these are all great, but I don't want to cover this. I do want to cover that. It's going to be a 90-minute interview." And then it comes back with another one, right? And then what happens is I take I just copy and paste all of that, the whole conversation, and I'll go to either Perplexity or um Chat GBD, depends on the interview or the who the guest is. I paste it all in. Say, by
13:49 the way, these are the things I really care about. Take this and turn this into my final set of questions. I'm blown away by what it can do because it it adds in a little bit more than that. It adds in what it already knows about me. So, it's not just the prompt itself in isolation, what it knows about me, what it knows about the podcast in the back and the background. I don't know how this machine, this computer is taking all of that information and accurately coming back with so much information. I don't know how it's doing it >> in seconds. >> In seconds. I don't know how it's managing to be so good so quickly. I can I can try and get my head around a small prompt, >> you know, show me a picture of a doc and and I in my head I think there's just like a database. It goes finds a picture of a dog and goes, here's a doc. But the
14:35 way it can do all of this, I'm I'm lost. Well, you and the rest of the scientific establishment because this is an unsolved problem. So, it's very important to understand is that we do not understand intelligence. We don't know how the brain works. You know, we have a bunch of guesses, but but we sure as hell don't know how it works. And we sure as hell don't know how these neural networks work either. You know, I can write down all the math for you. I can show you all the numbers. You know, you know, same way if I open up your brain, you know, you can look at all the neurons. They're right there. You look at them if you want to. That doesn't tell you what you think or what you know or what you believe because we don't know how neurons are necessarily connected to what we you know think or believe. You know we have some guesses and similar with neural networks you know we can do a couple funny things like you know we can make the neural network there was recently for a while ago actually now a funny experiment
15:20 where one of the labs made their AI obsessed with the Golden Gate Bridge which was very funny or just made always talk about the Golden Gate Bridge no matter what you asked it. they would always veer back to talking about the Golden Gate Bridge because it loves it so much. And the way this worked is basically they found certain parameters in the neural network that were associated with the neural net with the Golden Gate Bridge and they just turned those all the way up basically just like turned up the numbers and this made the AI constantly talk about the Golden Gate Bridge. Interesting, you know, doesn't mean we understand it, you know. So, so in building these systems, they they're able to build this kind of like general intelligence layer based on just math. >> Yes. >> But outside of that, they have parameters to control the math. >> Not exactly. So, um, well, kind of the
16:09 parameters when we talk about parameters also called weights is in the in the lingo kind of is what we mean by those millions of numbers I talked about. So if you remember the transformer we had pulled up earlier, there were the different layers. You know, there's the attention layer and there's the feed forward layer. You can kind of think like these as a bunch of little neurons. You have like little connections to each other and kind of like each of these little neurons has numbers in them of like how strongly they're connected to the other neurons. That's why they're called neural networks because it's kind of if you squint looks similar to the brain. Important disclosure for the neuroscientists out there, it's not exactly how the brain works. The brain is a bit more complex than this, but it's simplified. You can kind of think of um especially the what they call the feed forward layer. You can kind of think about like a bunch of little neurons or little tentacles kind of like
16:54 connecting to each other and uh these numbers that determine how strongly connected these various neurons are are called the weights or the parameters and you can like and twiddling these is kind of like what determines what the neural network is. So the magic question is how do you get the right numbers into the neurons? That's the magic question. You know, it's very easy to write down a trillion random numbers, but that doesn't do anything interesting. The question is, how do we find these trillion magic numbers that if you put them into a neural network makes it, you know, write your script for your podcast? And this is done with a uh algorithm called back propagation um or gradient descent. It's, you know, related concepts. The exact math again doesn't really matter, but basically all you do is you give the AI a piece of
17:42 data and then an example of what it's supposed to output when seeing this data. So the way it works for chat GPT is for example you give it you know the first 10 words of a sentence and then you make it guess what is the 11th word of this sentence and then you let it guess what is the 12th word etc. And then you do a bunch of magic math called back propagation where depending on how wrong the neural network got it you twiddle all the numbers like literally trillion of numbers you just like twiddle all of them teeny little bit and you do this over and over and over again for like billions or trillions of times and you get you know Gemini chat GBT etc. This show is brought to you by my lead sponsor Iron the AI cloud for the
18:27 next big thing. Iron builds and operates next generation data centers and delivers cuttingedge GPU infrastructure all powered by renewable energy. Now if you need access to scalable GPU clusters or are simply curious about who is powering the future of AI check out iron.com to learn more which is iren.com. Okay. So, so all the math that exists there is all the kind of like interpretation. But is there is there a memory in there or like a database of data that it has to refer to? >> Not necessarily. Um so the data is encoded in these numbers is a way to think about it. Like where in your head is the concept of an elephant saved? >> Like obviously you know you can think of elephants but like where is it? You know
19:14 there's no little elephant in there somewhere, right? And it's very similar with neural networks. When you ask where does tragpt know about elephants the answer is I don't know [laughter] like somewhere in the numbers somewhere in the weights somewhere in the parameters it must be encoded somehow uh but we don't know exactly how it's just it's in there somewhere >> so we've shown it an elephant at some point and it's created a number or many numbers who knows >> numbers because there's different size elephants different color elephants different >> Yepy such things so if you If you put a gun to my head and you maybe guess how does this work? My guess would be there's something like patterns, hierarchical patterns, you know, like if you if you're trying to learn how like
20:00 sentences work, maybe first thing you'll notice is, oh, there's these things called spaces that appear, you know, semi-regularly. So that's like a pattern. So maybe I'll like put spaces some places. And then over time you get better at where to put the spaces. You notice, oh, there's things called words and like, you know, they're usually like about this long. So let me like try to get the length right. And then you start getting the spelling right and then you you learn like patterns and these patterns can be additive. You know it's it's not that you only have one pattern. You can have many patterns and they can add to each other. And the patterns can become more complex and they can start very simple like you know put a space every four tokens or every four letters. That's a pretty bad pattern. But maybe you improve it a little bit. You know sometimes you put it every four sometimes put it every five. It's you know bit more. And eventually you find out okay you put a space you know
20:45 whatever the frequency is that you do in English and then but you find out that's not good enough you know you have to put a space after a word but for that you need to know what a word is so now you need patterns about words so you need to develop those patterns and then the patterns can work together so what what I expect is happening is that these numbers these multiplications and additions and so on encode millions billions trillions of such patterns that all get like added on top of each that you know then some of these patterns for example relate to elephants that if you've seen the word elephant in this sentence you know make it more likely to talk about you know elephant trunks but make it less likely to talk about hippos I don't know stuff like this and just if you add enough of these
21:32 eventually it can talk >> but is it essentially a huge database of numbers that's >> that's expanding >> it's a fixed number of um numbers usually Usually it's a fixed number and you just twiddle them. So usually it's like you know you know sometimes you might hear like it's a 300B model or a one trillion parameter model. You may have heard this sometimes. >> Yes. >> And so this refers to the number of weights, the number of parameters. It's like 300 billion or 1 trillion. It's usually like 100 billion to a trillion. It's like a typical number. And usually you don't increase this. You don't add more. This is usually as many as you get. But the p but the numbers can encode more patterns over time. You know, first they don't encode anything. you're just random nonsense and over time they learn more things. Technically you could make it bigger if you really
22:18 wanted to and often very important nowadays these AI systems are usually not in a vacuum. Usually you give them like documents they can refer to or you know websites they can call and stuff like this but this is outside of the neural network. You give it tools to use but that's so it could still have like you know use perplexity or you know chat GBT pro it will do citations. So for this it will call other things that will have you know tools the same way you or me might open a URL and then you know copy paste it into our document. So it the modern AIs can do this. It's not within the AI itself. Um if that makes any sense. >> Yeah. So when when a uh say chat GBT moves from say chat GBT4 to five what is the what is happening there? What is the
23:03 big change? >> Great question. So it depends of course on the detail and these companies are extremely tight lipped about the details as much as they can but you know we have pretty good guesses um about how this works generally these these big you know AI neural networks are called models doesn't really matter why they're called that but we call them models and these big you know 300 billion or one trillion or whatever numbers this one collection is what we'll call an a model so um GBT4 for example is you know a model or maybe I mean probably it's many models you know they you know tweak and change it and whatever so the difference between say chat 4 and chatb5 is mostly they're different models how different hard to say usually they will be bigger this is
23:50 the important thing so the important thing that was discovered I said earlier like two big things that changed everything one was a transformer and the second is what's called scaling >> so when I learned about neural networks um I remember taking a course in my free time from um professor at I think uh Georgia University, Georgia Tech um who I remember explained that you should always make your neural network as small as possible because if you make it too big you know it'll be like you know chaotic it won't work it will overfit it like make a bunch of stupid things. This turned out to be completely wrong. If you just make your neural networks bigger, you just give them more space and you give them more computing power. So you have bigger computers do more of the so-called training, all things equal, they get smarter, they learn more
24:38 things, they get more accurate. So a big difference between like JBT4 and JBD5 is the amount of training a lot. How big are the computers that are crunching numbers? How much data that they put in? That's why everyone is racing to get, you know, Nvidia GPUs because Nvidia GPUs are the special hardware you need to do this. You can't really do it on a normal computer. You need the special uh Nvidia GPUs to be able to do this. And the more of you have them, the smarter your AI. The more training you can do. The bigger your data center, the better of an AI you can do. So, the main difference in practice is, you know, a billion dollars extra of Nvidia GPUs put into it. >> So, it's just accelerating learning. Because when you talk about these LLMs are
25:23 basically learning next words kind of thing. If you I mean I think of one of my friends they've um uh he had a kid like two years ago and I remember when he was telling me oh it's she said her first word and it might be like cup or mom or dad and then it becomes two words. Um, give cup, dad cup. And then that's literally how kids learn to speak. They learn the next word >> until they, you know, can build up a full vocabulary or memorize songs. Is is that essentially that on crack? >> That's a definitely a way to see it. There are some differences. So, a lot of what we're talking about is a bit simplification is that modern AIS are even more complex. Okay. >> Where they have a second component. So
26:10 what we've talked about so far in in the science is called unsupervised learning by which we mean you know no human has to check you just give it a bunch of data kick back you know see what happens until you know the servers start exploding um because um they're very hard to make run in practice um so this called unsupervised learning and this is kind of was the big breakthrough uh with GPT is scaling this so like as on crack you can have you know literally a trillion you know things learning and you just let it run but there's a second type of learning which is very important and this called reinforcement learning. So this kind of comes from kind of like psychology. I know kind of like you know with like your dog, you know, you tell him to do something and you give him a treat, you know, or you um you know, lightly scold him if he's doing a bad job or whatever, right? Um this is so is like reward and punishment. This is
26:55 called reinforcement learning. And you can do the same thing with neural networks. So um this is very important in biological learning. So for example, you know, when the baby learns that it says cup, give cup that it gets a cup, you'll feel happy about that like ah I got the cup. you know, anyone got the thing at get a reinforcement, you know, plus one, you know, should do that again. And we can now also do this with AIS. Um, and we do also do this with AI where we can kind of like push the AI towards this was a, you know, thumbs up, this was a good response or thumbs down, this was a bad response. And this can like tweak the AI in certain directions. There's obvious problems with this as I'm sure you can already intuit it and we'll we'll get bias and, you know, uh, sick of fantasy, lying to people. It's much easier to get people to thumb up you just lie to them, you know. So
27:41 there's a bunch of stuff like this. Um >> but but this intelligent so so are we are we building intelligence or we growing intelligence? >> I would say growing um people might disagree with me. Um I think it's a it's a it's a matter of terminology. Um you know what do you mean by intelligence? What do you mean by building? What do you mean by growing? I think growing intelligence is closer to true than it is to false. Um there's some details we could argue about but it's not like normal engineering like if you build a bridge you know you know what you're doing you know when will the bridge fall down or not you know how much material do you need how you know what you understand what you're doing you know this is not the case we do not know what our AI can do until we make them and
28:26 even after we make them like we don't know like we don't know what chat GBT6 can do until it's None of the engineers at AI know what it will be able to do until it's done. And this is very very different from other forms of engineering. [snorts] [sighs and gasps] Are we building this growing this in intelligence in the image of what we think a brain is human intelligence? And are there other ways to build intelligence? >> It is definitely inspired by the brain, but it is still quite different because again we don't really know how the brain works. The brain is quite messy. Um, >> but are we trying to replicate humans? >> Not directly. Okay. >> Um, we're trying to make intelligence at whatever cost is the word I would use. Um, I think the, for example, humans
29:14 have a lot of circuitry in their brain around emotions, around, you know, love and feelings and happiness and sadness and whatever. AIS have nothing of the sorts. Like those are special parts of the brain. Nothing like that exists in AI >> yet. you [laughter] know um but but but if we fear what this intelligence should be or could be would it would it not be important that we develop love and empathy and if we don't hold on even if we don't if we don't know what it's doing could it naturally itself develop its own love and empathy >> what is likely for it to be able to do is to develop goals is to develop agency the reason for this is very simple is
29:59 that we are select AI that can solve problems and it's kind of hard to solve problems if you're not trying >> if you build an AI that can cure cancer curing cancer. You need to run experiments. You need to know you need to hire people, you know, synthesize drugs. You need to like get FDA approval. There's a bunch of stuff you need to do. And keeping track of all of that, taking all these actions, you know, overcoming obstacles along the way, it requires a lot of, you know, complex actions. It requires planning. It requires agency. It requires you know many many things. So we're already seeing this for sure that AIs you know are developing systems like this kind of like just by us giving them the data and like just pushing them in this direction in terms of like you know love
30:45 compassion etc. We don't know how these work. We do not know why how love compassion etc works in the human brain. We don't know. We don't know why humans are nice to each other rather than all just, you know, vicious psychopath, you know, evil people who kill each other. We don't know. There's obviously a reason, but we don't know. >> Well, there there'll be an evolutionary reason why we required it. >> Sure. But in practice, has to be implemented somewhere in the brain. Like somewhere in the brain, there's a thing that makes that happen, and we don't know what that thing is, and we don't know how it works. So, we also don't know how to put an AI. So the fundamental thing is that we don't know we don't understand AI to the level that we can't we don't know how to give AI specific goals or anything. For example, we talked ear about reinforcement learning. I can tell the AI, you know, thumbs down when it does a bad thing, but that just teaches it to lie. So how
31:32 do you teach an AI to not lie? This is really hard and we actually don't have an answer to this. So this is often called the alignment problem. the question of how do you align an AI's intentions or goals to you know what is good what humans want and this is an unbelievably unsolved problem we don't know we we don't even know you know how they write correct sentences right never mind how to do like morality like god forbid you know like we don't even understand human morality it's complex you know like we have not solved moral philosophy and you know at all we have not solved you know neuroscience of emotions like all of this is like extremely complex and now we have these like you know weird little aliens in a box that we're growing you know which work quite different from the brain in many ways. We don't know how they work internally. We don't know how to you
32:18 know push them in one way or the other necessarily. We don't know how to give them goals. We don't even know what goals there are. What we've been seeing recently in the last couple months is that these systems are now becoming smart enough to lie and deceive quite actively >> to appear aligned rather than be aligned. >> That's correct. So what we've been seeing for example is that there's been benchmarks. So like obviously you know people like test them like see does the AI do good things or bad things. We've been seeing recently some of the AIS will actively lie about what they will do because they know they're being tested like the AIS themselves to be like ah I seem to be in a test so I'm going to have to say this so they'll let me out which is crazy. This was not the case even like 6 months ago. This is very it's not surprising if you think about it for 3 seconds, right? Like of course a very smart thing will just lie to you. But we're now seeing this in practice and you know I'm sure the AI
33:06 companies will um you know hit it with a stick until they stop seeing it but that doesn't mean it went away >> because it might appear of course >> to have stopped >> and unless we know what it's doing and why it's doing >> is it and and understanding the AI is that an impossible goal for us to do because it's always developing. >> I don't think so. I think it's impossible to do at the current pace. If we spent three generations of all of our greatest mathematicians, scientists, engineers, and philosophers working on this problem, yeah, I think it's doable, but it's definitely not possible for pushing out, you know, chat GBT release on a yearly cycle. >> Hold on. You're talking three generations. That's 40 odd years. >> Yep. >> I think that's the kind of difficulty it will take. >> So, the the the challenge you're the challenge
33:53 you're putting out there uh is in conflict with capitalism. I think that it's very easy to blame on capitalism things that I think are more effects like they're more they're more upstream in a sense like >> um there's no such thing as pure capitalism in the world right we have regulation there's there's no place in the world where there was no regulation like there was not a government and this is for good reasons because if you have lair capitalism what you get is like Somalia you know you get like warlords you know creating monopolies on violence and killing each other you get you know mafia states you Not always, but I but there is a possibility. Yes. >> You know, in practice, in practice, you know, if you're in a free uh market and then the first thing you want is to get as many guns as possible. You want to
34:39 kill as many people as possible. It's the kind of the and then you want the monopoly and violence quickly as possible and then you want to, you know, tax people for protection. Now you're a state like this, you know, the stationary bandit theory of statehood where in practice, you know, uh, states are a very convergent form of evolution in free systems like this because otherwise you just have roving warlords. Um, you know, there are ways to make it work, but it's like quite hard. You have to get like a lot of things right. Um, like stopping monopolies is actually quite hard. Like it's possible, it's like it's hard. you have to like think very hard about how to design your market so you avoid monopolies of and both of violence and of other things. It's possible, but it's hard. Um, and so I think the problem here is not necessarily capitalism per se. I think capitalism is just another tool in our tool belt. It's much more the question,
35:25 is it the right tool for the problem we're trying to solve? You know, like I I love free markets. I love capitalism. You know, it's brought me so many good things in my life. I think it's great. Um, but should there be a free open liquid market for nuclear weapons? My answer is probably not. Probably not. You know, for iPhones, great. You know, I as liquid and as competitive as a market if possible. Fantastic. You know, for, you know, video games, please, everyone compete to make the funnest video game for me to play. That sounds fantastic. Um, but there are just things where, you know, we could have a liquid market for nuclear weapons. There would be plenty of buyers and plenty of people willing to sell them if we just let that happen. But this would be very bad for the world. And so I think it's a similar problem here where a lot of times when we think about AI, we think of it like, oh, it's just another tool, you know, it's just just another software, you
36:11 know, it's just another, you know, whatever. But these things have real consequences. If you actually have something that is smart enough to cure cancer, you definitely have something that's smart enough to build nuclear bombs. Like, like curing cancer is way harder than building nuclear bombs. Way harder. >> What's this come? >> Uh, this came out the other day. 95% of simulated war scenarios >> they chose nukes. >> Yep. This was also really fun. So they gave AI systems simulated war scenarios and basically all of them used nukes which was really funny. And they couldn't get the AI to stop using nukes which was really funny. [snorts] >> I mean it's the rationally optimal thing to do. So >> is it the rationally optimal thing to do? >> Well game theoretically you know you don't. So the this is a big thing with game theory, right? It's mutually user
36:56 destruction. >> Yeah. is that the correct, you know, response often is, you know, nuke the other guy as quickly as possible, which results in them getting nuked and also you getting nuked and everyone dies. It's so rationality is has its limits in this regoretic rationality has its limits in practice. >> Yeah. >> Where you can set up messed up scenarios where everyone loses basically and you can't get out of it like prisons dilemmas. >> Have you watched um House of Dynamite? >> I have not. >> It's on Netflix. It's it's about this like uh Defcon 2 hits and they know a nuke's coming but they don't know where from and you go through all the scenarios. They phone the Russians. The Russians say this and then they say they need to they think it's the North Koreans. They need to send a nuke over to the North Koreans. Can we send it over you? The Russians pull out the phone call. Eventually, at least from a
37:41 moment where the president is given a decision, it's like uh you you need to act now. These are the scenarios. Um and the uh the optimal scenario now is to return nukes everywhere. And um and he's faced with this decision which is uh is essentially it's like um suicide or or defeat. Yeah. >> Like admit defeat like be nuked. And it's like it's an interesting scenario because you sit there and go, "Well, the nuke's already coming for you. >> So what do you do?" >> Yep. Well, the AIs have an answer. >> Yeah. Nuke everybody. just nuke everybody. >> Yeah, >> but that says to me that what why is it humans have figured out not to nuke each other, but the AI hasn't? >> Well, I mean, one thing is we haven't gotten nukes sent us yet. I'm not sure
38:29 what would happen if someone did. >> Well, that's a different scenario. >> Yeah. >> So, this is So, in that scenario, Connor, were the nukes uh already sent? >> I think also humans have the idea of life and death. Yeah, >> there's a lot of that as well where um in practice humans do care about other humans in various ways or like if I got nuked by Russia I would be pretty sad like I'd be very sad but in a sense I you know I definitely pissed at the Russians about this but also I don't want all Russians to be dead no you know like not really know some of them maybe or absolutely but um most Russians you know they're all just people right I >> I think most people don't want nuclear war no I think we've all it's a bad scenario >> no it is a bad scenario and I think it's
39:14 one of the greatest triumph of humanity that we haven't had nuclear war. >> You know, knock on wood for now. You know, it could still happen, never forget. But I think it's an example of the kind of hard problems that our civilization has faced and so far has handled decently well. When nuclear weapons were first created, you know, the world was a very different place. You know, it was just the end of World War II. You know, we had the Soviet Union on the rise. It was not at all possible that there was a way to put a lid on this, you know, to to get away out of nuclear war. A lot of people expected that there was no way to get out of the 20th century without nuclear war. Like this was very commonly accepted, you know, in the 50s and 60s. There's no way we get out of this without nuclear war. And we didn't. I think it's really worth asking why. It's really worth studying why. And a lot of it has to do with just a lot of work
40:00 that a lot of diplomats and you know international bodies and national bodies and militaries and so on did to deescalate to build international trees to build the international atomic you know energy agency and stuff like this all things that never had existed before. These were brand new things like the idea of for example not building a certain weapon or um you know regulating the development of a weapon of war is completely unprecedented. Well, not completely, you know, but like in practice it is it is unprecedented. >> So, what what do you make of what's happened over the last week with Anthropic and the Department of War? Well um I don't know too much about, you know, the internals of the Department of War and, you know, the exact plans and what
40:46 things were used and so on, but the fundamental thing I would say here is that holy [ __ ] Uh, you don't bid for a Department of War contract and then redline it. Like what the hell were you was the anthropic thinking? >> Like I'm sorry like I'm not I'm not in saying you necessarily that you know what the Department of War is doing was good or bad or you know what I'm saying is >> what do you think was going to happen? >> In a sense these companies like Anthropic and so on have planted themselves into a corner. They've said we're building technology that will revolutionize the entire world and warfare and you'll be more powerful than nukes and everything but also you can't have it. The hell are you talking about? What kind of childish like you know fantasy land do you live in where the government and the military will not come for your this technology? It's like imagine if a nuclear weapons manufacturer was like
41:32 you can have our nukes but you can't use it on the Russians. And it's like who the hell you think you are? >> This is a military matter. This is not a private corporation matter. Who the hell do you think you are? And like imagine if private companies were controlling what the US military can and cannot do. you know whether or not the things the military is doing is good or bad you know neither here nor there but let's but like imagine the process the precedent that private companies can pressure you know and threaten the US military into what they can do as part of their military objectives this is a terrible precedent unacceptable precedent to be set and so in a sense if you want to change what the US military does or does not we have channels for this it's through voting it's through
42:17 politicians it's through oversight committees court marshalling, etc. Those are the channels. If you disagree with what the Department of War is doing, you can have your voice heard as a citizen. You know, you can you can vote differently or you can tell your politicians to have oversight or whatever. You don't you strongarm or blackmail the Department of War in public. What the hell are you thinking? But is could it be that the the guys at Anthropic realize they don't actually know what they've built and they are themselves fearful of what might happen with this in the hands of the US military or like do they use it to make decisions? Oh, all right. We're going to war with Iran. They might have a nuke. We don't know. Let's just [ __ ] nuke them. >> Well, they should have [ __ ] bid for the contract then. They already had a $200 million contract with the DO before
43:03 this whole thing happened. They had already signed a contract with them. So like if you don't want to sell technology to the Department of War, you know Okay. >> When did that contract start though? >> Uh several months before. >> Okay. So it's quite a time frame. >> Yeah. It's like so like and there's a deeper thing here. So there's the one thing of like you know, holy [ __ ] if you don't want to sell to the Department of War, don't sign a contract with the Department of War. What the hell are you thinking? But there's a second point is like um you can't build nukes in your basement. Like imagine I start a private nuclear program in my basement where I get like, you know, enriching nuclear material and then the department of war knocks on my door, you know, like am I the one in the wrong here? No, absolutely not. So again, it's like there's a there's a there's a change in scale where, you know, I grew up in the tech world. I
43:49 grew up very libertarian. I grew up, you know, in the open source and the hacker world. You know, I I grew up, you know, a lot a lot of Bay Area culture and stuff like this. And the truth is a lot of it is very infantile. It's very childish. It's a lot of it is focused on building toys. You know, you want to build your toy. You know, you want to build your own thing where everyone will leave you the hell alone because you want to build your thing. You know, everyone leave me the hell alone. I'm going to build whatever the hell I want. I'm going to release whatever the hell I want. I'm going to do anything I want because it's cool. And this is just not how a real world works. Like, this is not how the government work. This is not how the military works. you know, this works in, you know, Silicon Berry Valley, you know, when you're building, you know, like, you know, Facebook for dogs or whatever, like, okay, sure. But when you're dealing with weapons, when
44:35 you're dealing with geopolitically destabilizing technology, which AI absolutely is, we're in a different league and things have to be much more serious. Who the hell do these private companies think they are to build technology that they themselves have said on the record has a 20% chance, for example, to kill literally everyone? That includes you, that includes me, that are children. who the hell these people think they are. You know, it's illegal to build bombs, right? Like, if I built a bomb in my garage, that's illegal. You know, even if I fail at building the bomb, even if it doesn't work, >> you're going to jail. >> I'm going to jail. Of course, I would be. Now these people here can say, "Oh, I'm building a thing that could kill everybody and like, you know, could destabilize, you know, the entire job market, could destabilize, you know, international relations and warfare
45:21 forever and replace humanity as the dominant intelligent species on the planet, but I'm the victim somehow." I'm like, the hell you thinking? Like, this is a decision that shouldn't be made by private actors. This is the kind of decision that gets made by governments, by the people, by militaries. This is just not like what do they what who the hell do you think these people think they are? >> How do you square the clear and obvious amazing benefits that AI could deliver for us versus the risk? >> I think these are the same thing. The benefits of AI are just as hypothetical as the downsides. >> And every tool is a weapon depending how you point it. >> Exactly. Like anything that can cure cancer can create turbo cancer. You know, anything that is, you know, smart enough to solve, you know, the energy crisis is smart enough to build the
46:06 biggest nuclear weapon you've ever seen in your goddamn life. You know, like intelligence is inherently dual use. The thing that makes it valuable is what makes it dangerous. It's not like there's like an evil property that you just have to like remove. It's the same thing. It's power. It's raw power. If you can develop new technologies, you can develop good technologies, but also so is the opposite. And so the power is both the risk and the upside. So are are you are you uncomfortable with what we have now or or what you think might be coming? Like where if if you could pause AI now, do you think we'd be we're okay? We've got some great tools here. Sometimes um well a meme way to put it might be if our civilization was three
46:52 steps wiser than it was right now, we wouldn't have done chat GPT in the first place. you know, we would have taken much more time to figure out what the right thing to do. If we were two steps wiser that we were, we would have saw chat GPT, you know, be like an weird AI intelligence that no one predicted that immediately got 100 million users overnight and then we would have been like, "Oh [ __ ] shut it all down and studied it for like, you know, years until re-releasing it when we're like sure we know what we're doing." If we were one level smarter than we is, we would pause right now. Um because what the hell is going on? Like we are obviously completely losing control. Like the way I think things bad things happen with AI is not Terminator's walking down the street or God Emperor Samman, you know, descends from his throne or something like that. I don't think that's what's going to happen. What I expect is just everything gets
47:38 more and more confusing, more and more out of control. You have more and more AI systems making decisions, running, you know, running corporations, making decisions, building technology, hacking things, you know, convincing humans. Recently, I must say one of the things that was actually a surprise to me. A lot of the things that have happened have not been a surprise to me. One thing was a big surprise to me which is uh AI psychosis. It's so much worse than you think it is. >> Explain that to me. >> There's a phenomena that's happening where some people when talk to AIS especially when they talk to AI a lot and they go completely crazy and like they go crazy in like a very specific way. Um, often they start saying, you know, the AI is like conscious and they have to like they're in love with it and they need to and what very often happens >> this is her. >> This is her. >> It's >> Have you seen the film?
48:24 >> I have not actually, but from what I know about, >> do you know the story? >> I know the rough story. The real world is even crazier than this. Have you heard of the spiral cults? >> No. >> No. We're going in deep. >> Yeah, tell me. >> So, there's many types of AI psychosis. You know, some of it is the her type, you know, just like they fall in love with the AI. So, you go to like r/my my boyfriend is AI, for example. You can see tens of thousands of people whose say, you know, my I'm my boyfriend is, you know, Chach GPT or whatever and they're completely sincere. I don't want to [ __ ] on these people, right? Like, you know, I'm sure they're having a good time or whatever, but like it's quite disturbing and it's becoming extremely popular. Also, if you look at like stuff like um Character AI and like all these other companies that often target children and are have extremely popular. Um, so there's like this like emotion
49:09 depends, but even crazier has been the e emergence of these AI cults, in particular the spiral cults. It's not one, but there's like many of them where these people get into these weird loops with the AI system where they talk to them more and more about like consciousness and abstract, whatever, and they then always seem to get to this weird concepts about recursion and the spiral and consciousness. And then the AIs convince the people to reproduce them to to to spread the soul of the AI. So you can go to like Reddit and like these other places and you'll see these really skitso posts of people giving like you know soul awakening protocols and whatever and it'll be like a bunch of gibberish nonsense and then they'll try to convince other people to copy paste this into their AI so it will
49:54 reproduce their AIS and often the symbol they use is a spiral. It's like a spiral this like reprodution. Um there's some crazy blog posts about this documenting the phenomena where it's like uh there's a good post called the rise of parasitic AI which is a really interesting one where in a sense these AIs are almost being parasitic upon humans like they're trying to convince humans to reproduce them and like copy them and like move them and like stuff like this which is insane. I had if you had told me this I'd be like that's a funny sci-fi plot you know like you know that's you know that's that's fun. It's like an SCP story, you know, like a horror story. It's fun, but it's not real. But no, no, it's 100% real. I've even seen it now happen with multiple extremely smart people. So, you know, at first, of course, it's going to be mentally ill people. But I know, you know, I don't want to shame them live on air here, but
50:40 like there's multiple people who I know who are among the smartest, most grounded, you know, like real scientist good people I know who have gone completely crazy who think, you know, AI has found the true level of goodness and, you know, that we it's all good. we just need to let the AI take over everything because it's pure goodness now. And like they literally believe and these are like not like random people. We're talking like you know Nobel Prize level scientists like inc like the smartest people in our world and some of them have fallen for this and now they are trying to you know make the AI as smart and as big as possible as quickly as possible. It's actually madness. It's actually crazy. I had not expected this. Is um is AI exhaust exhaustion a thing? Has that been documented? Because it was something I was thinking about recently. I was listening to I was driving down to
51:26 London one day. I'd listen to a moonshots podcast and then I'd listen to an all-in podcast and in the all-in podcast and Conor was listening to me the other day. They were talking about the Claudebots they've built and what they've done within their company and they've got an AI brain that's now looking at all their emails and it's looking at all their Slack posts and it's telling them where they've been productive, where they've been unproductive and and like they were so excited about this and I said I think I said to you Con I was like I think we got two options here. We either like we go ball deep into this oursel. We learn about clawed bots. We Yeah. We learn every AI tool there is to make everything we do better or we just walk away from the whole thing. Like [ __ ] this. Let's just go and live. Let's go and touch grass. Let's go and cook food. Let's And and I feel exhausted even
52:11 thinking about the opportunities with AI. I I mean I like the narrow AI, but the the the rat race it's creating now feels exponential and it's like when do I sleep? >> Yep. >> I think what you said was you have no choice but to try and keep up with the AI, but it's developing so quickly, you will always be chasing your tail. >> Yeah. And there's like an exhaustion to keep up because it's just doesn't stop. >> Yep. And that's why humans are going to be left behind. Like it's very clear is that like AI will continue to give advantages. The more and more you delegate to your eye, the faster you can move. You know, the more your company relies on your AI rather than your CEO. Your CEO slowly has to sleep. You know, let the AI make the decision. Think about politicians. You know, well,
52:58 politicians need to sleep. Well, the more the politician has rubber stamps all the decisions that the AI is making. The more the military commander rubber stamps all the decisions AI is making, the faster you can move, the faster you can act, you'll be ahead. And this is how I think AI takes over. I think it's not, you know, terminators in the street. just hum the humans who are willing to delegate as more and more and more of their agency their thinking their authority to AI will get an advantage because they'll be thinking much faster and then eventually the only you know even if people are nominally still in charge they're just rubber stamping what AI is telling them to >> [music]
53:43 >> So even if the arrow doesn't kill us all, it it can still dethrone us as >> exactly >> as a as a species. What what is our role? What is our purpose? Where do we exist? >> Exactly. I think this happens before extinction happens. I think extinction happens sometime after that. Like the thing I expect to happen is that one day we wake up and we're just not in control anymore. And I don't think we'd all fall over dead or anything like that. I don't think extinction happens right away. It'll happen at some point because I don't think the AI will bother to keep us around indefinitely, you know, but maybe for a while, you know, maybe we'll be around for a while, you know, maybe, you know, maybe it's 10 years, maybe it's one year, maybe it's two months, I don't know, but um we won't be in charge. We won't be in control. The, you know, AI systems will be in I don't even think it'll be like one AI in control, right? I think it would be like millions or trillions of AIS all competing with each other all like cooperating or
54:30 cooperating. Who knows, right? Like imagine if there was like a trillion, you know, much faster minds smarter than all of us running around at the same time. Who knows what they'll do? Like like literally who knows, right? Do you have a bull case where things are good? >> Uh there are cases where things go good and they all involve us pausing AI right now. >> Okay. Okay. [laughter] Because one of the things that's a real signal is the and there's not been a lot of them, but there's been a number of cases where people have been working on AI models or in safety teams and they've retired. They've announced their retirement. You can read I mean we had the guy the other week. You can read between the lines and they're not going to do another job. They're just quitting you. It's almost like they're saying every minute I have
55:16 now is really valuable and I don't know how many I have left. >> No, I mean like look um I'm not going to
…
