Sergey Brin on the Future of AI & Gemini

0:00 [Music] Hey everyone, thanks for joining us. We've got an IO special. Sergey Bren, we're talking all things Google. Uh, thanks for thanks for taking the time to chat. Thank you, Logan. And, uh, you know, you and I are all the time in chat spaces and all kinds of products, but it's nice to hang out in real life. Yeah, it is. the my California experience is always it's incredibly fun to I spent spent a bunch of time with Korea yesterday and today and like he's just it's like you feel the warmth and the humanity of AI progress when you when you spend time in person with everyone. So it's um it's been a ton of fun. But we're sitting here at IO and I think the general sentiment from the world and I think also from the team internally is like incredibly great day for Google. Um a ton of progress across

0:48 models across all of our products. Um what's what's your take? What's your reaction? Obviously, there's a lot of stuff we still need to do, but um where's your where's your head at? Um yeah, I think it was definitely a phenomenal set of announcements. Um honestly, I probably didn't even know about 30% of them or so. Uh you know, there's only so much time and uh I'm kind of deep in the weeds on Gemini and I didn't even know uh about the virtual fit, for example, for products in Google search. Yeah. I didn't realize we were shipping that. Um, so there were many things that also even surprised me. Um, that was great. Uh, I think the reception's been great. Uh, there are so many things though. I think it takes a

1:34 while to um for people to explore them, wrap their heads around them. And uh, obviously we're very busy delivering all of them right now. Uh and uh it's uh a lot of energy uh across the way uh just make sure things are actually shipping shipping smoothly uh that people are you know able to sign up for Ultra have all those new features and so forth. Yeah. I feel like IO is the start of lots of work for lots of other people. It's like at the finish line for some teams and then it's like the starting line for a bunch of other teams. How do you think about um obviously we launched more there's a bunch of Gemini announcements Gemini diffusion we'll talk more about later deep think which is like continuing to push the frontier on reasoning models I

2:20 see you in the reasoning uh strike chat all the time sort of pushing on people to to keep pushing the frontier which I love to see um how do you think about sort of maybe maybe how you think about your focus but also just like generally the like deep mind team's focus around like VO and imagine and we have like a whole suite of generative media models. We just announced a LIA our music model. Um and then also at the same time like the core Gemini main model like how you know do you do you work on the gen media stuff at all or is it mostly like laser focus on on Gemini at the moment? Mostly I'm on Gemini, the core text model. Um, primarily because I think that that's um what will lead to uh for example

3:07 self-improvement uh will help us code and develop the science behind AI even better. Um so that's the number one focus that I have. At the same time the generative media is so amazing. Yeah. I mean that is just so super human you know uh I mean with a text model you know I mean there's some like math problems whatever that I might be able to solve that it gets wrong or something like that or stumbles on a piece of code although that's less and less frequent and I actually end up now relying on Gemini uh to to do some of my coding math and so forth. Uh but nevertheless, it's sort of in the range of human. Yeah. Um given my artistic

3:55 talent, there would be zero chance of me ever coming up with either, you know, an image or video. I mean, just the amount of work that that would I imagine would take if I was an expert, you know, videographer, 3D renderer, whatever, special effects person. I mean I mean that's got to be, you know, a month of solid work to get the thing that I can get in a few minutes. Um and obviously it's visually so compelling that um you know it just kind of sucks you in. You can't escape it. the the audio piece with VO h makes it feel like I I historically have you know personally I think like generating videos is awesome and stuff but it's always like kind of felt

4:41 slightly gimmicky to me um and I think when I saw audio in V3 uh on stage yesterday I think it it like made that was the moment for me where I was like okay this is actually so many people are going to be able to just cuz also practically historically like you could generate a video but then like you'd have to go and like where does audio come from, you know, how do you, you know, sync everything up? And now like you can make humans like talking and having a conversation. Uh, and like it just does it all well. And it it's it it blew my mind. Uh, yeah, you're right. I mean, I I've been a huge fan of that. I I'm like a pretty visual personally. I'm not like a very I guess audio person but uh over the years especially you know like with Google Glass I mean the moment we added some sound I mean that's just

5:27 like it adds so much richness to have sound. Yeah. I mean it's like you're better off adding audio than adding like 3D for example. Um although some of the 3D stuff's cool if you played with uh the big wearable thing. Yeah. Um but anyhow um yeah it's it's it's just an incredible change in perception uh when you get audio working and I I knew I saw the model training the last uh month or two uh and uh you know I just saw it from checkpoint to checkpoint and I knew that wow this is just going to feel different. Yeah, it'll be interesting to see how the like fusion of those capabil because it does seem like there's a lot of like similarities to mainline Gemini

6:13 like mainline Gemini model and obviously we landed native audio support um in the mainline Gemini model at IO um and in VO as well and I was having a conversation with Tulsi this morning about just like how you know are those similar breakthroughs are they different it sounds like it's actually technologically very different from a technology standpoint but it is cool that we have like other rails to do this innovation and like ideally it all upstreams back to Gemini in some way. Yeah. I mean honestly I think we've just taken a long time to ship the native audio in Gemini. It's been in there for a year. Oh really? A no no in the the base model has had audio that it's trained on for at least a year. And

6:58 um I don't know, there's always like I think honestly it's just there's just so much to do, so much to ship that nobody has uh for whatever reason uh gotten it out there. And I mean like native audio in, native audio out. Um I think native audio in has been in there even longer. Um, but to get through all the little hoops to make it really work well and so forth, I just think for whatever reason, um, took a long time. Uh, but yeah, that's finally out there. I don't think that's done, as you say, the same way Vio does it, I believe, does the audio also through diffusion, just like it does the video. Yeah. In fact, if you watch the during the training run, you can actually see it generating videos that

7:44 are like, you know, um, a couple percent into it, it's these kind of, you know, the shapes aren't quite right and the words are kind of war and stuff like that, but then it takes shape and it develops uh until, you know, at the end of the run you have what you have uh what you see today. Um, so yeah, I'm pretty sure that that's diffusionbased audio. Yeah, I mean diffusion is a really powerful technique. You know, as you know, we shipped uh text diffusion for, you know, small early test run. Um I mean I I think that's one of the things that I'm grateful that we have, you know, the bench of machine learning researchers in there that we can pursue simultaneously different base techniques

8:30 um for um across modalities. Yeah, the results of Gemini diffusion look super super promising so far. It I I'm hopeful the model progress is there and like it all fully works because the demo works. We were talking off camera, the demo looks really good. Um, so hopefully it like the capability translates well and and everything works from that perspective, but you you mentioned this before about like watching the training run. I actually haven't seen what this looks like. So like what does it actually mean to watch the training run? Oh, okay. Well, um maybe you've seen for our text models, but uh you know, we're able to um to test out the intermediate checkpoints, you know, in 10% of training, 20% of training and so forth. And it's um you know, the model is weak at those points in time, but it kind of

9:17 you can kind of get a sense for the trajectory. Yeah. Uh and so um you know usually especially if you have a big training run that you have a lot sort of you're using a lot of compute and you have high hopes for it um you're going to test it out in various ways through you know many times throughout the run. Um so you're going to have a pretty good sense of what it's on trend for. Yeah. Uh so that's true for the text models, that's true for uh the um the fusion uh you know video model for VO. Um yeah, all of these models kind of have these intermediate results that you can take a peek at. And if you're, you know, really deep in there, you're for sure checking them because you're, you

10:03 know, nervous and excited about what exactly it will produce. Yeah. I I need to come and hang out in the MK more and watch um Yeah. Come over later. Watch some of them. Yeah. Um, one of the I was listening to Sundar's conversation with uh Dave Freeberg and he Sunnar made the comment that um even 15 years ago you and Larry and him were having conversations about uh and like the team at Google were having conversations about like what this future facing AI moment would look like. Um and that there it's like eerily close to what you all were talking about 10 or 15 years ago. Um I'm curious like how like what what are the things that have been most surprising to you about this this moment like even and we can ground it in

10:49 products if you want to look at search or just the technology or like what what's been surprising and and what's been like almost what you would have expected to have happened. Yeah. You know, I I think from an intellectual standpoint, you can kind of reason your way through the singularity. And uh you know, famously Ray Kerswell did this, I don't know, however many decades ago. Um I mean, it kind of um I don't remember what date he said. It was 2037, I can't remember. He put some date on it based on his extrapolation. Um, today looks like maybe that was kind of conservative. I don't know. Uh, but you can intellectually sort of reason through

11:35 it. I think to see it happening is altogether different. Yeah. Um, and I think when you're kind of talking whatever 15 years ago, I won't say you're like joking around. you're like truly talking about it, but you're kind of like, you know, imagining the science fiction future, but it's almost like um like a game like a like you're just kind of uh you know, chatting with the other folks who are interested in it. Like I think it's fun. Um but yeah, as I said, seeing it actually start to happen, you know,

12:21 feels very different. Yeah. Uh and of course the way in which it happens is pretty surprising. Um and I can give you an example. I mean the fact that language models seem to be right now the way that you know AI is developing. Um I don't think you necessarily would have known that 15 years ago. In fact, deep mind um you know in the past and even now to a certain extent has bet a lot on this kind of physical grounding that it's important to have um kind of a physical world to ground on and and we're obviously doing experiments in that vein like genie and whatnot. Um but the fact that these language models have come as far as they

13:07 have wasn't obvious. And an interesting side effect of that especially with a thinking models is that they are also surprisingly interpretable. Like you can look through the thoughts of one of these thinking models and how it's coming to the conclusions that it's coming to. um you can't necessarily without a huge number of tools inspect the weights of the model and try to infer stuff from that. Uh but you can understand you know a lot of its reasoning in in you know very understandable terms. So I think that's kind of um you wouldn't necessarily have guessed that 15 years ago. Yeah. Yeah. Um, so that's that's

13:53 been an interesting surprise. Um, which which I think gives a lot of um comfort. Not infinite comfort. I'm not saying we should ignore it, but from a safety standpoint, the fact that these things do to some extent say what they're thinking, I think is a big plus. Uh, and yeah, there are papers out there about how they're sort of lying and stuff, but I I think that's a relatively smaller effect. What's your sense just being close to the model training process today as far as like how different or how similar it's going to look as the models move from being like to text in token in token out uh or text out to being like

14:38 actual systems and I think we're actually like we've already taken it like Gemini 2.0 you know, like search is natively in there, code execution is natively in there, like the model is learning that in the process. Do you think the like training infrastructure or just like the way we think about models um is going to be fundamentally different as they like they're not models anymore? They're really like full systems that we're baking for people. Um I I think it's the confluence of a few things. One thing it's kind of remarkable how similar architecturally all the different models are uh including for example VO which you would think video diffusions like very different than some text language model uh but architecturally there's a huge amount shared. It's kind of astonishing

15:25 how much is shared and a lot of it uh you know uses transformers at the core uh which uh thanks to Nome and the crew uh we've had now for uh approaching a decade. Um now we are adding things like tool use and so forth. Mostly those things uh come about in the during what we call post training. Uh and post-training is an increasing fraction of the overall training. you know, um it used to be that everything was like 99% uh pre-training and it's sort of shifting now to now maybe it's 90% will be 80% and so forth. Uh and this postraining which is sort of you know

16:11 some people call fine-tuning but it includes you know the RL kind of work that we do um you know used to be this just little bit of tiny bit of shaping you do in the end. Uh but now it's more and more material and the kind of things you're mentioning with tool use and so forth come in during that now much larger phase uh and um yeah I mean look that makes the model vastly more powerful. Yeah I've got uh two more questions because I want to I want to get you back in the office working so that we can keep making model progress. Um first one is just around like the the reasoning scaling like I think uh we announced we showed the results of deep think uh which is sort of continue with scale up 2.5 pro and

16:58 let it reason for longer and and have sort of parallel thought processes. Um what's your sort of general reaction to that? I think it it feels like we're so early in that scaling paradigm that there's like going to be a huge amount of additional unlock, but uh you're obviously you're you're in the weeds on this one. So I'm I'm curious what your what your thoughts are. Um yeah, we have a you know interesting we had about like five different approaches uh to doing that kind of thing and they all converged on this deep think. So it was uh um it was great to see all those people and those teams come together and that you know sometimes we fragment and it takes long time but in this case we kind of took the best of ideas of all of them uh

17:44 combine it in one go and uh yeah it's definitely yielding stronger results obviously um I think that the more that continues to happen um that that is like a superpower. If you can have these models um and I know lots of the top AI labs talk about this, but if you can have these models instead of just thinking for a minute spinning out an answer, if you can leave them to go for an hour, for a day or for a month maybe, and they actually get you uh significantly better answer to a really important question, um that can be incredibly valuable. Yeah. Uh, and that's that's kind of new

18:29 and it's um and it's non-trivial. It's a little bit like, you know, we cracked long context for input. Um, and we did that a while ago and we've had million plus context for um a year and a half or something like that. Now, we need infinite context. So, you got to keep pushing the No. Yeah, I'm not saying million's enough. But that generalization is is sort of non-trivial. Yeah. You know, to to for a model to um it's as if you were to go through like Groundhog Day, you know, you just get over and over. You get to sample one day as a person. You try this, you try that, and now all of a sudden you're meant to live your life where stuff happens from day to day and week to week and month to month. It's kind of a

19:16 non-trivial generalization. Uh but you know we figured out how to do that. Um on the output side it's also it's just kind of non-trivial if all you do is um say short little math problems. Um going from that well it's kind of like you know we interview people we ask them 10 interview questions or whatever and uh then we expect them to build these big systems over months and it's not clear if that's actually the right way to test a person. Uh but on the AI models, we do that, you know, a million times over. Like we're only training them to do, you know, short little clever math questions, coding things, whatever. Yeah. Uh and then the expectation from there that, hey, they're actually can spend a long time

20:03 to develop something new that requires, you know, thinking over days, let's say. um that's very non-trivial, but that that is a gap that um that we're starting to overcome and that's uh that's a that's a huge leap forward. Yeah, this example that you gave um around just like how we test and eval models is is always my consistent reminder that so much of life is a like this AI moment has taught me that so much of life is like actually an eval problem and like the challenge of like even interviewing people and like trying to build a great team and all that stuff like is an eval problem at its heart and like we humans haven't solved that like it's not surprising to me that we haven't solved the like AI eval problem as well. Um it's uh it is non-trivial to do that. My my sort of final question for you and this is just

20:49 like again in in reaction to everything that that we're seeing with IO and and the pace of innovation. Sundar had a slide on the screen actually know Demis did where it was like everything we shipped in 2024 and then everything we shipped in 2025 so far and it was like I'm pretty sure the 2025 section was bigger than the 2024 section. So like clear acceleration happening. Um, and it feels like at least on a personal level for me joining Google, I've been here more uh almost a year or a little more than a year now. Um, like Google truly feels like a startup experience to me and I'm curious to get your reaction to that, but also just like your reaction in general having like seen Google grow and you know expand and everything that's happened in the last 20 years, how you think about that. Um, and yeah,

21:37 great question. I mean first of all you know I think companies need to periodically reinvent themselves. Um and uh you know there are different important technology shifts I guess you know we started as a web company we had to make mobile work um we know we're never particularly good at social to be honest as an example but um you know now we're in the AI domain and I think from there like the it's exciting because in some ways Google has always been an AI company. We've always been about large scale data and analysis. Um we are also the place

22:24 that gave birth to a lot of the modern largecale machine learning um from Google brain to the transformer um and so forth. I mean it is in the company's DNA. So this is um this is a shift we should be really well equipped to make. Um you know any any shift is probably hard for on any company. Um but I'm feeling really good about it and uh I think going from 24 to where we were honestly catching up on many levels uh to 25 and particularly with the launch of uh 2.5 Pro. Yeah. I mean that was just that was just like a clear leap forward and I know whatever on different

23:09 benchmarks maybe we were for bit number one before not 2.5 pro was a big step forward uh kind of across the board um and even you know to date it remains on most leaderboards number one you know with style control without style control however you measure it um so that's been just a really exciting leap forward Um, and I think we've uh it's both um sort of cause and effect of the um uh kind of science uh the science engine we have going behind it. Um it's going to help propel us forward and it's thanks to what you know all the science

23:54 we were doing over the past year that we were able to finally uh produce that model. Um, and in quick succession after that, a lot of other things have followed. We've already gone through a few different iterations of the 2.5 Pro model. Um, I don't know if everybody noticed yesterday we launched the new 2.5 flash. Yeah. The I mean, do you notice that's actually like on many measurements, it's number two behind 2.5 Pro. Yeah. So, we're like one and two on many different leaderboards now with that flash model, which I think um with all the other announcements, I just think a lot of people might have glossed over. It got buried, but it's like a super fast model. It's really powerful. I think it's going to appeal uh to a lot of use cases.

24:39 Um, yeah, but really with that uh kind of uh cornerstone of 2.5 Pro this year, I think for us to be able to build off of that and continue the momentum is uh is really exciting. It's going to be a great year. Uh Sergey, I appreciate you for taking the time. I appreciate you for for pushing everyone hard. Um it's a ton of fun to see and we have a special a special gift for you. uh which I'd love to see you to see you unbox and somebody will somebody will bring it over to us in one sec. Um well thank you Logan and while while they bring it over I'll just say thank you Logan. Um I mean I see you working hard all the time and uh you know making all of our customers and partners happy and tracking you know the million of issues that might arise.

25:27 Uh I mean it's not that easy. um you know having these models that so many people in businesses want uh and uh you know getting them deployed and not having the TPUs meltdown and uh um you know every little nuance uh from function calling to caching to all the million things and I see you um being really good about putting uh the customers first communicating the needs back to the team being just really on top of the ball so the team's crushing it now thank thank you. The team's crushing it. Um, special gift for you. All right. Thank you. Shall I unbox it right now? Yeah. Yeah, you got to unbox it right now. We got to We got to capture. Uh, as as sort of the the thread of this was just like one of the pieces other than all the people inside

26:13 of Google that makes all this possible. It's our It's our TV. This is one This TPU V4, which uh internally, by the way, we call Pufferfish. I probably not uh it's probably not too too secret. Uh I think Pufferfish is V4, right? Yeah, I never know the external transmission. We just call these um I mean these were the hottest thing to come by like a year or two ago. We've, you know, moved on to newer generations now. Um but we still do a lot of our work on these. Um so this is great. Yeah. Hopefully we'll get a bunch of these in the MK for the team. And um this is really cool. It's a it's a real one. They had to take it out of a data center somewhere. It was not being used. We were not taking compute out of really we definitely need we need the TV. Sometimes some of the early samples are a little bit faulty and maybe maybe it's one of those but I appreciate it.

27:00 That's super nice. Of course. I'll give you a little zoom in on there. Thank you. Thank you. Thank you for tuning in. For folks who are listening, uh this is release notes. Thanks for watching. [Music]