Jensen Huang: NVIDIA - The $4 Trillion Company & the AI Revolution | Lex Fridman Podcast #494

0:00 The following is a conversation with Jensen Huang, CEO of NVIDIA, one of the most important and influential companies in the history of human civilization. NVIDIA is the engine powering the AI revolution, and a lot of its success can be directly attributed to Jensen's sheer force of will and his many brilliant bets and decisions as a leader, engineer, and innovator. This is the Lex Friedman Podcast, and now, Dear friends, here's Jensen Huang. You've propelled NVIDIA into a new era in AI. moving beyond its focus on chip scale design to now rack scale design. And I think it's fair to say that winning for NVIDIA

0:45 for a long time used to be about building the best GPU possible. And you still do, but now you've expanded that to extreme co-design of GPU, CPU, memory, networking, storage, power, cooling, software, the rack itself, the pod that you've announced, and even the data center. So let's talk about extreme co-design. What is the hardest part of co-designing a system with that many complex components and design variables? Yeah, thanks for that question. So first of all, the reason why extreme co-design is necessary is because the problem no longer fits inside one computer to be accelerated by one GPU. The problem that you're trying to solve is you would like to go faster than the number of computers that you add. So you add

1:31 a you know, 10,000 computers, but you would like it to go a million times faster. Then all of a sudden, you have to take the algorithm, you have to break up the algorithm, you have to refactor it, you have to shard the pipeline, you have to shard the data, you have to shard the model. Now, all of a sudden, when you distribute the problem this way, not just scaling up the problem, but you're distributing the problem, then everything... gets in the way. This is the Amdahl's law problem, where the amount of speed up you have for something depends on how much of the total workload it is. And so if computation represents 50% of the problem, and I sped up computation infinitely,

2:21 like a million times, you know, I only sped up the total workload by a factor of two. Now, all of a sudden, not only do you have to distribute the you have to shard the pipeline somehow. You also have to solve the networking problem because you've got all of these computers are all connected together. And so distributed computing at the scale that we do, the CPU is a problem, the GPU is a problem, the networking is a problem, the switching is a problem, and distributing the workload across all these computers are a problem. It's just a massively... complex computer science problem. And so we just got to bring every technology to bear. Otherwise, we scale up

3:06 linearly or we scale up based on the capabilities of Moore's law, which has largely slowed because Dennard's scaling has slowed. I'm sure there's trade-offs there. Plus, you have a complete disparate disciplines here. I'm sure you have specialists in each one of these, high bandwidth memory, The networking, the NVLink, the NICs, the optics and the copper that you're doing, the power delivery, the cooling, all of that. I mean, there's like world experts in each of those. How do you get them in a room together to figure out? That's why my staff is so large. What's the process? Can you take me through the process of the specialists and the journalists? Like, how do you put together the rack when you know the set of things you have to shove into a rack together? Like, what does that process look like of designing it all together? There's the first question, which is, what is Xtreme Co-Design?

3:55 You're optimizing across the entire stack of software from architectures to chips to systems, the system software to the algorithms to the applications. That's one layer. The second thing that you and I just talked about is goes beyond CPUs and GPUs and networking chips and scale up switches and scale out switches. And then, of course, you got to include power and cooling and all of that because, you know, all these computers are extremely, extremely power hungry. They do a lot of work and they're very energy efficient, but they, in aggregate, still consume a lot of power. And so that's one, the first question is, what is it? The second question is, why is it? And we just spoke about the reason, you know, you want to distribute the workload

4:40 so that you can exceed the benefit of just increasing the number of computers. And then the third question is, how is it? How do you do it? And And that's kind of the miracle of this company. You know, when you're designing a computer, you have to have operating system of computers. When you're designing a company, you should first think about what is it that you want the company to produce? You know, I see a lot of companies' organization charts, and they all look the same. Hamburger organization charts, software organization charts, and car company organization charts, they all look the same. And it doesn't make any sense to me. You know, the goal of a company is to be the machinery, the mechanism, the system that produces the

5:26 output. And that output is the product that we like to create. It is also designed, the architecture of the company should reflect the environment by which it exists. It almost directly says what you should do with the organization. My direct staff is 60 people. You know, I don't have one-on-ones with them because it's impossible. You can't have 60 people on your staff if you're, you know, going to get work done. So you still have 60 reports. You still have a course. More, yeah. More. And most stars at least have a foot in engineering. Almost all of them. There's experts in memory. There's experts in CPUs. There's experts in optical. That's incredible. Yeah, GPUs and architecture, algorithms, design. So you constantly have an eye on the entire stack. And you're

6:14 having intense discussions about the design of the entire stack. And no conversation. is ever one person. That's why I don't do one-on-ones. We present a problem and all of us attack it. You know, because we're doing extreme co-design. And literally the company is doing extreme co-design all the time. So even if you're talking about a particular component, like cooling, networking, everybody's listening in. Yeah, exactly. And they can contribute, well, this doesn't work for the power distribution. This doesn't work for the memory. This doesn't work for this. Exactly. And whoever wants to tune out, tune out. You know what I'm saying? Yeah. And the reason for that is because the people who are on the staff, they know when to pay attention. They're supposed, you know, something they

7:01 could have contributed to, they didn't contribute to, I'm going to call them out, you know, and say, hey, come on, let's get in here. So as you mentioned, NVIDIA is this company that's adapting to the environment. So at which point can you say, did the environment change and you began, sort of secretly in the early days from GPU for gaming, maybe the early deep learning revolution to we're now going to start thinking of it as an AI factory. What does NVIDIA do? It produces AI, let's build a factory that makes AI. I could reason through it just systematically. We started out as an accelerator company, but the problem with accelerators is that the application domain's too narrow. It has the benefit of of being incredibly optimized for the

7:46 job. You know, any specialist has that benefit. The problem with intense specialization is that, of course, your market reach is narrower, but that's even fine. The problem is the market size also dictates your R&D capacity. And your R&D capacity ultimately dictates the influence and impact that you can possibly have in computing. And so when we first started out as an accelerator, a very specific accelerator, we always knew that that was going to be our first step. We had to find a way to become accelerated computing. But the problem is when you become a computing company, it's too general purpose and it takes away from your specialization. I connected two words

8:36 that actually have fundamental tension. The better computing company we become, the worse we became as a specialist. The more of a specialist, the less capacity we have to do overall computing. And I connected those two words together on purpose, that the company has to find that really narrow path, step by step by step, to expand our aperture of computing, but not give up on the most important specialization that we had. Okay, so the first step that we took beyond acceleration was we invented the programmable pixel shader. So that was the first step towards programmability. It was our first journey towards moving into the world of computing. The second thing that we did was we put FP32 into

9:26 our shaders. That FP32 step, IEEE compatible FP32, was a huge step in the direction of computing. It was the reason why all of the people who were working on stream processors and, you know, other types of data flow processors discovered us. And they said, hey, all of a sudden, you know, we might be able to use this GPU that's incredibly computationally intensive. And it's now, you know, compliant with IEEE. I can take my software that I was writing, you know, previously on CPUs, and I can, you know, see about, you know, using the GPU for them. And which led us to create, put C, on top of FP32 was what we call CG. That CG path took us to eventually CUDA. CUDA,

10:15 step by step by step. Well, putting CUDA on GeForce, that was a strategic decision that was very, very hard to do because it cost the company enormous amounts of our profits and we couldn't afford it at the time. But we did it anyways because we wanted to be a computing company. A computing company, has a computing architecture, a computing architecture has to be compatible across all of the chips that we build. Can you take me to that decision? So putting CUDA on GeForce could not afford to do, can you explain that decision? Why boldly choose to do that anyway? Can you explain that decision? That was the first, I would say that that was the first, the first

11:02 strategic decision that, is it's close to an existential threat. For people who don't know, it turned out to be, spoiler alert, one of the most incredibly brilliant decisions ever made by a company. So CUDA turned out to be an incredible foundation for computation in this AI infrastructure world. So just setting the context, it turned out to be a good decision. Yeah, it turned out to have been a good decision. I think the, so here's the way it went. So we invented this thing called CUDA And it expanded the aperture of applications that we can accelerate with our accelerator. The question is, how do we attract developers to CUDA? Because a computing platform is all about

11:50 developers. And developers don't come to a computing platform just because it could perform something interesting. They come to a computing platform because the install base is large. Because a developer like anybody else wants to develop software that reaches a lot of people. So the install base is in fact the single most important part of an architecture. The architecture could attract enormous amounts of criticism. For example, no architecture has ever attracted more criticism than the x86. You know, as a less than elegant architecture. But yet, It is the defining architecture of today. It gives you an example that in fact so

12:35 many risk architectures, which were beautifully architected, incredibly well designed by some of the brightest computer scientists in the world, largely failed. And so I've given you two examples where one is elegant, the other one's barely aesthetic. And so... Yet x86 survived. Install base is everything. Install base defines an architecture. Not everything else is secondary. Okay. And so there were other architectures at the time. CUDA came out. OpenCL was here. There were, you know, there's several other competing architectures. But the thing that the decision that we made that was good was we said, hey, look, ultimately it's about install base. And

13:21 what is the best way we could get a new computing architecture into the world? By that time frame, GeForce had become successful. We were already selling millions and millions of GeForce GPUs a year. And we said, you know, we ought to put CUDA on GeForce and put it into every single PC, whether customers use it or not, and use it as a starting point of cultivating our install base. Meanwhile, we'll go and attract developers and went to universities and wrote books and taught classes and put CUDA everywhere. And eventually people discover, and at the time the PC was the primary computing vehicle. There was no cloud and we could put a supercomputer in the hands of every researcher

14:08 in school, every scientist, you know, every engineering school, every student in school. And eventually something amazing will happen. Well, the problem was CUDA increased our cost of that GPU, which is a consumer product, so tremendously it completely increased consumed all of the company's gross profit dollars. And so at the time, the company was probably, you know, worth, I don't know, at the time, eight, was it like $8 billion or something, six, $7 billion or something like that. After we launched CUDA, I recognized that it was going to add so much cost, but it was something we believed in. You know, our market cap went down to like one and a half billion dollars. And so we were down there for a while and we

14:58 clawed our way back slowly, but we carried CUDA on GeForce. I always say that NVIDIA is the house that GeForce built because it was GeForce that took CUDA out to everybody. Researchers, scientists, they discovered CUDA on GeForce because they were all, you know, many of them were gamers. Many of them built their own PCs anyways. in a university lab, many of them built clusters themselves, using PC components. And so that's kind of how we got going. And then that became the platform, the foundation for the deep learning revolution. That was also another great observation, yeah. That existential moment, do you remember, what were those meetings like? What were those discussions like deciding as a company, risking everything?

15:48 Well, I had to make it clear to the board what we're trying to do. And the management team knew our gross margins were to get crushed. So you could imagine a world where GeForce would carry the burden of CUDA and none of the gamers would appreciate it and none of the gamers would pay for it. You know, they only pay certain price and it doesn't matter what your cost is. And so, you know, we increased our cost by 50%. And that consumed, and we were a 35% gross margin company. And so it was quite a difficult decision to make. But you could imagine that someday this would go into workstations and it would go into supercomputers. And in those segments, maybe we can capture more

16:35 margin. So you could reason your way into being able to afford this. But... It still took a decade. But that's more of a conversation with the board convincing them, but you psychologically. Because NVIDIA has continued to make bold bets that predict the future and in part, especially now, define the future. So I'm almost looking for wisdom about how you're able to make those decisions to make leaps like that as a company. Well, first of all, I'm informed by a lot of curiosity.

17:21 At some point, there's a reasoning system that convinces me so clearly this outcome will happen, that this will happen. And so I believe it in my mind. And when I believe it in my mind, you know how it is. You manifest a future. And that future is so convincing. There's no way it won't happen. There's a lot of suffering in between, but you've got to believe what you believe. So you envision the future. Yeah. And you essentially, from a sort of engineering perspective, manifest it. Yeah, and you reason about how to get there. You reason about why it must exist.

18:07 And, you know, we all reason it here. The management team will reason about it. All the people that, we spent a lot of time reasoning about it. The thing that the next part of it is probably a skill thing, which is, you know, oftentimes in leadership, the leadership stays quiet or they learn about something and then they do some manifesto and it's a brand new year. And somehow at the end of the year, next year, we're going to have a brand new plan, big, huge layoff this way, big, huge organization change this way, new mission statement, brand new logos, you know, that kind of stuff. Yeah. I never do things that way. When I learn about something and it's starting to influence how I think, I'll make it very clear to everybody near me that this is interesting.

18:57 This is gonna make a difference. This is gonna impact that. And I reason about things step by step by step. Oftentimes I've already made up my mind, but I'll take every possible opportunity external information, new insights, new discoveries, new engineering revelations, new milestones developed. I'll take those opportunities and I'll use it to shape everybody else's belief system. And I'm doing that literally every single day. I'm doing that with my board. I'm doing that with my management team. I'm doing that with my employees. I'm trying to shape their belief system such that when I come The day I say, hey, let's buy Mellanox,

19:43 it's completely obvious to everybody that we absolutely should. On the day that I said, hey guys, let's go all in on deep learning and let me tell you why. I've already been laying down the bricks to different organizations inside the company. Every organization and everybody, many of the people might have heard everything, Most of the company hears, of course, pieces of it. And on the day that I announce it, everybody's kind of bought into many pieces of it. And in a lot of ways, I like to announce these things. And I imagine that the employees are kind of saying, you know, Jensen, what

20:29 took you so long? And in fact, I've been shaping their belief system for some time. And therefore, leadership sometimes it looks like you're leading from behind. But you've been shaping there, you know, to the point where on the day that I declared it, 100% buy-in. But that's what you want. You want to bring everybody along. You know, otherwise we announce something about deep learning and everybody goes, what are you talking about? You know, you announce something about let's go all in on this thing. And your management team, your board, your employees, your customers, they're kind of like, where's this coming from? You know, this is insane. And so... GTC, in fact, if you go back in time, you look at the keynotes. I'm also shaping the belief system of my partners in the industry. And I'm using that to shape

21:18 the belief system of my own employees. And so by the time that I announce something, like, for example, we just announced Grok. I've been talking about the stepping stones for two and a half years. You guys just go back and go, oh, my gosh. They've been talking about it for two and a half years. And so I've been laying the foundation step by step by step. So when the time comes, you announce it, everybody's saying, you know, what took you so long? But it's not just inside the company. You're shaping the landscape, the broader global landscape of innovation. Like putting those ideas out there, you really are manifesting reality. We don't build computers. We actually don't build clouds. We don't, as it turns out, we're a computing platform company. And so nobody can buy anything from us. That's the weird thing. You know, we vertically... design, vertically

22:07 integrate to design and optimize. But then we open up the entire platform at every single layer to be integrated into other companies' products and services and clouds and supercomputers and OEM computers. And so the amazing thing is I can't do what I do without having convinced them first. And so most of GTC is about manifesting a future that by the time that my product is ready, They're going, what took you so long? Yeah. So one of the things you've been a believer for a long time is scaling laws broadly defined. So are you still a believer in the scaling laws? Yeah, we have more scaling laws now. So I think you've outlined

22:53 four of them with pre-training, post-training, test time, and agentic scaling. What do you think when you think about the future, deep future and the near-term future, what are the blockers? that you're most concerned about to keep you up at night, that you have to overcome in order to keep scaling? Well, we can go back and reflect on what people thought were blockers. So in the beginning, we were the first, the pre-training scaling law, you know, people thought, well, rightfully so, that the amount of data that we have, high quality data that we have, will limit the intelligence that we achieve. And that scaling law was an important, very important scaling law. The larger the model, the correspondingly more data results in a better, results in a smarter AI.

23:41 And so that was pre-training. And Ilya, Suscover, Ilya said, we're out of data or something like that, pre-training is over or something like that. The industry panicked, you know, that this is the end of AI. And of course, of course, that's obviously not true. We're going to keep on scaling the amount of data that we have to train with. A lot of that data is probably going to be synthetic. And that also confused people, you know? And what people don't realize is they've kind of forgotten that most of the data that we are training, that we teach each other with, inform each other with, is synthetic. You know, it's synthetic because it didn't come out of nature. You created it. I'm consuming it. I

24:27 modify it, augment it. I regenerate it. Somebody else consumes it. And so... So we've now reached a level where AI is able to take ground truth, augment it, enhance it, synthetically generate an enormous amount of data, and that part of post-training continues to scale. And so the amount of data that we could use that is human-generated will be smaller and smaller and smaller. The amount of data that we use to train model data going to continue to scale to the point where we're no longer limited training is no longer limited by data is now limited by compute. And the reason for that is most of the data is synthetic. Then the next phase is

25:15 test time. And I still remember people telling me that inference, oh yeah, that's easy. Pre-training, that's hard. These are giant systems that people are talking about. Inference must be easy. And so inference chips are going to be little tiny chips and you know, they're not, they're not like NVIDIA's chips. Oh, those are going to be complicated and expensive. And, you know, we could make, and this is, and the future inference is going to be the biggest market and it's going to be easy and we're going to commoditize and, you know, everybody can build their own chips. And, and that was always illogical to me because inference is thinking. And I think thinking is hard. Thinking is way harder than reading. You know, pre-training is just memorization and generalization, you know, and looking for patterns and

26:05 relationships. You're reading versus thinking, reasoning, solving problems, taking unexplored experiences, new experiences, and breaking it down into decomposing it into, you know, solvable pieces that we then go off either through first principle reasoning or you know, through previous examples, prior experiences, you know, or just exploration and search and, you know, trying different things. And that whole process of test time scaling inference is really about thinking. And it's about reasoning, it's about planning, it's about search, it's about... And so how

26:51 could that possibly be compute light? And we were absolutely right about that, you know. So test time scaling... is intensely compute intensive. Then the question is, okay, now we're at inference and we're at test time scaling, what's beyond that? Well, obviously, we have now created, you know, one agentic person. And that one agentic person has a large language model that we've now, you know, developed. But during test time, that agentic system goes off and does research and bangs on databases and It goes and uses tools. And one of the most important things it does is spins off and spawns off a whole bunch of sub-agents, which means we're now creating large teams.

27:36 It's so much easier to scale NVIDIA by hiring more employees than it is to scale myself. And so the next scaling law is the agentic scaling law. It's kind of like multiplying AI. Multiplying AI, we could spin off agents as fast as you want to spin off agents. And so, you know, you're going to have four scaling laws. And as we use the agentic systems, they're going to create a lot more data. They're going to create a lot of experiences. Some of it we're going to say, wow, this is really good. We ought to memorize this. That data set then comes all the way back to pre-training. We memorize and generalize it. We then refine it and fine tune it back into post-training, then we enhance it

28:23 even more with test time, you know, and the agentic systems, you know, put it out into the industry. And so this loop, the cycle, is going to go on and on and on. It kind of comes down to basically intelligence is going to scale by one thing, and it's compute. But there's a tricky thing there that you have to anticipate and predict, which is some of these components, it requires... different kind of hardware to really do it optimally. So you have to anticipate where the AI innovation is going to lead. For example, make sure of experts with sparsity. Perfect. With hardware, you can't just pivot on a week's notice. You have to anticipate what that's going to look like. So good. That's so scary and difficult to do,

29:09 right? For example, these AI model architectures are being invented about once every six months. Yeah. Right? And, uh, system architectures and hardware architectures kind of every three years and so you need to anticipate what likely is going to happen you know two three years from now and there's a couple ways that you could do that first of all we could do research internally ourselves and that's one of the reasons why we have basic research we have applied research we create our own models and so we have we have hands-on life experience right here this is part of the co-design that i'm talking about We're also the only AI company in the world that works with literally every AI company in the world. And to the extent that we can, we

29:55 try to get a sense of what are the challenges that people are experiencing. So you're listening to the whispers across the industry, the ad labs? That's right. You've got to listen and learn from everybody. And then the last part is to have an architecture that's flexible, that can adapt and move with the wind. And one of the benefits of CUDA is that it's, you know, on the one hand, an incredible accelerator. On the other hand, it's really flexible. And so that balance, incredible balance between specialization, otherwise we can't accelerate the CPU, versus generalization so that we can adapt with changing algorithms, that's really, really important. That's the reason why CUDA has been so resilient on the one hand, and yet we

30:43 continue to enhance it. We're at CUDA 13.2, And so we're evolving the architecture so fast that we can stay with the modern algorithms. For example, when Mixture Experts came out, that's the reason why we had MVLink 72 instead of MVLink 8. We could now take an entire 4 trillion, 10 trillion parameter model and put it in one computing domain as if it's running on one GPU. People probably didn't notice, I said it, but if you look at the architecture of the Grace Blackwell racks, it was completely focused on doing one thing, processing the

31:28 LLM. All of a sudden, one year later, you're looking at a Vera Rubin rack. It has storage accelerators. It has this incredible new CPU called Vera. It has Vera Rubin and MVLink 72 to run the It also has this new additional rack called GROC. And so this entire rack system is completely different than the previous one. And it's got all these new components in it. And the reason for that is because the last one was designed to run MOE large language models, inference. And this one is to run agents and agents bang on tools. And obviously the design of the system had to have been

32:14 done before Claude Code, Codex, OpenClaw, so you were anticipating the future, essentially. And that comes from what? From the whispers, from the understanding of what all the state of the art is doing? No, it's easier than that. You just reason about it. First of all, you just reason. No matter what happens, at some point, in order for that large language model to be a digital worker, let's just Use that metaphor. Let's say that we want the LLM to be a digital worker. What does it have to do? It has to access ground truth. That's our file system. It has to be able to do research. It doesn't know everything. And I don't want to wait until this AI becomes universally smart about everything past, present, and future before I make it useful.

33:06 And so therefore, I might as well let it go do research. It's obviously, if it wants to help me, it's got to use my tools. You know, a lot of people would say, you know, AI is going to completely destroy software. We don't need software anymore. We don't even need tools anymore. That's ridiculous. Let's use a thought experiment. And you could just sit there, enjoy a glass of whiskey, and think about all these things, and it would become completely obvious. Like if I were to create the most amazing agent that we can imagine in the next 10 years, let's say it'd be a human or robot, If that human or robot were to be created, is it more likely that the human or robot comes into my house and uses the tools that I have to do

33:52 the work that it needs to do? Or does this hand turns into a 10-pound hammer in one instance, turns into a scalpel in another instance, and in order to boil water, it beams microwaves out of its fingers? Or is it more likely just to use the microwave? And the first time it goes up to the microwave, it probably doesn't know how to use it. But that's okay. It's connected to the internet. It reads the manual of this microwave, reads it, instantly becomes an expert. And so it uses it. And so I think the, I just described, in fact, almost all of the properties of OpenClaw. You know, that it's going to use tools, that it's going to access files, it's

34:38 going to be able to do research. It has IO subsystems. And when you're done reasoning about it in that way, then you say, oh my gosh, the impact to the future computing is deeply profound. And the reason for that is, I think we've just reinvented the computer. And then now you say, okay, when did we reason about that? When did we reason about OpenClaw? If you take the OpenClaw schematic that I used at GTC, you will find it two years ago. Literally two years ago at GTC, I was talking about agentic systems that exactly reflect OpenClaw today. And of course, the confluence

35:25 of many things had to happen. First of all, we needed Claw and GPT and all of these models to reach a level of capabilities. So their innovation and their breakthroughs and their continued advances was really important. And then of course, somebody had to create a an open source project that was sufficiently robust and sufficiently complete and that we can all put to work. And I think OpenClaw did for agentic systems what ChatGPT did for generative systems. And I just think it's a very big deal. Yeah, it's a really special moment. I'm not exactly sure why it captured so much of the world's attention, but it did more than CloudCode and Codex. And so on. Because

36:12 consumers could reach it. Sure. Yeah. But there's also so much of this is vibes. And Peter, I had a podcast with him. He's a wonderful human being. So part of it is also the humans that represent the thing. Yeah, no doubt. Part of it is memes. Because we're all trying to figure it out. There's really serious and complicated security concerns about... when you have such powerful technology, how do you hand over your data so they can do useful stuff, but then there's scary things associated with that. And we as a civilization, as individual people and as a civilization figuring out how to find that right balance. Yeah, we jumped on it right away and we sent a bunch of security experts this way. And we did this thing called OpenShell. It's already been integrated into OpenClaw. And Nvidia put forward NemoClaw. Yeah, exactly. They install super easy. it makes sure that it's secure.

37:03 We give you two out of three rights. Agentic systems can access sensitive information, it can execute code, and it can communicate externally. We could keep things safe if we gave you two out of those three capabilities at any time, but not all three. And out of those two out of three capabilities, we also give you access control based on whatever rights that you're given by enterprise. And then we connect it to a policy engine that all these enterprises already have. And so we're gonna try to do our best to help OpenClaw become a better Claw. So you eloquently explained how we have a long history of blockers that we thought were gonna be blockers and we overcame them. But now looking into the future,

37:48 what do you think might be the blockers now that it's clear that agents will be everywhere? So it's obviously we're gonna need compute. So what is going to be the blocker for that scaling? Power is a concern, but it's not the only concern, but that's the reason why we're pushing so hard on extreme co-design so that we can improve the tokens per second, per watt, orders of magnitude every single year. And so in the last 10 years, Moore's law would have progressed computing about a hundred times in the last 10 years. We progressed and scaled up computing by a million times in the last 10 years. And so we're gonna keep on doing that through extreme co-design. So energy efficiency, perf per

38:35 watt, completely affects the revenues of a company, it affects the revenues of a factory, and we're just gonna push that to the limit so that we can keep on driving token costs down as fast as we can. Our computer price is going up, but our token generation effectiveness is going up so much faster The token cost is coming down. It's coming down an order of magnitude every year. So power, that's an interesting one. So the way to try to get around the power blocker is to try to, with the tokens per second per watt, try to make it more and more efficient. Of course, there's the question of how do we get more power. We should also get more power. That's a really complicated one. You've talked about small modular nuclear power plants. There's all kinds of ideas

39:22 for energy. How much does it keep you up at night? the bottlenecks in the supply chain of AI, like ASML with EUV lithography machines, GSMC with advanced packaging, like Coas, and SK Hennix with high bandwidth memory. All the time. And we're working on it all the time. No company in history has ever grown at a scale that we're growing while accelerating that growth. It's incredible. And it's hard for people to even understand this. In the overall world of AI computing, we're increasing share. And so supply chain, upstream and downstream, are really important to us. I spent a lot of time informing all the CEOs that I work with, what

40:09 are the dynamics that's going to cause the growth to continue or even accelerate. It's part of the reasons why, to the entire right-hand side of me, we're CEOs of practically the entire IT industry. upstream and practically the entire infrastructure industry downstream. And there were all, there were several hundred CEOs and I don't think there's ever been keynotes where several hundred CEOs show up. And part of it is I'm telling them about our business condition now. I'm telling them about the growth drivers in the very near future and what's happening. And I'm also describing where are we gonna go next? So that they could use all of this information and all of the dynamics that are

40:57 here to inform how they want to invest. And so I inform them that way, like I inform my own employees. And then, of course, then I make trips out to them and make sure that, hey, listen, I want you to know this quarter, this coming year, this next year, these things are going to happen. And if you look at the CEOs of the DRAM industry, the number one DRAM in the world was DDR memory for CPUs in data centers. About three years ago, I was able to convince several of the CEOs that even though at the time HBM memory was used quite scarcely, you know, and barely by supercomputers, that this was going to be a mainstream memory for data centers in the

41:46 future. And at first it sounded ridiculous, but several of the CEOs believed me and decided to invest in building HBM memories. Another memory was rather odd to put into a data center is the low power memories that we use for cell phones. And we wanted them to adapt them for supercomputers in the data center. And they go, cell phone memory for supercomputers? And I explained to them why. Well, look at these two memories, LPDDR5, HBM4. The volumes are so incredible. All three of them had record years in history, and these are 45-year companies. And so, you know, that's part of my job is to inform

42:31 and shape, inspire, you know? So you're not just manifesting the future and maybe inspiring NVIDIA, the different engineers of the company. You're manifesting the supply chain of the future. So you're having conversations with TSMC, with ASML, Upstream, downstream. Upstream, downstream. So that's the thing. GEV, Caterpillar. Yeah, that's downstream from us. Yeah, yeah. Yeah, the whole thing. I mean, but that's so, there's so much incredibly difficult engineering that happens in the entire semiconductor industry. And it just feels scary how intricate the supply chain is, how many components there are, but it works somehow.

43:18 Exactly. The deep science. the deep engineering, the incredible manufacturing, and so much of the manufacturing is already robotics. But we have a couple of hundred suppliers that contribute the technology that goes into our 1.3 million component rack. Each rack is 1.3, one and a half million components. There are 200 suppliers across the Vera Rubin rack. So it's interesting that you don't list that as the thing that keeps you up at night in the list of blockers. But I'm doing all the things necessary to see. I can go to sleep because I checked it off. I said, okay. I can go to sleep. I go, well, let's see. Let's reason about this. What's important for us? Okay, let's reason about this. Because

44:07 we changed the system architecture from the original DGX1 that you remembered to MVLink 72 rack scale computing. What does that mean? What does that mean to software? What does that mean to engineering? What does that mean to how we design and test? And what does that mean to the supply chain? Well, one of the things that it meant was we moved supercomputer integration at the data center into supercomputer manufacturing in the supply chain. If you're doing that, you also have to recognize you're going to move, and if you're If you're in a total footprint of whatever data center you're going to build, let's say you would like to have

44:56 50 gigawatts of supercomputers that are running simultaneously, and it takes one week to manufacture that 50 gigawatts of supercomputers, then each week in the supply chain, the supercomputers are going to need a gigawatt of power. And so we're going to need the supply chain to increase the amount of power it has to build, test, to build and test the supercomputers in the supply chain before I ship it. Well, NVLink 72 literally builds supercomputers in the supply chain and ships them two, three tons at a time per rack. It used to be, they used to come in parts and we used to assemble them inside the data center. But that's impossible now because NVLink 72 is so dense. And so that's

45:41 an example. And I would have to go into... You know, I fly into the supply chain, go meet my partners and say, hey, I said, guess what? So here's what I'm going to do with, this is the way we used to build our DGXs. We're going to build them this way. This is going to be so much better because we're going to need them for inference. The market for inference is, you know, coming. The inflection point for inference is coming. It's going to be a big market. And so I first explain to them what's going on, why it's going to happen. And then I ask them, to make several billion dollars of capital investments each. And because they trust me and I'm very respectful of them and I give them every opportunity to question me and I spend time to explain things to people and I reason about it, I

46:28 draw them pictures and I reason about it in first principles and by the time I'm done with them, there's no what to do. So a lot of it's about relationships and building a shared view of the future. Yeah. But do you worry about certain bottlenecks? I mean, what are the biggest bottlenecks in the supply chain? Are you worried about ASMLs, EUV tooling? Are you worried about the packaging, COOS packaging of TSMC, about how fast it could scale? Like you said, you're not only growing incredibly fast, you're accelerating in growth. So it feels like everybody in the supply chain, and those are certainly bottlenecks, would have to scale up. Are you having conversations with them? Like, how can you scale this faster? Do you worry about it? No. Okay.

47:15 Because I told them what I needed. They understood what I need. They told me what they're going to go do. And I believe in what they're going to do. Interesting. Yeah. That's great to hear. So maybe if we can just linger on the power for a little bit. What are your hopes for how to solve the energy problem? One of the areas that I would love, I would love, love us to talk about and just get the message out. You know, our power grid is designed for the worst case condition with some margin. Well, 99% of the time, we're nowhere near the worst case condition because the worst case condition is a few days in the winter, a few days in the summer, and extreme weather. Most of the time, we're nowhere near worst case condition

48:03 and we're probably running around, call it 60% of peak. And so 99% of the time, our power grid has excess power and they're just sitting idle, but they have to be there sitting idle because just in case, when the time comes, hospitals have to be powered and infrastructure has to be powered and airports have to run and so on and so forth. And so the question that I have is whether we could go and help them understand and create contractual agreements and design computer architecture systems, data centers, such that when they need the maximum power for infrastructure in society, that the data centers would get less. But that's in a very

48:50 rare instance anyways. And during that time, we either have a backup generator for that little part of it, or we just have our computers shift the workload somewhere else, or we have the computers just run slower. You know, we could degrade our performance, reduce our power consumption, and provide for, you know, a slightly longer latency response, you know, when somebody asks for, you know, asks for an answer. And so I think that that way of using computers, of building data centers, instead of expecting 100% uptime, and these contracts that are really, really quite rigorous, it's putting a lot of pressure on the grid to be able to, now they're going to have to increase from their maximum. I just wanna use their excess.

49:35 It's just sitting there. Yeah, that's not talked about enough. So what's stopping there? Is it regulation? Is it bureaucracy? I think it's a through-way problem. It starts with the end customer. The end customer puts requirements on the data centers that they can never not be available. Okay, so that the end customer expects perfection. Now, in order to deliver that perfection, you need a combination of backup generators and your grid power supplier to deliver on perfection. And so everybody's gotta have six nines. Well, I think first of all, right now, we ought to have everybody understand that when the customer asks for these things, you have somebody in your data center operations

50:23 team disconnected from the CEO. I bet the CEO doesn't know this. I'm gonna talk to all the CEOs. The CEOs are probably not paying any attention to the contracts that are being signed. And so everybody wants to sign the best contract, of course, and they go down to the cloud service providers and the contract, the two contract negotiators that are, I could just see them now, you know, negotiating these multi-year contracts, both sides want, you know, the best contract. As a result, the CSPs then have to go down to the, utilities and they expect the six nines. And so I think the first thing is just make sure that all of the customers, the CEOs of the customers realize what they're asking for. Now, the second

51:09 thing is we have to build data centers that gracefully degrade. And so if the power, if the utility of the grid tells us, listen, we're gonna have to back you down to about 80%, we're gonna say that's no problem at all. We're just gonna move our workload around. We're gonna make sure that data is never lost. but we can reduce the computing rate and use less energy, the quality of service degrades a little bit. For the critical workloads, I shift that somewhere else right away. So I don't have that problem. And so, you know, whoever, whichever data center still has 100% uptime. And so- How difficult of an engineering problem is that the smart dynamic allocation of power in the data center? As soon as you could specify, you could engineer it. Beautifully put. So long as it obeys the laws of physics on first

51:56 principles, I think we're good. What was the third thing you were mentioning? So the second thing is the data centers. And the third thing is we need utilities to also recognize that this is an opportunity. And instead of saying, look, it's going to take me five years to increase my grid capability, if you're willing to take power of this level of guarantee, I can make them available for you next month and at this price. And so if utilities also offered more segments of power delivery promises, then I think everybody will figure out what to do with it. Yeah, but there's just way too much waste in the grid right now. We should go after it. You've highly lauded

52:45 Elon and XAI's accomplishment in Memphis in building Colossus Supercomputer. probably in record time in just four months. It's now at 200,000 GPUs and growing very quickly. Is there something that you could speak to, understand about his approach that's instructive broadly to all the data center creators that enable that kind of accomplishment? His approach to engineering, his approach to the whole management of construction, everything. First of all, Elon is deep in so many different topics, yet he's also a really good systems thinker. And so he's able to think through multiple disciplines. And he

53:30 obviously pushes things, questions everything, whether number one, is it necessary? Number two, does it have to be done this way? And in other words, does it have to take this long? And so he has the ability to think to question everything to the point where everything is down to its minimal amount that's necessary. You can't take anything else out. And yet the necessary capabilities of the product retains, you know? And so he is as minimalist as you could possibly imagine. And he does it at a system scale. I also love the fact that he

54:16 is... He is represented. He is present at the point of action. You know, he'll just go there. If there's a problem, he'll just go there and show me the problem. You know, when you do all of this in combination, you overcome a lot of previous, this is just the way we do it. You know, I'm waiting for them. You know, I mean, it's just everybody has a lot of excuses. And so, and then the last thing is when you act personally with so much urgency, it causes everybody else to act with urgency, you know? And every supplier has a lot of customers going on. Every supplier has a lot of projects going on. And he makes it his business

55:04 that he's the top priority of everybody else's, you know, projects. And so he does that by demonstrating it. Yeah, I've been in a bunch of those meetings. It's fun to watch because really not enough people ask the question like, okay, so can this be done a lot faster and how? Why does it have to take this long? Yeah, right. And then that becomes an engineering question often. And yes, I think when you get the ground truth of actually, I remember one of the times I was hanging out with him, he literally is going through the entire process of how to plug in cables into a rack. He was working with an engineer, on the ground that's doing that task. And he's just trying to understand what does that process look like so it can be less error prone. And just building up that intuition from every single task involved in putting together the data center, you start to immediately get

55:53 a sense at the detailed scale and at the broad system scale of where the inefficiencies are. And so you can make it more and more and more efficient. Plus you have the big hammer of, being able to say, let's do it totally different and remove all possible blockers. That's right. Is there parallels in the NVIDIA Extreme Systems co-design approach that you see in the way Elon approaches systems engineering? Well, first of all, co-design is an ultimate systems engineering problem. And so we approach the work that we do from that principle. The other thing that we do, and this is a philosophy that a thought, a state of mind, I guess, a method that I started 30 years ago

56:42 and it's called the speed of light. A speed of light is not just about the speed. A speed of light is my shorthand for what's the limit of what physics can do. And so everything that we do is compared against the speed of light. Memory speed, math speed, power, cost, time, effort, number of people, manufacturing cycle time, and when you think about latency versus throughput, when you think about cost versus throughput, cost versus capacity, all of these things, you test against the speed of light to achieve all of these different constraints separately. And

57:29 then when you consider it together, you know you have to make compromises because a system that achieves extremely low latency versus a system that achieves very high throughput, are architected fundamentally differently. But you want to know what's the speed of light of a system that achieves high throughput, what's the speed of light of a system that achieves low latency. And then when you think about the total system, you could make trade-offs. And so I force everybody to think about what the first principles, the limits, the physical limits, for everything before we, you know, before we do anything. And we test everything against that. And so that's a good frame of mind. I don't love the other methods, which is continuous improvement.

58:19 The problem with continuous improvement, first of all, you should engineer something from first principles at the speed, you know, with speed of light thinking, limited only by physical limits. And and physics limits. And after that, of course, you would improve it over time. But I don't like going into a problem and somebody says, hey, you know, it takes 74 days to do this today, right now, and we can do it for you in 72 days. You know, I'd rather strip it all back to zero and say, first of all, explain to me why it's 74 days in the first place. And let's think about what's possible today. And if I were to build it completely from scratch, how long would it take? Oftentimes you'd be surprised and might come to six days. Now the rest of the

59:09 six days to 74 could be very well reasoned and compromises and cost reductions and all kinds of different things, but at least you know what they are. And then now that you know that six days is possible, then the conversation from 74 to six, surprisingly much more effective. In such incredibly complex systems that you're working with, is simplicity sometimes a good heuristic to reach for? I mean, if I can just... I mean, the pod, the Vera Rubin pod that you announced is just incredible. We're talking about seven chips, seven chip types, five purpose built rack types, 40 racks, 1.2 quadrillion transistors, nearly 20,000 NVIDIA dies,

59:56 Over 1,100 Rubin GPUs, 60 exaflops, 10 petabytes per second of scale of bandwidth. That's all just one. That's just one pod. That's just one pod. Yeah, that's just one pod. I mean, so you have the, and then even the NVL72 rack alone is 1.3 million components. 1,300 chips, 4,000 pounds crammed into a single 19-inch wide rack. And Lex, we'll probably kind of crank out about 200 of these pods a week, just to put in perspective. The amount of different components, I suppose simplicity is impossible, but is that a metric that you kind of reach for in trying to design things? You know, the phrase that I use most often is we need things to be as complex as necessary, but as simple as possible. And

1:00:44 so the question is, is all that complexity there necessary? And we ought to test for that. And we ought to challenge that. And then after that, everything else above it, you know, It's gratuitous. But it's some of the most incredible semiconductor industry broadly, but what NVIDIA is doing, some of the greatest engineering in history. So these systems are just truly, truly marvels of engineering. It is the most complex computer the world has ever made. Yeah, the engineering teams, I mean, I don't know. It's not a competition, but I don't know. If it was like an Olympics of engineering teams, I mean, TSMC does incredible engineering. Like I said, ASML at every scale. But NVIDIA is going to give them a run for their money. Just incredible, incredible teams. Gold medalists in every single sport all assembled right

1:01:33 here. And have to work together and report directly to you. This is wonderful. You've recently traveled to China. So... It's interesting to ask you, China has been incredibly successful in building up its technology sector. What do you understand about how China is able to, over the past 10 years, build so many incredible world-class companies, world-class engineering teams, and just this technology ecosystem that produces so many incredible products? A whole bunch of reasons. Well, first of all, let's start with some facts. 50% of the world's AI researchers are Chinese. or minus and they're mostly in China still we have many of

1:02:19 them here but there's amazing researchers still in China um they their tech industry showed up at precisely the right time at the time of the mobile cloud era their way of contributing was software and so this is a country's incredible science of math really well educated kids um Their tech industry was created during the era of software. They're very comfortable with modern software. China is not one giant economic country. It's got many provinces and cities with mayors all competing with each other. That's the reason why there's so many EV companies. That's the reason why there's so

1:03:05 many AI companies. That's the reason why there's so many... Every company you could imagine, they all create some of them. And as a result, they have insane competition internally. And what remains is an incredible company. They also have a social culture where it's family first, friends second, and company third. And so the amount of conversation that goes back and forth between, they're essentially open source all the time. So the fact that they contribute more to open source is so sensible because they're probably, what are we protecting? My engineers,

1:03:55 their brothers are in that company, their friends are in that company, and they're all schoolmates, the schoolmate concept. One schoolmate, your brother for life. And so they share knowledge very, very quickly. And so there's no sense keeping technology hidden. You might as well put it on open source. And so the open source community then amplifies, accelerates the innovation process. So you get this rapid, incredible, great talent, rapid innovation because of open source and just, you know, the nature of friends and insane competition among the company. What emerges is incredible stuff. And so

1:04:41 This is the fastest innovating country in the world today. And this is something that has everything that I've just said is fundamental to just how the kids were grown, the fact that they have excellent education, the fact that parents want them to do well in school, the fact that their culture is that way. These are just the thing about their country. And they showed up at precisely the time when technology is going through that exponential. Plus culturally, It's pretty cool to be an engineer. It connects to all the components that you're mentioning. It's a builder nation. It's a builder nation. Yeah, it's a builder nation. Our country's leaders, incredible, but they're mostly lawyers. Their country's leaders, because they're trying to keep us safe, rule of law, governing,

1:05:31 their country was built out of poverty. And so most of their leaders are incredible engineers. some of the brightest minds. To take a small tangent, because you mentioned open source, I have to go to Perplexity here, who you've been a fan of a long time. I love it, yeah. And thank you for releasing open source Nemetron 3 Super, which you can also use inside Perplexity to look stuff up, which is a 120 billion parameter open weight MOE model. What's your vision with open source? So you mentioned, China with DeepSeek and Minimax with all these companies really pushing forward the open source AI movement and NVIDIA is really leading the way

1:06:22 in close to state of the art open source LLMs. What's your vision there? First, if we're going to be a great AI computing company, we have to understand how AI models are evolving. One of the things that I love about Nemotron 3 is it's not a just a pure transformer model, it's transformer and SSMs. And we were early in developing the conditional GANs, that progressive GANs, which led step-by-step to diffusion. And so the fact that we're doing basic research in model architecture and in different domains gives us visibility into what kind of computing systems would do a good job for future models. And so

1:07:10 it is part of our extreme co-design strategy. Second, I think we rightfully recognize that on the one hand, we want world-class models as products and they should be proprietary. On the other hand, we also want AI to diffuse into every industry and every country every researcher, every student. And if everything is proprietary, it's hard to do research and it's hard to innovate on top of, around, with. And so open source is fundamentally necessary for many industries to join the AI revolution. NVIDIA has the scale and we have the motives

1:07:58 to not only skills, scale, and motivation to build and continue to build these AI models for as long as we shall live. And so therefore we ought to do that. We can open up, we can activate every industry, every researcher, every country to be able to join the AI revolution. There's a third reason, which is recognizing that AI is not just language. These AIs will likely use tools and models and sub-agents that were trained on other modalities of information. Maybe it's biology or chemistry or laws of physics

1:08:44 or fluids and thermodynamics. Not all of it is in language structure. And so somebody has to go make sure that whether prediction, biology, AI, AI for biology, physical AI, all of that stuff stays can be pushed to the limits and pushed to the frontier. We don't build cars, but we want to make sure every car company has access to great models. We don't discover drugs, but I want to make sure that Lilly has the world's best biology AI systems so that they can go use it for discovering drugs. So these three fundamental reasons, both in recognizing that AI is not just language, that AI is really broad, that we want to engage everybody into the world of AI, and then

1:09:30 also co-design of AI. Well, I have to say, once again, thank you for open sourcing, really, truly open sourcing Nemetron 3. Yeah, I appreciate you for saying that. We open source the models, we open source the weights, we open source the data, we open source how we created it. Yeah, it's pretty amazing. It's really incredible. You're originally from Taiwan and have a close relationship with TSMC. So I have to

…