0:00 Across the planet, entire landscapes are getting wiped and rebuilt. Fields, suburbs, open land, all turning into AI data centers almost overnight. And each new one is bigger, pulling more power and more water. The scale keeps rising. The physical limits stay put. Silicon and electrons carried us as far as they could and that gap can only be closed by an entirely new technology. It's the hottest problem in the industry today and a search for a solution triggering a new gold rush. I am coming from chip design and in this industry it's quite rare that you see a shift that big and even
0:46 more rare that you can explain it as it happens. So in today's episode, we will cover three key innovations that form the blueprint for how AI data centers will scale over the next two decades. Subscribe to the channel and let's get into it. A single next generation AI campus now draws 1 to 2 GW of power. That is the same order of magnitude as an entire metropolitan area. In some regions like Virginia, data centers already swallow a quarter of the grid and we are still just getting started. Inside these buildings, millions of GPUs crunch data and dump heat into the air. And once
1:34 this heat hits the room, it has to be pushed away just as fast. Almost 40% of the total power inside a modern data center doesn't go to the chips at all. It goes straight into cooling, eaten up by giant chillers humming in the background. The real problem is not even this massive compute at this intense AI workloads. It's this sheer volume of data being moved because we simply moving too much data and doing it wrong. Because AI data centers aren't just bigger data centers, they are something entirely different. This is Meta's Hyperion project in Richland Parish, Louisiana.
2:21 The largest AI data center ever attempted in North America. Hyperion is a computer the size of Manhattan. And this is just one AI cluster planned to scale to 5 GW. Inside over 1 million GPUs, acting as one single AI supercomputer. What used to be a warehouse of computers has now become a single computer this size of a city. And when a single workload spans an entire building, every link delay becomes a choke point. And here the network stops being the connection and becomes the computation itself. The bottleneck is no longer inside the AI chips. It lives between them.
3:08 And this is why the new gold rush in AI data centers is actually in the network itself. Who is getting rich when there is a gold rush? The guys who sell the shovels. Now let's find the shovel sellers. At the center of all of it is copper. The metal that carried the entire digital revolution. Right now it's everywhere inside data centers between chips across boards. across racks. A single hypers scale AI site can swallow 50,000 tons of it. For decades, copper was familiar, perfect, cheap, and easy to route. But today, it's exactly the thing that holds us back. At modest
3:55 speeds, copper can carry signal over hundreds of meters. But at tens of gigabits per second, that shrinks to just a few meters. So companies are trying to get their racks as close as they can and lace them together as tight as possible with copper. But obviously that's not something that's going to scale going forward. And if you try to push signal even faster, you are getting down to just a few inches because the faster you're trying to push the signal, the more it's getting attenuated. Basically the faster it travels, the weaker it becomes. that at some point it just becomes unusable. To keep the signal alive, you stack equalizers,
4:40 amplifiers, retimers, and each of these burning more power. And suddenly every cm cost you even more power and heat. And this is the biggest pain point in AI infrastructure today. At some point, you can double the number of processors and see almost no speed up because the network is suffocating them. At this point, copper has taken us as far as it can. And the only medium left with enough runway is light. Because once information rides photons, all the old constraints fall away. Light plays by a completely different set of rules. Photons do not grind through resistance in
5:28 metal. They do not spill energy as heat when you push them to move faster or further. You finally get a medium where sending more data does not mean fighting more heat. It's like switching from a car which is constantly stuck in the city traffic to a train that is gliding on its own rails. And optics is already working beautifully in a distance. It's a backbone of every data center, every rack-to-rack switch, every long haul fiber that keeps internet alive. Right now, optics is literally everywhere except for the place where we need it the most, right next to compute on the
6:15 chip. There is still an electrical path between the chip dice and between the chip and those optical switches. And that short copper hop is where all the power, latency, and heat spike. And inside an AI supercluster, that difference is everything. This last few centimeters bringing optics to the chip is where the entire industry has been stuck for over a decade. And this is where things start to get really interesting because bringing optics to the chip sound quite simple, but it was actually a physics nightmare. The problem is classical photonic parts
7:01 force a painful tradeoff. The device you see on the screen is basically a transistor of the photonic world. It's so-called modulator. It takes your data and turn it into light. For now, you can imagine it as a device that shapes and controls photons the way transistor controls electrons. As you can see, some of these devices are huge by microchip standards. Look at this Mach-Zehnder interferometer. It is obviously too big to place it on a tiny chip next to nanoscopic transistors. Others are tiny like this micro ring modulator, but they drift with the slightest temperature
7:47 change. These are so sensitive that even a 2° temperature shift knocks them out of tune, while high performance GPUs leave through brutal temperature swings. Imagine hundreds of watts of power packed into a few square cm of silicon. Here temperatures jump from 30° C at idle to 90° C under training load. Sometimes it happens in a few seconds. Thermal stability becomes the hill where the most of photonic ideas die. This is why every skeptic argued that this would never work. How can you put something so sensitive next to something so hot? And for the last 20 years, this is exactly
8:35 where photonics failed. These last 2 centimeters are the most important 2 cm in a data center. And now for the first time, we can actually cross them. If you look at what happened over the last years in a nutshell, it's fun because it's sort of a steady march of flight trying to get closer and closer and closer to the transistor. At first, optics lived far away out in the cables safely isolated from heat and noise. Then they moved onto the motherboard. Then co-packaging brought them right beside the compute die. And each step was a small rebellion against physics.
9:20 And now we are chasing the final form, the optical interposer where light moves between compute dies the same way copper does today. That's the real end game. And this is why this gold rush is so fierce because the one who cracks it will control AI infrastructure of the future. But to get there, we still have two problems to solve. First, we need a good on-chip modulator. A device which is tiny enough, fast and can survive huge temperature changes. Second, we need something that can actually generate light inside the chip. At the same time, live inside this thermal chaos. Today, this light source usually sits outside the package. But we want to bring it in. And
10:11 this is where everyone been stuck because light hates heat. Silicon hates light. And traditional photonics was never designed to sit inside a multi-kilowatt package. But now something changed. A big innovation, a breakthrough cracked both of these problems. And we will deep dive into it in a moment. But before that, here is the same idea playing out right on your desk. A good computer today is no longer just convenience. It shapes how efficiently you think, build, and ship results. Whenever I build a PC, I start with the part that matters the most, the processor. And here I go for
10:56 the one that can actually keep up with my insane workflow. That's AMD Threadripper Pro 9000 series. This one has highest specs for professional workloads. Here you get massive memory bandwidth, huge RAM capacity and enough PCIe lanes to drive multiple GPUs and fast storage without slowing anything down. Threadripper Pro has 96 cores and up to 192 threads. While AI development scales with course, you get faster training and inference and much better responsiveness when running models locally. It's also essential when you're running large engineering simulations or any workloads which pushes large data sets through the system. These workloads
11:46 often involve sensitive data. The AMD secure processor provides hardware level protection. It verifies code before it runs and blocks anything untrusted. And this is essential for enterprise and research environments. AMD Threadripper Pro 9000 series is targeted at professional workstation users, especially across the fields of engineering, research, manufacturing, design, and AI. Check it out through the link below. And thank you AMD for sponsoring this episode. Now back to the gold rush. Traditional photonics was never designed to survive inside the hot GPU package. For 20 years is where every optical dream was challenged until one group refused to give up.
12:38 Imec. It is a small semiconductor research hub in Belgium. a nonprofit semiconductor lab where the biggest tech companies test ideas long before they reach industry. Imec is strongest whereas the rest of the world struggles the most materials. Over the years they kept looking for something better than silicon and they found one. The entire modern world runs on silicon. It switches current beautifully. It holds charge well. It lets you to pack billions of transistors into a single die. But the moment it needs to emit light, the physics refuses to play along. To generate a photon, you need a clean direct energy drop inside the crystal. Silicon doesn't give you that. Instead
13:29 of emitting photons, it's just loses energy as heat. That's why we need something better, something that actually likes producing photons. Imec went through almost the entire Mendeleev Table, testing crystals, tuning chemistries. Most materials broke down. Too lossy, too fragile, impossible to grow on real wafers. But one kept rising to the top. Gallium arsenide. It blends light cleanly, runs cooler. If you want a stable, bright, efficient light source, gallium arsenide is at the top of the food chain. But there was actually one huge problem. Manufacturing. Silicon
14:17 comes on 300 mm wafers and we have mastered that world. Gallium arsenide wafers are small, expensive, and mechanically nothing like silicon. For most of the industry, that clash was the end of the road. This means you could pick silicon or you could pick gallium arsenide, but not both. While we need them both at the same time layered on top of each other so we can bring light right above the compute so that light and logic can finally leave on the same chip. Through iterations, they figured out how to grow tiny gallium arsenide lasers directly on silicon. Imec's trick was simple in idea but hard in execution. Instead of trying to force a
15:10 big block of gallium arsenide on top of silicon and watching it crack, they craved tiny V-shaped trenches into silicon and grew the material inside those microscopic channels. These walls trapped the defects before it could spread. The crystal aligned correctly. And for the first time, gallium arsenide could sit on silicon without tearing itself apart. That tiny geometrical trig changed everything. And now we can build lasers on silicon and reproduce it at mass scale. But this breakthrough exposes a bigger challenge. Now we need to find a way to encode data into that light. Remember those modulators? Now we need a device which is tiny and reliable and which
16:02 can sit right next to the chip and survive these extreme temperature swings. Silicon, the backbone of every chip you ever touched, just can't do it. It's too slow, too lossy. Great for logic, but terrible for bending light at this extreme speeds. So, the search continued. Labs started tearing apart the periodic table, looking for something tougher, something faster, something that wouldn't fall apart at 90° C. And that trail leads straight to Silicon Germanium. That's quite a legendary material for photonics. on periodic table. It sits right below silicon and
16:48 it's close enough to cooperate but still different enough to do things which silicon can't. It react fast to tiny electric fields and it stays stable even when the temperature swings wildly. Honestly, I wish I handle stress the way Germanium handles temperature swings because this single difference changes everything. We knew that Germanium is great for modulation at least for a decade. But one huge problem stopped everyone manufacturing. Silicon and silicon germanium lattices are not naturally compatible. On paper they should work, but the moment you force them into the
17:34 same crystal, everything clashes. Their crystal lattice don't line up. That mismatch builds stress between the layers, create defects, dark current, and long-term reliability problems. So, Imec had to fix the material itself. And they did it the hard way layer by layer. years of optimizing how Germanium grows on silicon, controlling stress, tuning, dopping to keep dark current low, shrinking the footprint until the device was both fast and stable. As a result, it finally behaves well on top of silicon wafer. And now everything snapped into the right place. With that,
18:21 they build a modulator device that can push about 440 gigabits per second per lane. That's more than the double of the industry standard. Just think about it. This tiny microscopic device can fire half a trillion pulses every second. This one device moves as much data as an entire bundle of high-end copper lanes with a fraction of the power for AI systems and especially for SerDes. This is massive. Now combine these two innovations we discussed, laser and silicon germanium modulator and we can finally cross the last two cm and bring light right to the chip. That's why
19:11 Celestial AI went all in on Germanium. They spent years building their own Germanium modulators. At the Hot Chips conference this year, they were showing chiplets at speeds that made copper and MZIs look old. And just some days ago, Marvell bought them for more than $3 billion. This is why everyone calls it an "all or zero bet" because the outcome determines who controls the gateway 200 terabit per second per rack. While some teams bet on this innovative Germanium modulators, others bet on completely different path new TSMC COUPE technology and this one is based on micro
19:59 ring modulators and typically TSMC knows what they're doing. The next big thing is being built inside TSMC factory so-called COUPE technology compact universal photonic engine. Think of it as a complete optical module inside a single package. Inside COUPE, you have an electronic chip tightly integrated with a photonic chip. The electronic chip drives modulators and read photo detectors while the photonic chip roots the light and modulates it. Here TSMC manages to bond them micrometers apart. So, electrons and photons hand off almost instantly. And that stack does
20:45 the whole optical pipeline in one place. It takes electrical signals from the accelerator or a GPU, turn it into light, send that light over fibers, catch it on the other side, and convert it back into signals that GPU or an AI ASICC can actually use. And that's a big deal because it unlocks terabit speeds, slashes power by at least three times, and finally gives us a clean path to scale it all the way to million GPU AI factories. And just a few days ago, Ayar Labs unveiled the first chip built using COUPE technology. And the biggest semiconductor giants are already lining up to get
21:32 their hands on it. While the biggest players are betting on COUPE, a couple of bold teams are taking a huge moonshot on Germanium. But for Germanium, they have to still figure out actually the hardest part reliability. And the stakes are very high here because if Germanium bet eventually work out, if these devices survive long-term stress, keep dark current low, that will be the huge success story. This is a multi-billion dollar market. This will change how we build and scale AI data centers and that's why entire industry is watching it so closely. And if you want to stay ahead of shifts like this and be informed about the most important trends
22:21 in the industry in chips, data centers, and AI, remember to subscribe to the channel. But probably the most interesting company in AI right now is Lightmatter and they are not building bigger GPUs. They're going after the most annoying problem in AI infrastructure, the last two centimeters. This is Passage. It's an optical layer that sits under the processor. It lets many chips inside one giant package talk directly to the outside world using light. It works by sending data vertically from each chip into a photonic layer, then out through optical waveguides. So every chip has its own fast
23:09 exit lane. That technology is the future. As we build larger and more powerful chips, we pack more chiplets into one package. Passage enables all chips in a huge package to send and receive data at the speed of light without waiting in line. That means you can build much larger chips and keep them all busy instead of idle. So, Lightmatter is really the bleeding edge on how fast these systems can be. We're about 8 to 10x faster than any of the companies that are out there. People are announcing 8 TB and 6 TB. we're at 64 and Passage has a couple unique properties. one of them is that it allows you to build very large chip packages. So, if you look at
23:56 the way that people do interconnect today for scale out networking and some, you know, forms of scale up, they use these pluggable optical engines. Pluggable optics, they cost, let's say, roughly 70 cents per gigabit per second. The cost of integrated optics like what we do at Lightmatter is significantly lower. So you actually have a lot more performance for the same amount of money. So it's kind of an obvious win and that's why the industry will adopt it. It's clear that the future of AI factories will run on light, not copper. And the technology we discussed today form the new blueprint. Optics is about to become the real continuation of the Moore's law. Your speed is no longer about how many transistors we can pack on a chip or how fast this chip is,
24:47 but how fast can hundreds of thousands of AI chips behave as one computer. If you zoom out and look at the bigger picture, every jump in human capability at civilization level came basically from two things. Better ways to compute and better ways to move information. Writing let knowledge survive over time. Transistors let us automate thought and build thinking machines. The internet linked billions of minds and machines at once. AI is the next layer. The story of human progress is about how fast information can move through our tools. And copper got us to the first smartphones and early AI. Light is what will push us to the next milestone in our civilization and
25:39 right now optics becomes the next main driver in how computing evolves and with it humanity itself. If you enjoyed this episode, you will definitely love this one where I explain the largest AI data center without a single GPU. Or this one where I take you inside a secret microchip factory where the future of technology is being invented before it comes to your phones in 2030. Check it out. And I wish you happy, warm, beautiful holidays. Thank you so much for watching this channel this year, for being a part of the community. I love you guys and I'm going to see you in 2026. ciao
