Full transcript of “중국 반도체 주가 폭등... 딥시크 V3.1 출시, NVIDIA GPU 필요 없는 기술적 이유 | FP8, FP4”. Click a timestamp to open the video at that moment. Need another video? Use the YouTube transcript generator.
0:00 Do you know what the difference is? The United States has a very fragile power grid. I'm really low on energy. But China has been very well prepared for all kinds of energy since long ago. So, as long as you can achieve computing, you can use a lot of organizing power. Yes, nice to meet you all. This is an error. DeepK has a new model. The model you are looking at right now is DeepK V3.1, model 1. The model that I rowed so hard for that was the one that caused a stir before was called the Deepk R1. I told you that the R1 model was based on the DeepK V3, but what has come out this time is not the DeepK R series, but the next version of the V3. Actually, Deep Seek is a bit out of style now. That feeling is not so fancy anymore, and from the perspective that there are a lot of famous things coming out now, I wasn't really that
0:46 surprised or anything like that . But rather than the fact that the performance of this model itself has improved, what is important is what Deep Seek said this time? We introduced a technology that we created in the optimization of Chinese semiconductors. It's at a stage where we can only make guesses, but this will not depend on MBDIA GPU in the future. China will become self-reliant and incorporate AI semiconductors . Well, recently, there's been talk at CXMT that HBM 3 will soon be mass-produced . So, even though the US imposed sanctions, DeepSec continued to improve its AI model performance and open-sourced it, even leading to OpenAI being made open. Hardware is also continuously improving to a feasible level. Let's take a look at some of the technical points we need to understand . These are the parts that became a hot topic.
1:32 DeepSeek posted this. With just this one statement, the entire Chinese semiconductor stock market skyrocketed. What do you mean? When DeepSec released version V3.1, they left a clear message that it was a new architecture and next-generation Chinese chip . And then they say they used a technology called UE8M0 FP8, and this itself caused a huge stir in the AI community . Reporter Park Chang of AI Times has summarized it well, but the stock price of Cambricon Technology has fallen by up to 20%. There's a lot of talk right now that DeepSec uses semiconductors from Cambricon Technologies . And you can see that SMIC and Huayong Semiconductor are also rising by a huge amount, 10% and 18%, right? That's why this kind of story comes up. The Chinese government is saying that they can block the purchase of GPUs from H20, that is, MByte, because they are confident that there are alternatives, but it
2:22 seems that they are not banning H20 or anything like that, but rather rescuing semiconductor chips from Huawei, such as Ascend, or Cambricon. That's why it's scary. Before going into more details, let me briefly explain the DeepScrew V3.1 release. First of all, DeepScrew V3.1 also uses a 1-degree hybrid release model . A mode where you can think as one model and a mode where you can think as another. This is the trend these days. As you know, pink mode is used for things like inference, coding, and math problems. Previously, a specialized model like R1 was used, and now chain of sorts are used to think deeply step by step . They focus on logical answers like this, and say that the number of tokens in the derivation process has been reduced by about 20 to 50% compared to the previous model. So, they're claiming that they can now generate answers a bit faster, and then they're talking about the agent upgrades, and these benchmarks have gone up a lot. Of course, there are some parts that were compared to Deep Seek before, but overall, it has gone up considerably. Well, actually,
3:08 even when I look at the benchmarks, there are some parts that are a bit less exciting. In fact, if you look at Ariphysical Intelligence, you can see that DeepSec V3.1's intelligence is still lower than that of the GPT open model, so you can't do it . But, as for the performance improvement of the AI model itself, we will probably have to wait for the next R2, which is an improvement on V3.1, to come out, and the difference in degree is not that big. So we are working together to prepare semiconductors now . A much more important point is that you are using the right technology for the job. But anyway, things like Code A agents and search agents have gotten a lot higher than before. This is also open source now. Our country's companies are also bringing in a lot of these and will be using them now. I'm still using it though. And the context window has now grown to 128K . The number of windows that can be input at once has increased significantly. This is also very welcome news. When testing whether De
3:54 Yoren could do complex coding and other things like SW Bench, it was said that it showed superior performance to Claude 3 Offs. There are people who do a lot of Claude coding, right? But the fact that it has been released this much as open source means that even if it is released as open source, ordinary people will be able to run it because it has 670 billion parameters . But, you know, we tell people to use something, but there are so many people all over the world, including our country, who can't make foundation models . But if you can take this and build it into a server and run it, that would be really great. There are some parts where it is expected that Chinese AIs are infiltrating while encroaching a little bit like this, and I have given some detailed explanations, but I think this is the most important part among them . This is what I'm getting attention for right now. What is this thing called DeepK V3.1? It is used in UE8M0 FP8 scale data format
4:40 . As I showed you earlier, even if the US imposes sanctions on semiconductors, Nvidia's high-speed chips, such as H100, are still being sanctioned, and H20 and other things are still being banned. Originally, DeepSeek also used the H800, but when they released the V3.1 model, they officially announced that they designed it for the next-generation Chinese chip. This is the official version. So, what we're saying is that DeepSeek not only creates AI models, but also does something to fit the hardware . So, to understand what this FP8 yogurt is and what U28M0 is, you need to understand FP a little bit. As many of you may know, let's take a look at this . This is a while back when Nvidia was talking about how much Black has improved, and the computational power of the GB 200 has improved tremendously over the past 8 years. Originally, it was only 19 teraflops, but it was Pascal. Oh, and even if it was V and A bat, it was 620, but it
5:27 went up to 20,000 now . But when this came out, there were voices from AI engineers criticizing Nvidia. Because look down here. It's the same Terrafloss, but if you look below, it goes to FP 16 and then changes to FP8 in the hopper. Then it changes to FP4. So, to show that it goes up to the extreme, even at this time, there was a lot of talk that among AI models, not only learning with FP4, but also inference was very poor. In particular, learning is not going well, and it is understandable that they reduced the number of bits used per parameter from 16 bits to 8 bits, but it is nonsense to express it as a real number using only FP4 4 bits . There were some people who argued like this. But I think people who say things like, "Even if you're an expert in this field, it's nonsense in the AI era," should be careful. The reason I do n't insist on this so strongly is
6:15 because it's developing so quickly. At the time, not many people thought that the AI model would be FP4, but Nvidia was looking a little further ahead and waiting for someone to do it. And now we have reached a level where FP4 is possible . So, we need to understand what this FP is . FP is just a number. It's 10 billion parameters, 175 billion parameters, 1 parameter. After all, that parameter is just a number. I think there are a lot of numbers in the numerical neural network, and we actually store those numbers. If we use our hands, for example, if we write a file, we will write it like this: 3.141592 65358979. Let's think about actually putting that into HBM memory . In HB memory, binary numbers like 0 or 1 are converted and stored, but depending on whether that number is expressed as 16 or 8, it can make a huge difference . What's different? This means that the amount of memory used to store each number
7:02 varies. For example, if this is FP8, it means that it expresses the same real number but uses eight times as many bits . FP4 uses only four bits to represent a number, so this is now 16 bits, which is less than the four bits used. Then, compared to FP16, you can use less memory. You can use eight times less than FP 32 . So the reason why GPUs are used a lot now is because there is one HBM capacity attached to the GPU. So, you have to use several of these by linking them together, and since communication between GPs and servers in MV Link, MV Switch, and InfiniBand all use HBM capacity bottlenecks, if you can make an AI model that can run it by making the floating point smaller, it is that much more advantageous . But this is how the preseason came out. If we express numbers by writing all the numbers from 1 to 100 and you
7:47 only write 1 and 100, the accuracy will inevitably decrease. The key question is whether we can increase accuracy even if we reduce the number of bits like this . When we talk about quantization and limit it to AI models, it's a very classic field. Precision is a calculation that lowers the floating point or precision in order to digitize analog numbers . Then, there is a sign part that represents plus or minus, and there is an exponential part that simply represents the exponent part, and there are parts that represent the mantissa part like this. So Nvidia prepared. Of course, M-Bee also creates AI models, but most people create AI models, and everyone knows this . The moment you go from FP8 to FP4, you just sit there and the amount of computing doubles even if you use the same hardware . A part that visually expresses that well
8:34 is the part of the announcement made by Nvidia at Dance Party. If you look at this. Okay, so the horizontal axis is the response speed. The response speed refers to how quickly a response can be made per user, and the vertical axis is throughput. So, we're talking about how many people can be processed at once, and if we go from FP8 to FP4, the response speed is also very fast. It's not only that FP4 can respond much faster than FP8 for the same axis, but even if we just receive it based on the loop standard, you can see that it can process a lot of people for the same user, right? That means you can process much more with the same power . In the first place, if you go to FP4, the amount that can be processed per unit will be reduced, so you can do it with a little more memory bottleneck . DeepSeak has also done a lot of research related to FP4. We've been doing a lot of research on how to output these things in association with FP4 quantization in BF 16. From China's perspective, they
9:21 can't get many Nvidia indicators, so they've been doing a lot of research on ways to reduce the preseason in order to do more calculations within the hardware they have . But, when you look at this new concept called UE8M0 FP8 that came out this time, it's called UE8M0 . I was personally guessing that the part called M might be the part that corresponds to the mentor, and I happened to see that there were Chinese media outlets that analyzed this as well. If we look at this as using 4 for exponential and 3 for mentee, the exponent part of the power of how many times will be used as 4 bits and the mentee part will be used as 3 bits. But isn't the U measured as unsigned? So, without even talking about the signature, only the exponential part is expressed as victory, and the mentor singer part is not expressed at all . So, I guess that means that in the singer section, there will be
10:06 no placement at all for any data expression part, and only the part about scaling with an exponent will be expressed . Now then, look at it. Just 2 to the power of 1, so 2. 2 to the power of 7 is 128. It means that you can do a very wide range. It seems that they have found a way to reduce the error rate in the floating point 8 method to minimize information loss while sufficiently covering such a wide range, and it appears that Chinese companies are preparing AI semiconductors that support this . Actually, I did some calculations here, and there are small models like the Jara Rama 3 that cost 8 billion, and there are models that cost 70 billion. And GPT 3 is 175 billion, and if you recently reach GPT 4, it's over 1 trillion. Based on FP 16, you need up to 4 to run something like GPT 4 . If you need something like GPT3, you need up to five. It is very much needed. Ordinary people wouldn't be able to run this, but even if you go from FP8 to FP4, you
10:53 can only run two more in the case of GPT 3 . It can really reduce a lot. So, as the number of floating point bits decreases, the required capacity itself becomes much smaller, so it would be much less expensive to use 0.5 baht per number than to use 2 baht per number. Of course, this is only at the parameter level, but in addition to this, the intermediate result values and data that are responsible for memory, called the KV cache, are all stored, so although it is a rough calculation, it can be reduced to this extreme . From that perspective, I recently mentioned the Chinese chip, the Cloud Matrix 384, which is made by Huawei. There's this GPU called the Ascend 9C, which a lot of people are criticizing. Well, they say it's not as good as Nvidia, but if you link all 384 of them together, the overall computing power increases significantly . The cost-effectiveness will drop. Obviously the power-to-performance ratio will be lower,
11:38 but do you know what the difference is? The United States has a very fragile power grid. I'm really low on energy. But China has been developing wind power, solar power, and even hydroelectric power for a long time. Well, I've been very well prepared for all kinds of energy. So, as long as you can achieve computing, you can use a lot of organizing power. However, since MB is not coming in, we are also considering lowering the unit level from FP8 to FP4, and if this happens, it seems that the AI model that China can become self-sufficient in AI semiconductors has already become self-sufficient. The most promising candidate right now seems to be Cambricon, but since it's already natively FP- native, does the hardware fully support FP itself? So, when we use the term native, if it is not native, a little bit of conversion is needed in software. Just because we use FP8 doesn't mean we have to use it unconditionally. If the hardware doesn't support it, it only supports FP 16, so we need to convert FP8 to FCP 16 and
12:24 process it . In fact, even in the case of Huawei, I know that the 910C does not support FP8 natively . Since it seems that the next version of the DeCha, such as the 90 D, will support it, it seems that DeepSec is probably collaborating with Huawei's next version or Cambricon's next version to make this compatible. It seems that they are collaborating like this at a national level. So, as the US imposes stronger sanctions on Chinese semiconductor companies, SIC is also having a hard time because it is only producing DUV, but it seems that they are looking for ways to optimize in such a situation . It's not just about making it more detailed by simply using multi-patterns, but the Unsigned X8 Manti 4 brought in a lot of technologies like Zero and FP8 to make it a usable level. Even if only the infrastructure for useful hardware like this is built, AI models can continue to develop and be
13:10 made to the highest level, so if this combination continues to be successful in the future, it seems that there are parts that judged that they can continue to take the lead in AI based on the domestic market, and so this time, rather than a simple term like this being posted as something like, "Oh, I guess FP8 or something," there are parts that can be seen as the start of a trend of mergers between Chinese AI model- making companies and hardware-making companies, so I organized it . We will continue to update you with relevant information . It was an error so far. [music]
