China Just Dropped 1 Trillion Parameter AI Model That Shocks OpenAI

0:02 China just dropped a 1 trillion parameter AI model. And the crazy part is, it actually became faster and smarter after deleting a third of itself. This thing is now beating some of the biggest AI systems on Earth in real-world tasks. So what exactly did they build here? Alright, so the model is called Yuan 3.0 Ultra, developed by a group called Yuan Lab AI. And it's built using a system called Mixture of Experts, which has quietly become one of the most important architectures in modern AI. Now, the first number you'll hear about this model sounds crazy. Yuan 3.0 Ultra has roughly one trillion parameters in total, with about 68.8 billion parameters activated when it runs. That already puts it in the same scale category as some of the largest

0:50 models in the world. But the real story here is actually the opposite of what people usually expect. Normally, AI models keep getting bigger, Bigger networks, more GPUs, more compute. That's been the pattern for years. Yuan Lab did something different. They started with an even bigger model and then cut a huge part of it away while training it. The original version of Yuan 3.0 Ultra had around 1,515 billion parameters. During development, the researchers discovered that a big portion of those parameters were barely doing anything useful. So they built a new system that automatically finds those weak parts of the model and removes them while the training is still happening. By the time the final model was finished, it had about 1,010 billion parameters, which means

1:39 roughly 33% of the original model disappeared during training. And here's the surprising part. Instead of slowing the model down, this actually made it train much faster. Training efficiency improved by about 49%. So naturally, the question becomes, how can removing a third of a trillion parameter model actually make it work better? To understand that, you need to look at the architecture they're using. Yuan 3.0 Ultra is a mixture of experts model, often called MOE. The idea behind MOE is actually pretty clever. Instead of one giant neural network doing everything, the model is divided into many smaller specialized networks called experts. When a piece of text enters the system, the

2:25 model doesn't activate every expert. Instead, it chooses a few experts that seem most relevant for that specific token or sentence. Think of it like a huge company with thousands of specialists. Instead of asking every employee to work on every task, the system routes each problem to the few employees best suited for it. This design allows the model to grow massively in total size without requiring the full network to run every time. But mixture of experts' models have a hidden weakness. Some experts end up doing almost all the work, while others barely get used. The Yuan Lab researchers studied this carefully during training. They found that expert usage follows a very clear pattern. At the beginning of training, everything is chaotic. The model starts from random initialization, so tokens bounce around

3:12 between experts unpredictably. This stage is called the transition phase. After a while, the system settles into something much more stable. Certain experts start receiving a lot of tokens over and over again, while others consistently receive very few. This stage is called the stable phase. Once the model reaches this stage, the ranking of experts becomes surprisingly predictable. A small group of experts handles most of the work, while another group barely contributes anything meaningful. That discovery led to the core innovation behind Yuan 3.0 Ultra. The researchers created a system called Layer Adaptive Expert Pruning, or LAEP. Instead of waiting until the model is finished training and then trimming it afterward, LAEP

3:57 starts removing weak experts while the training process is still running. The algorithm monitors how many tokens each expert processes. If an expert consistently handles far fewer tokens than the others, it becomes a candidate for removal. Two conditions control this process. One rule checks whether the expert's workload is far below the average workload of the layer. The second rule checks whether a group of experts together contributes very little to the total amount of token processing. If both conditions are met, that expert is removed from the model entirely. In Yuan 3.0 Ultra, the model originally contained 64 experts per layer. After pruning, the system kept no more than 48 experts per layer. That alone removed a huge amount of unused parameters. But the team noticed another problem

4:47 while training these systems. Large mixture of experts models are usually spread across many GPUs. Each GPU hosts some of the experts. If certain experts receive far more work than others, some GPUs become overloaded while others sit idle. That wastes computing power. So the researchers built a second system called expert rearrangement. This system constantly looks at how much work each expert is receiving and then redistributes experts across GPUs so the workload stays balanced. Experts that receive lots of tokens get placed on different devices from experts that receive fewer tokens. Over time, this evens out the load across the entire cluster. When these two systems work together, pruning unused experts and balancing the remaining ones across GPUs, the improvement becomes pretty dramatic. Originally, the model achieved about 62

5:37 TFLOPs of training performance per GPU. With the new system in place, that number jumped to about 92 TFLOPs per GPU. That's where the 49% improvement in training efficiency comes from. Roughly 32% of the improvement came from removing unused experts, while another 15% came from balancing workloads across GPUs. The researchers didn't stop there, though. They wanted to make sure this pruning system actually works across different models. So they ran experiments on a smaller 10 billion parameter model first. They tested different pruning thresholds and measured how the model behaved. Even when they removed a large number of experts, the model's accuracy stayed almost identical to the original system. In some cases, the prune models even performed slightly better because removing weak experts allowed the remaining ones to

6:26 specialize more effectively. They also compared this approach to the traditional way of balancing expert workloads. Other MOE models use something called auxiliary load balancing loss. This tries to force experts to receive similar workloads during training. But there's a trade-off. If you push that balancing too hard, the model's accuracy drops. The LAEP system avoids that problem entirely. Instead of forcing experts to balance their workloads, it simply removes the experts that never learn anything useful. The team then tested the idea on a 20 billion parameter model trained on 100 billion tokens. Across multiple reasoning and knowledge benchmarks including CMATH, GSM-8K, Crux Evol, Trivia QA, and DROP, the pruned models performed just as well as the original system. In some

7:15 cases, they actually achieved lower test loss, meaning the model made fewer mistakes. The researchers also experimented with scaling the number of experts. When they doubled the number of experts from 64 to 128, the pruning system still worked. Parameters dropped by about 16% while computational throughput increased. At 256 experts, the system removed about 26% of the parameters while increasing performance even further. After all these experiments, the team finally applied the full system to the main model. The final Yuan 3.0 Ultra architecture contained 103 layers, about 1 trillion total parameters, and 68.8 billion active parameters. Training ran across 824 AI chips using BF16 precision. Once the base

8:03 model was finished, the researchers moved into the post-training stage. This is where the model learns reasoning behavior and improves its responses. They used a technique called Reflection Inhibition Reward Mechanism, or RIRM. One common problem in modern AI models is something researchers call overthinking. When the model solves a reasoning problem, it sometimes produces extremely long chains of thought, even for simple questions. That wastes tokens and slows responses. The RIRM system tries to prevent that. During reinforcement learning, The model receives higher rewards when it solves problems with fewer reasoning steps. If it generates too many reflection steps, especially more than three steps, the reward drops, or even becomes a penalty. This encourages the system to give concise answers when possible while still allowing deeper reasoning for complex problems. After

8:52 training with this system, reasoning accuracy improved by about 16%, while the average response length dropped by roughly 14%. Finally, the team evaluated Yuan 3.0 Ultra across a range of real-world benchmarks. On the Dogmatics Multimodal Retrieval Benchmark, the model reached 67.4% accuracy, outperforming systems like GPT 5.2, Claude Opus 4.6, and Gemini 3.1 Pro. On the Chatrag Benchmark, which tests long context retrieval across 10 tasks, Yuan 3.0 Ultra achieved 68.2% accuracy and led 9 out of the 10 tasks. For table reasoning and enterprise data analysis, the model scored 62.6% across the 15 datasets in the

9:37 MMTAB benchmark. On the Sum of All Summarization benchmark, the model reached 62.8%, beating DeepSeq v3, GPT 5.2, Gemini 3.1 Pro, and Kimi K 2.5. The model also performed strongly on structured database tasks. On the Spyder Text to SQL benchmark, it achieved 83.9% execution accuracy. For coding and reasoning tests, Yuan 3.0 Ultra reached 93.9% 93.1% on Math 500, 91.4% on Human Evil, 82% on MBPP, 87.8% on MMLU, and 71.9% on MMLU Pro. Overall, the system shows strong performance across enterprise-style tasks like document retrieval, table understanding,

10:25 summarization, and database reasoning. That's it for this one. The full research paper and the model code are already available on Yuan Labs' GitHub if you want to Let me know what you think about this approach to building trillion-parameter AI system.