Summary

Large models have pushed AI Infra from a back-office cost-cutting tool to the center of model competition, creating what 朱亦博 sees as an industry window that opens only once every 10 or 20 years. Search engines turned Google into an Infra company because of their massive data and compute demands, and large models are repeating that process; the underlying protagonist has simply shifted from CPUs to GPUs, with compute, networking, and storage all custom-built around the model.
The economics of AI Infra are highly quantifiable, and the larger the scale, the more compelling it becomes to build an in-house team. 朱亦博’s math: renting 10,000 relatively expensive GPUs costs about $100M a month, so a 10% utilization improvement can save or generate roughly $10M monthly; at smaller scale, a general-purpose cloud baseline may be sufficient, and even 1M DAUs may not justify building a full Infra stack.
An independent AI Infra vendor positioned only between commodity hardware and commodity models will struggle to build a durable moat. A single optimization may lead for a few months, but “there is no technology that cannot be caught within a few months,” making a slide into price competition likely; the real way out is to move toward hardware or models and build stickiness through proprietary compute, first-party models, or deep co-design, because “you should not be the person stuck in the middle.”
In 朱亦博’s view, o1 and reinforcement learning shifted the industry’s primary metric from training MFU to decoding speed and cost, though consensus has yet to fully form. DeepSeek initially optimized for low inference cost, leaving its training MFU relatively low, and its base model was not necessarily ahead in the first half of 2024; after o1 introduced test-time scaling in September 2024, low-cost inference translated directly into reinforcement learning running several times faster, a key condition for it to produce R1 first. “The most important thing is always the choice of direction.”
Model competition is not a single event for algorithm teams, but a “three-legged stool” of algorithms, systems, and data. If two teams train with 5,000 GPUs for 3 months and Infra raises efficiency by 20%, the model can learn 20% more data under otherwise identical assumptions, potentially improving the final result; 朱亦博 goes further, arguing that model architecture determines operating cost, systems teams should be deeply involved in design, data teams should own outcomes, and algorithm teams should focus on training methods.
StepFun is pursuing two-way vertical integration between a visual reasoning model and Chinese-made chips. 朱亦博 says its new model should be China’s first visual reasoning model in the hundreds-of-billions-parameter range available for commercial use by third parties, capable of analyzing images directly to decompose tasks rather than converting them to text first; StepFun will offer free commercial licenses, share weights, and help adapt the model to all Chinese chipmakers, while architectural innovations push inference costs on domestic cards down to levels competitive with Nvidia solutions.
The next paradigm shift may come from unifying multimodal understanding and generation, while specialized capabilities and Agent applications still have a window of opportunity. Using roughly 2 years as a rule of thumb, 朱亦博 places the next shift after GPT-3.5 in 2022 and o1 in September 2024 perhaps in 2026; Claude has differentiated itself through investment in code data, while Agent companies and model vendors are in a relationship where “there is co-existence, but they are also hurting each other”（有共生，但是又在互相杀伤）. His long-term constant is The Bitter Lesson: “In the long run, the winner is always the method that can make the most use of computation.”

Deep dive

1. AI Infra Has Gone from Single-Machine Frameworks to Industrial-Scale Thousand-GPU Deployments

朱亦博 breaks AI Infra into hardware and 3 software layers. Hardware includes GPUs, network cards, and switches; IaaS manages servers, networking, operations, and large-scale storage across compute, networking, and storage; PaaS covers scheduling, resource management, and MaaS; training and inference framework optimization sits closer to the application layer.
In his generational breakdown, there was not even a clear term for “AI Infra” before AlphaGo; the field was more commonly called machine learning infra or ML systems. The first generation, including 贾扬清, 李沐, and 陈天奇, started from algorithmic needs and built frameworks such as Caffe to make use of GPUs; his generation handled the actual scaling, taking systems to thousands of GPUs.
His own career followed the same industrialization path: distributed-systems research at Microsoft Research, building AI Infra at ByteDance after joining in 2018, a brief stint at Google after leaving in early 2023, and then joining the founding team of StepFun in May of the same year.

2. Large Models Created a Once-in-a-Decade-or-Two Window for Infra Founders

Traditional startups typically build a business first and add high concurrency and infrastructure later, leaving Infra talent on the sidelines during the early stages. The large-model era brought massive data and compute requirements at the same time, leading 朱亦博 to call it an Infra opportunity that appears only once every 10 or 20 years to move into the center of entrepreneurship.
He uses search engines as a historical analogy: faced with internet-scale data, Google had to build world-class Infra, so “Google was actually an Infra company.” Large models have raised data and compute requirements by another one or two orders of magnitude; the difference is that the work is now done by training models on GPUs.
The objective of AI Infra has not changed from traditional internet Infra: run large-scale workloads reliably and efficiently, and combine compute, networking, and storage. “There is not much new under the sun,” but GPUs have different hardware characteristics from CPUs, changing networking and storage requirements and making engineering more customized and more aggressively optimized.

3. The Threshold for Building Infra In-House Is Ultimately a Utilization Equation

Online inference still maps onto familiar service metrics: time to first token is analogous to the time it takes to see the first screen after opening an app, while stable and fluid token generation maps to sustained service quality. Teams must optimize these experience metrics while driving per-request cost as low as possible.
Training looks more like the big-data computing of the Hadoop and Spark era: instead of using large numbers of CPUs to process data, teams now use large numbers of GPUs to learn from data and train models. The technology stack has changed, but the abstraction of large-scale computing remains.
朱亦博’s scale math is straightforward: if renting 10,000 relatively expensive GPUs costs about $100M a month, raising utilization by 10% saves or generates roughly $10M each month. “How many people are you willing to hire for $10M a month?” At that scale, covering Infra headcount is not difficult.
Small companies also depend on Infra, but do not necessarily need to develop a leading solution in-house. Teams should compare R&D spending with potential performance gains; at modest scale, a general-purpose baseline from a public cloud or MaaS may be enough, and even the 1M-DAU scenario cited by 曲凯 may not require a dedicated in-house team.

4. Third-Party Vendors Must Leave the Layer Between Models and Hardware

MaaS can become a “bazaar” aggregating APIs from multiple providers, while public clouds can host both their own and third-party models. But 朱亦博 emphasizes that promotions and price cuts explain short-term customer acquisition; the long-term question is whether a third party has irreplaceable value.
If participants are using hardware and models available to everyone, the optimization layer in the middle will struggle to sustain a meaningful lead: “There is no technology that cannot be caught within a few months.” The likely result is commoditized competition and a price war, not a technical premium that can be monetized over time.
His answer is vertical integration: move down to hardware vendors to gain advantages in compute pricing, technical support, and adaptation, or move up into deep model integration. What truly attracts users to PS5 and Steam is “exclusive games”; likewise, distinctive models or exclusive supply are the content that keeps MaaS users.
曲凯 asked about the risk of being tied to the wrong route. 朱亦博’s answer was that “you are a participant,” not someone passively placing a bet. If an Infra engineer understands models better than the hardware team and hardware better than the model team, they should directly shape product direction; if it ultimately fails, that is also the participant’s own responsibility.

5. Infra Efficiency Converts Directly into Model Capability

Every large-model team is competing in the same race: “Given a fixed amount of compute, how do you train the best model?” Cost optimization is not auxiliary to the outcome, because higher compute efficiency directly expands the amount of data a model can learn within a fixed period.
朱亦博’s example: two teams both train for 3 months on 5,000 GPUs; if better Infra raises efficiency by 20%, the model can learn 20% more data under otherwise identical assumptions, and that difference may show up in model quality after 3 months.
The numerator of the common MFU metric is the computation the hardware actually performs for the model; the denominator is theoretical compute, with a higher ratio generally indicating better utilization. But MFU is also shaped by the model, hardware, and optimization target, so it cannot be used to judge a team in isolation from its overall route.
A few exchanges between peers about “what are you optimizing now, and what numbers have you reached?” can often reveal differences in understanding. Technical execution matters, 朱亦博 says, but “the most important thing is always the choice of direction”; metrics must serve the model paradigm of the moment.

6. DeepSeek Won by Choosing Reinforcement Learning’s Metric Early

朱亦博 points out that DeepSeek’s training MFU was actually relatively low. At least in the first half of 2024, most teams were optimizing for the best base-model performance at a fixed training-compute budget, while DeepSeek prioritized the best model quality at a given inference cost, so it was not number one across every public benchmark at the time.
The turning point came in September 2024, when OpenAI released o1. Test-time scaling meant that allowing a model to think longer during inference could improve answer quality; reasoning models also rely on reinforcement learning, which requires repeatedly generating long chains of reasoning, filtering results, and assigning rewards.
DeepSeek’s earlier decision to reduce inference cost suddenly became a training advantage: much of reinforcement learning is inference work, allowing its base model to run reinforcement learning several times faster than other teams. 朱亦博 sees R1’s early arrival as “the victory that followed everyone choosing different optimization targets,” while acknowledging that luck may have played a role because o1 may not have been anticipated at the outset.
In his view, the primary metric now is decoding—the speed and cost of the model’s output stage—not training MFU. But industry consensus is still incomplete. Decoding determines online business cost on one side and the speed of earning rewards in reinforcement learning on the other; continuing to focus only on MFU may indicate that one’s understanding is stuck in the previous phase.

7. The Power Structure Among Algorithms, Systems, and Data Sets the Model Ceiling

朱亦博 calls model development a “three-legged stool” of algorithms, systems, and data. Every model-architecture choice involves a trade-off: a small algorithmic gain may cost half of system performance, and sometimes the reverse is true. Decisions must be made jointly rather than having the algorithm team design the model and throw the optimization task over the wall to Infra.
He proposes a more demanding division of labor: model quality is determined to a large extent by data, while efficiency and cost are determined by systems. Because model architecture has the greatest impact on cost, “model architecture should actually be designed by systems people”; the data team should own parameter count and the ceiling for model quality, while the algorithm team’s core responsibility should be the training method.
曲凯 argues that the outside world usually treats algorithms as the absolute center. 朱亦博 does not deny their importance, but says organizational structure can overwhelm individual initiative: if systems and algorithm teams report to a leader who understands only algorithms, Infra can easily degrade into a support function and lose the power to influence the model in return.
A small team can make the trade-offs collectively as a single team, while large companies may create a mismatch by preserving the finely divided roles of the mobile-internet era. Product managers also need a deeper understanding of technology and data; even if an old organization retains the same people, it may no longer fit the new AI production model.

8. Model Architecture and MoE Reveal Infra’s Real Position

朱亦博 mentions an unnamed company whose model appears relatively small and whose algorithmic performance can “punch above its weight” against larger models, but whose architecture makes its actual hardware operating cost higher. Algorithm teams draw a “model size—quality” chart, while what the business really needs is an “actual operating cost—quality” chart.
The timing of MoE adoption is also an organizational litmus test: “The earlier you do it, the greater the Infra team’s influence.” In algorithm researchers’ eyes, at the same 100B scale, Dense models are generally stronger than MoE models; MoE’s first function is to reduce active parameters and inference cost, not directly raise the capability ceiling of a model of the same size.
In early 2022, after the team at his former employer ByteDance reproduced GPT-3, the business recognized the model’s capability but objected to its cost. 朱亦博’s Infra team therefore led the study of MoE. An older algorithm colleague in charge at the time even thought “this had nothing to do with him”; DeepSeek’s decision to use MoE from the start is likewise evidence, in his view, that systems teams can influence architecture.
StepFun began training its second-generation model in December 2023, a period 朱亦博 places in the context of its earlier MoE research. Algorithm researchers now understand the importance of active parameter count and cost, but this history shows that many choices later viewed as capability strategies originally emerged from Infra’s constraints around commercial usability.

9. StepFun Shifted from the Lesson of Being “Bigger Than Llama” to Efficiency and Multimodality

朱亦博 admits that StepFun was “overconfident” about its compute and execution capabilities early on, building a model larger than Llama and starting a year earlier than DeepSeek. The team did complete the training, but the huge model brought additional problems; in hindsight, it was still a mistake that needed to be acknowledged.
His research mindset is that every research bet can be wrong, and what matters is continuous correction: “You won this round; I’ll take it back next round.” That also explains StepFun’s subsequent focus on a more efficient language-base architecture rather than continuing to chase scale alone.
Multimodality remains StepFun’s distinctive strength. 朱亦博 says it is “firmly in China’s top 2” in some multimodal areas. Priorities include fusing language and vision, coordinating edge hardware with deployment methods, and tackling the complex engineering of reinforcement-learning inference; in his view, the language-base architecture is at least a 50-50 problem between systems and algorithms.

10. The Truly Revolutionary Barrier Is Joint Model-Chip Design

Export controls, patents, and reinforcement-learning engineering are difficult, but 朱亦博 still classifies them as problems where “you see a path, then execute it to completion.” More difficult are revolutionary changes with no clear path, such as genuinely co-designing models and hardware.
Today’s models are largely optimized around Nvidia GPUs. Google’s TPU has distinctive features, but its underlying compute paradigm remains close to the GPU’s. Even if H20 chips can enter China, they are still relatively weak cards; Chinese chips face more than the question of whether models can run at all, because their cost-performance ratio when running existing open-source models often lags.
His envisioned breakthrough is a new chip with materially different characteristics, followed by a team designing a model around those characteristics that is a full tier above everyone else. “The Turing Award is right in front of you,” and a commercial company could build an unassailable moat from it, though he explicitly acknowledges that this path can only be navigated one step at a time.
The gap between China and overseas markets first comes down to scale: the problems faced by 10,000 GPUs differ from those faced by 100,000. The second factor is the ability to influence upstream and downstream players. OpenAI and Google have the capital and talent to reach into both chips and models; only a few large Chinese companies have comparable ingredients, and whether they can organize them into an effective integrated system remains an open question.

11. Google’s Vertical Integration Is a Risk, Not a Requirement for Every Company

Google owns both its chips and models, potentially giving it a meaningful training-efficiency advantage. 朱亦博 says OpenAI has always been most wary of Google precisely because Google is better positioned to optimize the entire stack from hardware to models.
曲凯 then asked whether all model companies would ultimately have to build their own chips. 朱亦博 did not call it inevitable, but emphasized a “dangerous possibility”: if one company truly integrates chips, systems, and models to the highest level, competitors will face significant pressure.
Over a long enough horizon, he expects vertical integrators similar to Google to emerge. Companies that do not follow that route will still have opportunities, but they must differentiate themselves elsewhere rather than assuming the layer between commodity hardware and models is naturally safe.

12. Third Parties Are Better Suited to Inference, While Open Source Amplifies Nvidia’s Advantage

In 朱亦博’s classification, Snowflake and Databricks are not typical AI Infra companies, but Infra “for serving data.” Snowflake manages multi-cloud data through a unified abstraction and layers on processing services; Databricks grew out of Spark, then helped traditional enterprises store, manage, connect, and use data before naturally extending into AI.
More typical third-party participants include CoreWeave, CloudWalk, and China’s Wuwen Xinqiong, Luchen Technology, and SiliconFlow. 朱亦博 is cautious about commenting on each company’s current position, but his broad observation is that startups are gradually shifting toward inference. Training is mainly done by highly specialized experts themselves, making a third-party training business model difficult, while the training method is itself a model company’s core competitive advantage and is hard to hand to a third party early on.
Popular open-source models attract large numbers of teams studying how to run them efficiently, objectively advancing Infra. The other side is that resources may become locked into a single model, with part of the accumulated optimization rendered obsolete when a new architecture arrives. Open source also reinforces incumbent hardware: DeepSeek, Qwen, and others are optimized for Nvidia, so all-in-one machines using Nvidia cards often have better cost performance than those using Chinese cards.
朱亦博 is not satisfied with attributing this outcome simply to the practical constraints of domestic substitution. He asks whether teams can “design models for Chinese chips,” bringing them to DeepSeek-level performance while achieving extremely high efficiency. The core idea remains to adapt the model to the hardware, rather than forcing hardware to passively support models built around Nvidia’s paradigm.

13. Visual Reasoning Is the Point of Coexistence Between StepFun and Chinese Chips

朱亦博 says StepFun’s new model should be China’s first visual reasoning model in the hundreds-of-billions-parameter range available for commercial use by third parties. It does not merely convert an image into text and then call a language model; it reasons directly from visual information, while video can also be processed through frame sampling.
He uses a robot retrieving an object from a cabinet to explain the value of end-to-end reasoning: the target may be partly blocked by other objects, and text cannot fully describe every object and its spatial relationship. The model can directly inspect the image and decide, “Move this thing first, then move that thing,” decomposing a complex task.
Higher-level capabilities are not yet fully solved, such as navigating a maze in visual space while “drawing a draft.” Solving a math problem through text reasoning can be viewed as navigating a maze in language space, but visual relationships are difficult to describe accurately in text, leaving substantial room for native visual reasoning.
StepFun will provide free commercial licenses to all Chinese chipmakers, share weights, and help with adaptation wherever possible. Architectural innovations also enable low-cost inference on domestic cards. 朱亦博 hopes chipmakers can gain distinctive products and commercial competitiveness, while StepFun can expand its model’s reach, creating “a symbiotic relationship.”

14. The Next Paradigm Still Depends on Who Can Unlock the Most Compute

朱亦博 sees GPT-3.5 in 2022 as the previous major paradigm and o1 in September 2024 as the current one; if major shifts occur roughly every 2 years, the next may arrive in 2026. He favors unifying multimodal understanding and generation, with success defined as a single model outperforming specialized models at both tasks, just as GPT-3.5 displaced many language-only models.
In his view, Veo 3 is closer to a previous-generation specialized generation model combined with engineering and product integration, including features such as music, rather than a new paradigm unifying understanding and generation. Multimodal understanding is no longer especially expensive, while generation—video in particular—remains costly; he cannot say whether costs will fall to one-tenth in a year, but believes they “definitely” can fall to a fraction of today’s level.
Specialized capabilities still offer temporary differentiation. He believes Anthropic, even without especially strong reinforcement learning, has made Claude’s coding ability “stand alone”（一枝独秀）through massive investment in constructing and cleaning code data. The risk is that a new training paradigm could quickly erase that lead. Agent companies face a similar dynamic: workflows first make models “stand on tiptoe,” but once successful, the capability may be natively absorbed by the next-generation model, creating an ecosystem where “there is co-existence, but they are also hurting each other”（有共生，但是又在互相杀伤）.
His advice to practitioners is to “move closer to models and closer to hardware” and participate directly in vertical co-design. 朱亦博 most strongly endorses The Bitter Lesson: “In the long run, the winner is always the method that can make the most use of computation.” GPUs trade flexibility for parallel compute and roughly doubling performance every 2 years, ultimately opening a 10x or 100x gap over CPUs; Infra’s fundamental job is to keep capturing that compute curve.

The AI Revolution in Education with Shawn Jansepar, Director of Engineering at Khan Academy Trump AI Speech & Action Plan, DC Summit Recap, Hot GDP Print, Trade Deals, Altman Warns No Privacy