A Sentence-by-Sentence Guide to DeepSeek-R1, Kimi K1.5 & OpenAI o1
Summary
DeepSeek-R1 did not actually rewrite who has achieved AGI; it put the reasoning-model playbook face-up: OpenAI posed a puzzle with o1, and DeepSeek was the first to publish a sufficiently elegant answer, showing the industry that post-training compute and test-time compute could become new growth axes beyond pretraining. 潘佳怡’s judgment is measured: DeepSeek has forced U.S. tech giants to reassess the size of their lead and made the U.S. take Chinese AI teams’ innovative capacity seriously, but it has not yet shown that it can repeatedly create paradigm shifts the way OpenAI has.
R1-Zero’s most important finding is that a sufficiently strong base model may develop search, reflection, and error correction on its own when exposed directly to large-scale reinforcement learning without SFT: when the model suddenly says, “Wait, wait…there’s an aha moment here,” that is not a script written in advance by researchers. The core paradigm is “Don’t Teach, Incentivize”: when a problem is too complex for humans to demonstrate step by step, define only what is right and wrong, then let the model find the “divine move” humans never thought of.
The economic point is that post-training remains a rounding error beside pretraining costs. Using roughly 10,000 steps, about 1,000 responses per step, and up to roughly 10,000 tokens per response, 潘佳怡 estimates that R1 generated around 100 billion tokens during training; at its API price of $2.2 per million tokens, that would cost only about $220,000, or perhaps $100,000–$1 million after optimization, versus roughly $5.3 million–$6 million for V3 pretraining. 张小珺’s key correction is that R1 cannot be priced independently of V3: low incremental cost does not mean expensive foundation models are no longer necessary.
R1’s algorithmic elegance comes from subtraction: GRPO removes the value model from PPO, which is difficult to estimate accurately and expensive to run, and instead makes only within-group relative comparisons—“learn more from the good, do less of the bad”—while using a KL constraint to keep the model near its initial policy. Mathematics and code rely as much as possible on verifiable ground-truth rewards, avoiding reward-model exploitation; “the most elegant algorithm is the cleanest” is here both an aesthetic judgment and a judgment about cost and scalability.
A production-ready R1 is not the same thing as R1-Zero: it adds a small amount of cold-start long chain-of-thought data, reasoning RL, roughly 600,000 reasoning samples and 200,000 general-purpose samples for distillation, followed by full-domain RL, trading some scientific purity for readability, language stability, and general capability. Large-model reasoning traces can also be distilled into 1.5B–70B models; but “a dumb student cannot merely copy the answer,” as an experiment cited on the show illustrates: the original score was 43.8, distillation lifted it to 49, and RL raised it further to 52.
Kimi k1.5 and R1 follow strikingly similar technical paths; Kimi simply lost the attention war to R1-Zero’s “aha moment” and DeepSeek’s accumulated open-source culture. Its report is arguably richer in engineering detail. The most notable result is its chain-of-thought reward model: a conventional classification-style reward model achieved roughly 84.4% accuracy, while having the model reason before grading lifted accuracy to 98.5%; length penalties, curriculum sampling, partial rollouts, and long-to-short RL also offer more direct levers for optimizing reasoning cost, latency, and throughput.
One release did not reset the competitive landscape: DeepSeek showed that limited resources can still be converted into innovation efficiently, while open source lowers replication costs across the industry and compresses the pricing power of commoditized APIs. OpenAI still has, by the show’s estimate, hundreds of thousands of H100s, future resources on the scale of millions of B100/B200-class GPUs, and a research organization that has not visibly broken down. The scarce inputs in the next phase will be compute, hard problems, expert annotation, agent environments, and reliable rewards—not simply more user data.
Deep dive
1. R1 made the route public, but declared no winner in the AGI race
张小珺 summarized the shock of the Spring Festival period as DeepSeek “rewriting the global AGI narrative.” 潘佳怡 first drew a boundary: she mainly studies post-training and is reluctant to make sweeping judgments about the macro narrative, but several things are “definitely true.”
The first is that U.S. tech giants will reassess their training efficiency and innovation lead. The second is that the U.S. has begun taking Chinese AI teams seriously, rather than viewing them simply as imitators one or two years behind.
The more direct technical significance is that the route hidden by o1’s release finally became visible: reinforcement learning can scale reasoning ability, and R1-Zero and R1 may push both model reasoning and agent capabilities into a period of rapid iteration.
2. OpenAI posed a puzzle; DeepSeek was the first to submit an elegant answer
潘佳怡 does not reduce R1 to an o1 replica. Her analogy is that OpenAI posed the industry “a very interesting riddle” with o1, while DeepSeek was the first team to reveal the answer, and the answer was concise and elegant enough to stand on its own.
DeepSeek is not merely a follower. She specifically pointed to V3’s super-sparse MoE and innovations in self-attention, arguing that these already qualify as outstanding original work.
But she reserved a higher bar: DeepSeek has not yet repeatedly delivered paradigm-changing products like ChatGPT, Sora, and o1, as OpenAI has. Over the past few years, that ability to break through continuously has been demonstrated by almost no one else.
3. R1 set the industry alight because it turned a black box into a roadmap
Researchers who were already working on reasoning had started exploring after o1 was released. What pulled many more people into the field was R1, because the technical route went from black box to open book, finally showing researchers how to enter the space.
潘佳怡 observed that friends working on vision-language-model pretraining were already considering a shift toward visual reasoning. She compared the migration to the wave of researchers who entered RLHF after ChatGPT first appeared.
Researchers around her from Google DeepMind, OpenAI, and other institutions were “extremely surprised” that DeepSeek had produced this work at such low cost. The volume of discussion triggered by R1 was at least comparable to that surrounding o1’s release.
4. Kimi k1.5 lost attention, not the technical race
潘佳怡 considers Kimi k1.5 “a very elegant piece of work,” technically doing much the same thing as R1. Its report even contains more data, systems information, and engineering detail; read together, the two reports provide the most complete picture.
Kimi’s first problem is accumulated credibility. It had mainly served the domestic market and lacked DeepSeek’s tradition of sustained open sourcing and long-term technical publication, so overseas research communities never formed the same organic distribution network around it.
The second problem is narrative. R1-Zero has an “aha moment” capable of carrying an entire paper; Kimi k1.5 reads more like a rigorous engineering manual—information-dense, but lacking a hook that non-specialists can immediately repeat.
5. Open source creates culture before it lowers costs
潘佳怡 referred to a similar idea in an earlier interview with 梁文峰: “Open source creates a culture and a sense of identity.” By repeatedly publishing models and high-quality technical reports, DeepSeek allowed researchers to learn incrementally from its progress and built long-term credibility.
Open source reduces duplicated dead ends. If a company publishes its framework and code, later teams can reuse them directly instead of rediscovering the same technical and implementation details from scratch.
It also pushes technology into the mainstream faster and keeps deployment prices under pressure. R1’s reasoning architecture is already cheap, while community self-hosting and inference-framework optimization can drive costs down further.
6. Open source compresses the API moat without necessarily destroying subscriptions
张小珺 asked, “If everything is open source, how do foundation-model companies make money?” 潘佳怡 admitted there is no mature answer, but relayed a view held by some OpenAI researchers: ChatGPT subscriptions are better than selling APIs one request at a time.
APIs are already highly commoditized. Many services are compatible with the OpenAI interface, and switching providers may require only a few lines of code and a new payment method. Vendors must continuously offer low prices and leading performance, making substitution easy.
ChatGPT’s $20 monthly subscription bundles o1, 4o, Sora, and other frontier capabilities into a continuing product relationship, creating a stronger moat. She also relayed Sam Altman’s comments from the o3-mini AMA: OpenAI had been on “the wrong side of history” on open source and might eventually release some older models that were no longer used to make money.
7. Pretraining is still advancing, but iteration time and capital intensity are worsening
GPT-3 was released in June 2020, and GPT-4 arrived around March 2023. By the time of the program, in February 2025, GPT-5 had still not appeared after nearly 2 years. Meanwhile, foundation models such as DeepSeek-V3 and Llama 405B had not clearly surpassed GPT-4.
Most of the obvious experience gains in recent years came from post-training: tool use, value alignment, and user-preference optimization all improved substantially, but foundation-model generation jumps are slowing.
The industry estimate relayed by 潘佳怡 put GPT-4 pretraining at roughly $50 million. Algorithmic efficiency allowed DeepSeek-V3 to reach comparable capability for roughly $5.3 million–$6 million, while the next generation of pretraining could exceed $200 million. Even if pretraining can produce another generation, cost and cycle time have become hard constraints.
8. o1 turned both training compute and inference compute into growth axes
o1’s public charts separately examined post-training compute and test-time compute, using logarithmic scales. The vertical axis measured accuracy on high-school-olympiad-level tasks such as AIME. As compute increased, performance rose from roughly 20%–30% to nearly 80%.
The key point is not merely that training longer produces better results. More compute can also be spent during inference: the model generates a longer chain of thought, decomposes the problem, tries alternatives, checks its work, and then answers.
潘佳怡 described this as a new scaling path: beyond pretraining compute, post-training and test-time compute can also scale, with early curves rising approximately steadily with the logarithm of compute.
9. Reinforcement learning no longer teaches answers; it defines what is worth pursuing
The basic action in pretraining and SFT is imitation: provide the model with a “correct answer” and teach it what to output next. Reinforcement learning tells the model only whether the result is good, leaving it to discover more effective behavior.
The central analogy 潘佳怡 kept was: in the past, people said, “Give a man a fish and you feed him for a day; teach him to fish and you feed him for a lifetime.” Now the approach is to show him “how delicious fish is, then leave him hungry,” and he may learn to fish on his own—and do it better than the teacher.
The reason is straightforward. When ideal behavior is too complex for humans to turn into a step-by-step textbook, demonstrations constrain the search space. Verifiable rewards specify only the destination, leaving room for the model to find paths humans never considered.
10. Long chain-of-thought turns a chatbot’s fast response into slow thinking
A traditional chatbot is closer to “fast thinking”: it reads a question and starts answering. A reasoning model first plans the task, tries approach A, switches to C or D after finding an error, and then checks the conclusion.
o1’s public description specifically emphasized that the model can identify and correct errors on its own, break difficult steps into simpler ones, and choose another route when the current one fails. These behaviors were not taught line by line.
张小珺 sees pretraining as expanding the knowledge base, while reinforcement learning determines how to use it for a particular task. 潘佳怡 broadly agreed: the former stores world knowledge and reasoning templates; the latter organizes them into behavior suited to chat, reasoning, or agents.
11. The AGI route went from betting everything on pretraining to running on two legs
潘佳怡 offered an admittedly imprecise but intuitive analogy: model capability is roughly pretraining compute × test-time compute. Historically, most capital went into the former, while inference was extremely cheap, making the marginal cost of scaling pretraining increasingly painful.
She relayed one OpenAI researcher’s line: “o1 is sort of a GPT-6 in your hand.” This does not mean o1 is literally GPT-6. It means more computation at inference time may deliver capabilities earlier that would otherwise require a much larger pretrained model.
张小珺 asked whether the O-series could reach AGI directly. 潘佳怡 believes companies will not choose one route over the other. As Mark Chen has said publicly, pretraining and post-training for reasoning will advance simultaneously—“two legs running.”
12. o3 pushed reasoning models further toward actionable agents
潘佳怡 used the roughly 3- to 4-month iteration from o1 to o3 to show that post-training remains a high-return area. Her examples: ARC-AGI rose from roughly 20%–30% on o1 to about 70%–80% on o3, while FrontierMath climbed from roughly 1% to more than 20%.
o3-mini delivered higher performance while cutting the price to roughly one-third of o1-mini. She was unsure whether DeepSeek directly forced the move, but considered it “very likely” that competition played a role.
The next step is not merely answering one question but using tools across multiple rounds: think one step, operate the environment, receive feedback, and decide what to do next. OpenAI’s recently launched Deep Research is, in her view, a continuation of this direction.
13. An agent’s essence is letting a large model orchestrate the outside world
A coding agent can receive an instruction to fix a bug or implement a feature, then read the codebase, edit files, run terminal commands, execute Git operations, and consult GitHub or the broader web.
A Computer Use Agent directly takes over the keyboard and mouse to shop, create PowerPoint decks, and perform other tasks. It no longer merely generates text; it connects the language model’s intelligence to real-world tools.
张小珺 compressed the idea into one sentence: “A large language model orchestrates the outside world.” 潘佳怡 considers the description accurate and says it also clarifies the boundary between reasoning models and agents: the former solve problems internally, while the latter must continually interact with an environment.
14. “Incentivizing” is the most important verb in the entire R1 paper
The title of the DeepSeek-R1 paper is “Incentivizing Reasoning Capability in LLMs via Reinforcement Learning,” not “teaching reasoning.” 潘佳怡 connected it to the title of an MIT talk by OpenAI researcher Hongyu: “Don’t Teach, Incentivize.”
The phrase does not reject all human knowledge. It describes how behavior is acquired: the model has already been pretrained on internet text, so researchers no longer prescribe every reasoning move; they provide the task and the reward.
She explains the convergence among the 3 companies’ approaches this way: it is less a single company’s invention than a discovery—a sufficiently strong language model exposed to reinforcement learning will develop long chain-of-thought reasoning and improve substantially on reasoning tasks.
15. R1-Zero sent the base model straight into reinforcement learning
DeepSeek-R1-Zero starts from the V3 base model and runs large-scale reinforcement learning without SFT. That differs from the traditional 3-stage sequence of pretraining, supervised fine-tuning, and RLHF.
After thousands of training steps, the model steadily improved from a low baseline on mathematics and related tasks, eventually approaching o1-level performance. At around 10,000 steps, it broadly entered o1’s range while remaining somewhat lower, and naturally developed search, reflection, and self-correction.
The costs were equally clear: poor readability, mixed languages, and weak generality. R1-Zero is therefore closer to a scientific discovery; the model actually deployed to users is R1, after multiple rounds of engineering work.
16. GRPO removes the value model that is hardest to estimate in long chains
PPO typically trains a policy model and a value model together. The latter tries to estimate the value of each token or step, distributing the final reward across the entire trajectory.
Long chain-of-thought makes that judgment ambiguous. If a model first writes “one plus one equals three,” then catches the mistake and changes it to two, is the earlier “three” a bad action—or the exploration necessary for successful self-correction? Precise labeling is difficult.
DeepSeek proposed GRPO in earlier DeepSeekMath work: if the critic is inaccurate and expensive, discard it and perform policy gradients using only the relative outcomes of a group of responses. 潘佳怡 called the method “very simple and direct.”
17. Within-group comparison, data reuse, and KL constraints form a minimal viable algorithm
GRPO has the model attempt the same problem multiple times and then standardizes the results: increase the probability of relatively better trajectories and reduce that of relatively worse ones. The within-group mean is approximately zero and the standard deviation approximately one, stabilizing training.
To make expensive long outputs more useful, training tolerates some off-policy data: samples generated by slightly earlier versions of the model can be reused, with importance-sampling-style techniques correcting the bias.
KL divergence keeps the current model near a reference model. The new policy can evolve, but cannot suddenly move far away from a starting point that at least produces coherent outputs, reducing the risk of training collapse.
18. R1-Zero rewards only the answer and the format, deliberately ignoring reasoning style
The reward function has 2 main components: accuracy reward checks whether the final answer is correct, while format reward checks whether the model reasons first inside a
thinktag and answers afterward inside ananswertag.The format is not merely cosmetic. It lets programs reliably extract the final answer and compare it with the ground truth. When 潘佳怡’s team reproduced TinyZero, assigning 1 point for correctness and 0.1 for format was already enough to produce results.
She and her collaborators completed the early small-scale experiment in a single evening. On small, simple tasks, even a 3B model can learn search and correction as long as task difficulty matches model capability.
19. The value of verifiable rewards is that they leave the model little room to game the rules
R1-Zero does not use neural outcome or process reward models. In mathematics, code, and similar domains, it relies as much as possible on answer keys, simplification rules, and unit tests.
潘佳怡 used emojis to illustrate reward hacking in RLHF. If a reward model mistakenly learns that more emojis mean greater friendliness, reinforcement learning may produce screens full of emojis that score highly for the machine but leave humans speechless.
Ground-truth rewards cover narrower domains, but strong models cannot easily deceive them. When 张小珺 asked whether OpenAI’s early paper “Let’s Verify Step by Step” had deliberately sent the field down the wrong path, 潘佳怡 said there was no need for a conspiracy theory; OpenAI itself may still have been exploring the route at the time.
20. R1-Zero’s performance gains are fundamentally the model learning to spend more inference compute
In DeepSeek’s chart, AIME accuracy rises steadily with reinforcement-learning steps. A single response eventually reaches roughly 70%, while majority voting over 16 responses produces an even higher result, approaching o1’s range.
Another chart explains where the performance comes from: the average response grows from a few hundred tokens initially to nearly 10,000 tokens after around 8,000 training steps, following an almost straight upward line.
The model discovers that “thinking a little longer” earns more reward and therefore increases test-time compute on its own. 潘佳怡 leaves an open question: the curve is still rising—why did DeepSeek stop there, and would more training continue to expand performance?
21. The “aha moment” shows the model is not merely stretching an existing template
In one trajectory DeepSeek displayed, the model suddenly says midway through its derivation: “Wait, wait…there’s an aha moment I can flag here,” then rechecks the formula, finds the problem, and corrects the answer.
The important point is not the anthropomorphic line itself but the behavioral chain behind it: pause, review, locate the error, change paths, and verify again. Researchers did not script “reflect here”; the behavior emerged under reward pressure.
潘佳怡 called reinforcement learning possibly the only known method currently capable of reaching superhuman performance. She placed the phenomenon alongside the “divine move” in Go, arguing that it strengthens the case for continuing to scale data, compute, and algorithms.
22. Emergent behavior makes the question of consciousness serious, but offers no answer
张小珺 asked whether models might develop consciousness. 潘佳怡’s answer was that it depends on how consciousness is defined. If one insists that a model is merely predicting the next word, one can of course say it is not conscious; if one compares only inputs and outputs, the boundary is less clear.
She relayed Geoffrey Hinton’s position: a model can receive a sentence like a human and produce an equally good or better response. Is a difference in underlying implementation alone enough to conclude that it has no consciousness?
She did not turn the discussion into a conclusion, saying only that models have already surpassed humans in many domains while still needing breakthroughs in others. The fact that people are beginning to debate consciousness seriously, she believes, shows how extraordinary the models’ capabilities have become and how they are forcing people to reflect on their own existence and value.
23. R1 packaged R1-Zero’s scientific discovery as a deployable product
潘佳怡 draws a clear distinction: R1-Zero is technically cleaner because it relies less on human priors, while R1 is stronger commercially and operationally because it is readable, general, and linguistically stable.
The R1 process is roughly: a small amount of cold-start long chain-of-thought data, reasoning RL, SFT on data generated by multiple expert models, and finally one round of full-domain RL.
When 张小珺 asked which was more important, 潘佳怡 said it depends on the objective: study R1-Zero to understand the ceiling of future research; use R1 for real users. DeepSeek’s current products use R1, and almost no one serves users directly with R1-Zero.
24. Cold-start data trades a small number of demonstrations for a better search starting point
Cold-start data consists of carefully selected long chains of thought that demonstrate search, verification, reflection, and self-correction patterns while improving the mixed-language output and poor readability of R1-Zero.
This still introduces human priors, but it may make later exploration far more efficient. The model does not need to spend a large number of RL steps discovering the most basic reasoning behaviors; it can optimize from a better initialization.
The subsequent reinforcement learning focuses mainly on mathematics, code, science, and logic—domains with verifiable outcomes. If a response mixes languages, the reward is reduced, pushing the model toward more stable use of the target language.
25. 800,000 expert samples turn a reasoning specialist back into a generalist
The model produced during the reasoning phase is a specialist: excellent at mathematics and code, but not necessarily good at chat, writing, translation, or role-play. DeepSeek therefore used an R1 intermediate checkpoint to generate reasoning-task data, then applied rejection sampling to remove incorrect, mixed-language, overly long, and poorly formatted answers.
This produced roughly 600,000 reasoning samples. For general-purpose tasks, DeepSeek-V3 acted as the expert: it answered simple questions directly and generated chains of thought for harder ones, producing roughly 200,000 samples.
The 2 sets totaled about 800,000 samples and were used to fine-tune the V3 base model, allowing it to absorb both reasoning and general-purpose expertise. Full-domain RL followed: rule-based rewards for reasoning tasks and a reward model for general tasks, producing the final R1.
26. “Zero” inherits AlphaGo Zero’s ideal, but it did not truly start from zero
AlphaGo first learned from a large corpus of human game records and then used reinforcement learning. AlphaGo Zero used only rules and rewards to self-play from scratch, eventually reaching a higher ceiling and discovering moves humans had never considered.
R1-Zero is similar in that it does not use cold-start chains of thought to teach reasoning patterns. But it was already pretrained on internet text, so 潘佳怡 emphasized: “It isn’t quite that zero yet.”
R1 also did not necessarily need R1-Zero to exist first as an engineering step. Kimi k1.5 did not publicly disclose an equivalent Zero stage and still achieved strong performance with cold start plus RL. R1-Zero’s unique value is proving that a route with fewer human priors can work.
27. Large models discover reasoning; small models can carry it away through distillation
DeepSeek distilled data generated by R1 into Qwen and Llama models, publishing models ranging from 1.5B, 7B, 8B, 14B, and 32B to 70B, allowing developers to run reasoning models locally.
According to the rankings cited on the show, the distilled Qwen-1.5B from R1 even scored roughly 10–20 percentage points above GPT-4o, Claude Sonnet, and other traditional frontier chat models on AIME. Long chain-of-thought does not have to live only in giant models.
This broadens the competition from “who can train the largest base model” to “who can provide the best teacher data, distillation method, and inference deployment.” A small model does not replicate all of its teacher’s capabilities, but it can inherit a substantial share of its behavior patterns.
28. Distilling a strong teacher beats having a weak student explore from scratch—but the student still needs practice
A comparison using Qwen-32B showed that distilling from an R1 teacher built on a 600B-class MoE was clearly better than sending a 32B model directly through the R1-Zero-style large-scale RL route. A larger model is more likely to discover complex and effective strategies.
Qwen-32B-Zero trained through direct RL performed similarly to QwQ-32B-Preview. 潘佳怡 believes this may indicate that different teams’ routes eventually converge toward similar performance, though the paper does not reveal QwQ’s internal pipeline.
Another experiment cited on the show scored the original model at 43.8 on average, lifted it to about 49 after distillation, and then to 52 after another round of RL. Her summary was vivid: “A dumb student cannot merely copy the answer.” It must use reinforcement learning to discover which of the teacher’s tricks actually fit its own capabilities.
29. Process reward models are not suited to directly steering large-scale long-chain training
PRMs attempt to judge whether the first step, second step, and so on are correct. But in natural language, even defining a “step” is difficult: is one line of long division one step, or the entire calculation? General-purpose tasks have no consistent boundary at all.
Self-correction makes labels even less determinate. An intermediate step may look wrong but trigger a later correction; human step-by-step annotation is expensive and hard to scale, while automatic labeling is not reliable enough.
DeepSeek therefore did not use PRMs for large-scale RL. 潘佳怡 still sees value in best-of-N selection: have the model generate 10 or 100 responses, then use a PRM to select the best one. That may still outperform a scorer that looks only at the final result.
30. MCTS is elegant in Go but runs into an enormous tree in language
Monte Carlo tree search decomposes a task into nodes, tries multiple branches at each step, and uses value estimation to decide which paths deserve further investment. AlphaGo’s success led many teams to believe the method could transfer to language reasoning.
The first obstacle is that language has no clear equivalent of a move. A step might be one token, one sentence, or one paragraph. If the tree expands by token, a sequence can reach tens of thousands of levels, with a huge vocabulary at each level; the search space and pruning cost explode.
Pruning also depends on an accurate value function, which is precisely the hardest component to train in long chain-of-thought. DeepSeek experimented with PRMs and MCTS in areas such as mathematical proofs, but ultimately found direct RL cleaner and more scalable.
31. R1’s incremental training may cost only 6 figures, but that is not the full bill
潘佳怡 performed a back-of-the-envelope calculation: roughly 10,000 steps × 1,000 responses per step × up to 10,000 tokens per response, for a total of about 100 billion tokens.
At R1’s then-current API price of $2.2 per million tokens, purchasing all of that would cost roughly $220,000. After accounting for API margins and additional training costs, she estimated that a highly optimized run might cost around $100,000—“at the absolute outside,” no more than $1 million.
张小珺 immediately supplied the missing premise: R1 depends on V3 and cannot be costed as post-training alone. 潘佳怡 agreed; V3 pretraining cost roughly $5.3 million–$6 million. R1 is cheap as an incremental cost on top of a strong base model.
The paper does not disclose the GPU count or full hyperparameters. 潘佳怡 estimates that training R1 on 100–200 GPUs would not be surprising, nor would training the 600B-class MoE V3 on several hundred GPUs. She explicitly leaves these figures at the level of estimates.
32. Vertical companies can start from V3; AGI labs still need the full stack
If the goal is to train a reasoning model, code agent, or specialized model for a particular industry, 潘佳怡 believes post-training directly on V3 may be enough. With the same resources, post-training generally offers better short-term returns than repeating pretraining.
If the goal is artificial general intelligence, the company needs control of pretraining, post-training, inference, and agent environments. It cannot depend indefinitely on another company’s open-source release or give up validating its own architectural techniques.
She also cooled down the claim that SFT is unnecessary. R1-Zero proves that SFT can be skipped, but both R1 and Kimi k1.5 used small amounts of carefully designed cold-start data. The shift is from large-scale demonstration to small amounts of precise behavioral guidance.
33. Kimi k1.5 follows the same mainline while publishing more replication detail
k1.5 also starts with cold-start long chain-of-thought data and then applies reinforcement learning, making it closer to R1 than R1-Zero. It deliberately covers 4 modes: planning, self-verification, reflection, and exploration.
It does not use GRPO, instead adopting a variant of online policy mirror descent. 潘佳怡 believes its objective and gradient form are close to GRPO’s, but the report does not explain in enough detail whether the difference matters.
Kimi’s advantage is engineering transparency: data construction, length control, reward models, training architecture, and ablation experiments are discussed in greater detail. Its weakness is the absence of R1-Zero’s immediately visible scientific climax.
34. Length penalties and curriculum learning turn “thinking more” into a manageable resource
k1.5 adds a length penalty to the reward: when an answer is wrong, the model should stop quickly; when it is correct, shorter answers are still preferred, but the correctness reward must dominate. The goal is not to ban long chains of thought but to make the model get the answer right with fewer tokens.
张小珺 offered a user-side counterexample: DeepSeek’s outputs are longer, but they also provide more emotional value, and users may not want less. 潘佳怡 accepted the trade-off—direct chat may benefit from richness, while APIs and embedded services care more about latency and cost.
Curriculum sampling no longer selects questions randomly from a pool. It continually chooses problems the model has a chance of solving but has not yet mastered. Experiments show that capability-matched questions make the model learn faster and ultimately perform better.
35. A chain-of-thought reward model cuts the error rate from roughly 15% to 1.5%
Kimi cannot rely entirely on rules for mathematics, especially proof problems, so it still trains a reward model. Its input includes the problem, the model’s answer, and a standard reference answer.
The conventional approach feeds the language-model activation into a small classification head and asks it to judge correctness directly, achieving about 84.4% accuracy. The problem is that language models are naturally good at generating language but are forced to emit a label in one shot.
The new approach has the reward model generate a chain of thought first, compare the solution with the reference answer, check for errors, and then issue a verdict. According to the show, fine-tuning on roughly 80 annotated chains of thought lifted accuracy to 98.5%, cutting the error rate from about 15% to 1.5%. 潘佳怡 considers this k1.5’s most likely contribution to see broad industry adoption.
36. Multimodal reasoning also depends on verifiable and synthetic data
Code tasks use the problem, an algorithmic answer, and test cases. An automatic generator creates small programs and unit tests; running the tests provides clear feedback and turns coding ability into an RL-compatible environment.
Some visual data comes from real-world scenes, while some is synthesized by labeling existing images with spatial-relation questions, such as asking what is above or below an object. This gives visual reasoning structured supervision.
The team also rendered code and ordinary text into screenshots or PDF-like images, requiring the model to perform on visual inputs at a level close to the original text. The aim is to prevent a sharp drop in capability when products encounter screenshots.
37. Partial rollouts prevent long chains of thought from crushing training throughput
An RL system must accommodate policy training, long-text rollouts, reference-model probability calculations, and reward evaluation at the same time. Kimi uses Ray-like tools for scheduling, Megatron on the training side, and vLLM for inference; the community can also combine FSDP, SGLang, and other systems.
Training and inference shard parameters differently, so the system must move weights, manage GPU memory, and schedule optimizer states between CPUs and GPUs. 潘佳怡 believes another 50%–100% efficiency improvement in existing frameworks would not be surprising.
The slowest response may run to tens of thousands of tokens. If the system waits for the entire batch to finish, large numbers of GPUs sit idle. Partial rollouts pause unfinished trajectories at a threshold and resume them in the next round, preventing “the slowest horse” from blocking every device.
Code tests, rule checks, and chain-of-thought reward models also add latency. k1.5 therefore computes rewards asynchronously, increasing throughput and reducing bubbles in the training pipeline.
38. Long-to-short experiments show that reinforcement learning remains the best way to compress reasoning
Kimi tested 4 direct approaches: merge the parameters of long- and short-chain models; use only short correct responses after rejection sampling for SFT; use DPO to prefer short responses; and apply long-to-short RL with a length penalty.
Parameter merging, SFT, and DPO all worked, but RL performed best at the same token count and used the fewest tokens at the same performance level. The broader relationship remained intact: longer chains of thought produced higher performance, approximately linearly.
Model-size ablations point to a potentially favorable outcome for edge devices. Larger models are stronger at the same chain length, but smaller models allowed to generate longer chains may compete with larger ones and could eventually approach server-model performance on phones.
Simplified ReST, STaR, or filter-based rejection-sampling fine-tuning learns only from correct samples and applies no negative gradient to incorrect ones, often improving scores by just 3–4 points. k1.5- and R1-style algorithms can improve them by 10–20 points. “Do less of the bad” is not an optional detail.
39. The next competitive cycle will revolve around agent data, compute, and organizational efficiency
DeepSeek’s remaining weaknesses include function calling, multi-turn dialogue, complex role-play, JSON-structured output, mixed non-Chinese and non-English languages, and reasoning models’ sensitivity to prompts. The larger goal is to extend RL from competition code to real software-engineering agents, including SWE-Bench, MLE-Bench, and Kaggle-style tasks. 潘佳怡 also mentioned SWE-Gym, a public project she built with collaborators, calling it one of the earlier efforts in this direction.
Data will not disappear; it will change form. Future systems need hard problems, solution paths, and standard answers, as well as complete trajectories for software development, industry research, and tool use. She relayed that OpenAI pays PhD students roughly $100–$200 per hour to annotate specialized data, while Kimi’s appendix lists around 20 data-annotation contributors. 张小珺 called them “AI coaches and AI trainers.”
Routes that rely less on humans may include self-play, weak supervision, and chain-of-thought reward models, but 潘佳怡 emphasized that these areas are still exploratory. Large volumes of user data have not yet become a stable moat; their value may change only when models can observe screen operations and workflows in specialized software.
She does not believe DeepSeek has already “defeated OpenAI.” By her estimate, OpenAI has hundreds of thousands of H100s, while Stargate targets millions of B100/B200-class GPUs, leaving a 1- to 2-order-of-magnitude gap in China. But DeepSeek has demonstrated exceptional resource-conversion efficiency. Compared with Meta, which is burdened by bureaucracy, KPIs, and internal friction, DeepSeek’s organization can turn fewer resources into frontier innovation.
The show’s final judgment is not that scale no longer matters. It is that post-training remains at a very early stage, compute may increase by an order of magnitude every few months, and algorithms may continue steepening scaling laws. After reading R1, the reaction that excited 潘佳怡 most was: “It turns out it’s that simple.” 张小珺’s closing formulation was: “Elegant algorithms, or elegant technology, are often the simplest and cleanest technology.”