121. An Interview with DeepMind’s 谭捷: Robots, Cross-Embodiment Transfer, World Models, Gemini Robotics 1.5, and Google

Summary

• Robots are still 2-3 years from their GPT-3 moment, and another 5-10 years from real-world deployment. 谭杰’s timeline is clear: “we’ve seen enough signs of life to show that scaling up the current training recipe works,” but it takes 1-2 years to go from paper to a live demo people are willing to show, and 5-10 years from demo to deployment—just as autonomous driving took a decade to move from early demos to real products. The hard numbers today: simple pick-and-place succeeds over 90% of the time, while fine manipulation such as zipping a zipper is only at 30-40%; “a 30-40% success rate is unusable in real life.”
• Gemini Robotics 1.5 delivers two breakthroughs, the second aimed directly at data scarcity. First, it inserts a thinking trace into the VLA, allowing a robot to break “sort the clothes by color” into multiple steps while thinking and acting. Second, it introduces cross-embodiment Motion Transfer: Aloha, a tabletop robot that had never seen vertical scenarios, was mixed with data from a Franka and “suddenly could do those things too”—“suppose you learned to drive, and I had never learned the task of driving, but I learned to drive too.” The method itself is “secret sauce”; he refused to elaborate even after two follow-ups.
• The endgame bet is synthetic data, not teleoperation. He broadly accepts 王鹤’s math—a humanoid costs at least $100K, 10,000 units cost $1B, and monthly maintenance runs from several hundred million to $1B: “I would be very surprised if the endgame is solving robotics through fully teleoperated data. The probability is very low.” He counts himself among the believers in “scalable data”—simulation, egocentric video collected by people wearing cameras, and data generated by video models. His final rapid-fire answer was: “Real data alone cannot solve robotics.”
• Video generation is the new simulation, and world models are Silicon Valley’s next paradigm bet. “In the not-too-distant future, traditional physics-based simulation will gradually be replaced by simulation from generative models”—500 household scenes once required manual construction; now they require 500 prompts. The test is interactivity: Genie, for example, can let you drive a dragon, turn left or right, and see entirely different worlds, making it more like a world model; static video is not. He once predicted usable video generation would arrive only in 2030: “I definitely underestimated how fast video generation would develop.”
• To date, the overwhelming majority of robot foundation models are still fine-tuned multimodal models; “so far, not yet” describes robotics as an independent discipline—a structural advantage for companies with foundation models. Nobody is pretraining a robot foundation model from scratch: it would require tens of thousands of GPUs for months, and there is not enough data. He believes Gemini’s vision encoder has seen vast amounts of internet data, so “visual generalization comes for free.” 张小珺 speculated that those emphasizing the independence of the robot brain may precisely be startups without a foundation model.
• Chinese robotics hardware is sprouting everywhere, but the data flywheel has not started turning. Responding to the idea that Chinese companies have hardware and resemble Tesla’s route, he cut through it in one sentence: “The fundamental difference is that Tesla’s cars are useful,” which is why Tesla has a data flywheel; today’s robots have not cleared the usefulness bar. The cultural difference is patience: Silicon Valley will burn money for a decade on a belief, while in China “you collect a dozen hours of data, look at the results, and decide whether to invest more—but with only a few hundred hours of data, you cannot even validate the scaling law.”
• The talent war has already reset prices: Silicon Valley is now working 996 too, and Meta’s $100M packages have shaken the entire industry. He works 70-80 hours a week; after comparing that with China, he “realized that China may not necessarily be more intense than my schedule.” Retention is not always about money: top talent cares more about whether they can sit in the driver’s seat during a transformation—“someone with a sense of mission will not tolerate ‘I’m on a wrong ship.’” The episode’s two single-variable bets are humanoid for embodiment and end-to-end for architecture. At the same time, he repeatedly poured cold water on the hype: a demo is the best of ten attempts, and “people should not overestimate what robots can do today, then underestimate how long it will take robots to become real.”

Deep dive

1. From Graphics to Robotics: “Robotics Is Graphics in the Real World”

• 谭杰’s path: after graduating from Shanghai Jiao Tong University, he founded a graphics startup in Shanghai somewhat like Kujiale; he then earned a graphics PhD at Georgia Tech under Karen Liu, now at Stanford, and interned at Pixar working on physics-based character animation. After graduation he joined light-field camera company Lytro—“take the photo first, focus later”—and moved to Google Brain after 1.5 years, where he has spent nearly a decade.
• His core framing for the transition was: “You can think of robotics as doing graphics in the real world, or graphics as doing robotics inside a simulation.” The evidence is that Berkeley’s Sergey Levine also came from graphics; the two gave presentations back-to-back at SIGGRAPH before independently moving into robotics.
• The opportunity window was the era when robotics was still rule-based and dominated by MPC: “in the past, if you did robotics, you definitely needed a PhD, otherwise you couldn’t understand the math.” In simulation, robots could already do flips and run, while real robots in the DARPA Challenge were still falling over. “If we could put what worked in simulation into the real world, robotics would make extremely substantial progress.”

2. The Paper That Opened the First Paradigm Shift

• In 2015, he went on the academic job market with the slogan “use graphics to transform robotics,” interviewing at Cornell and elsewhere. The field was skeptical, and “I didn’t get the faculty job I wanted, so I went to work in Silicon Valley.”
• His first Google paper, “Sim-to-Real: Learning Agile Locomotion for Quadruped Robots,” applied deep reinforcement learning to quadruped locomotion, borrowing PPO from the AlphaGo era rather than from large language models. “That first paper of mine actually opened the entire application of reinforcement learning and sim-to-real to robotics.”
• The evidence is visible in hindsight: a decade ago, Boston Dynamics was the best, pushing MPC to its limits while “the entire academic community had no idea how they did it.” Five years later, Unitree robots were boxing, dancing, climbing walls, and doing flips—“essentially because everyone adopted the same technology, reinforcement learning, and then sim-to-real.” His ten-year summary: the first paradigm shift was reinforcement learning; the second was large language models.

3. Large Language Models Are the Brain; Reinforcement Learning Is the Cerebellum

• Before LLMs, robots had no common sense: tell one to make coffee and “it had no idea what you meant”; everything had to be programmed. LLMs added two things: a natural-language interface—“Anyone can now tell a robot what to do”—and commonsense decomposition. Ask Gemini for the steps to make coffee and it can produce a detailed plan; “if every step is simple enough, it can execute them one by one.”
• The division of labor runs through the entire episode: the brain handles reasoning and planning, through the large language model; the cerebellum executes—walking, balancing, and manipulation—where “reinforcement learning is extremely strong.” Both brain and cerebellum are necessary. His definition of intelligence in embodied AI is also twofold: the brain needs planning and problem-solving ability, while the cerebellum needs dexterity.

4. Is Embodied Intelligence an Independent Discipline? “So Far, Not Yet”

• 张小珺 brought in the Yunqi Conference view that an embodied foundation model is an independent direction, not an extension of the LLM. 谭杰 did not hedge: recent progress in robot intelligence “has mainly depended on multimodal large models.” What is missing is robot-action output, so the standard approach is to take a multimodal model as the base and fine-tune it with robot-action data. “So far, there hasn’t really been a qualitative change.”
• He left room for a future split: when robotics hits bottlenecks around data formats or world models, “it may become a more independent discipline.” For now, the overwhelming majority of VLA and robot-foundation-model work remains fine-tuning on top of existing models.

5. The State of Play and the Timeline: Single Tasks Work, Generalization Is Still Far Away

• The generalization gap is quantifiable. Narrow-domain demos such as folding clothes can look excellent, but a general-purpose model may achieve “90-something percent” on simple pick-and-place while fine manipulation such as zipping a zipper may achieve only “30-40%.” “A 30-40% success rate is unusable in real life.” His conclusion: “our progress is extremely fast, but the gap is also extremely large.”
• Using GPT as the benchmark, robotics may need another 2-3 years to reach GPT-3 or GPT-4 levels, “so you genuinely feel, wow, this thing is actually useful.” After that, it needs another 5-10 years to become truly deployed.
• The reason deployment is slow follows a general formula: idea to paper takes 6 months to 1 year; paper to a live demo takes 1-2 years because “when you write a paper, you make a lot of assumptions”; demo to deployment takes 5-10 years. Autonomous driving went from DARPA vehicles traveling a few dozen kilometers to Waymo and Tesla products over a decade. “That is not an especially surprising gap.”

6. Why Bet on General-Purpose Robots Instead of Solving One Domain at a Time?

• 张小珺’s pushback was pointed: autonomous driving has such a limited action space and still took a decade, so why not solve robotics one domain at a time? 谭杰 acknowledged that “a lot of the debate here is philosophical. Both paths are possible; neither is right or wrong.”
• His answer was more sociological: “There may be a big shot—perhaps Elon Musk, perhaps Steve Jobs. Once a big shot speaks, many followers appear, and a lot of money and talent flow in.” The general-purpose humanoid path then becomes the answer everyone builds toward.
• The precedent is the LLM: there used to be one model for machine translation and another for VQA, but “once you truly have a generalist model, specialist models simply cannot compete with it.”

7. Why Progress Accelerated So Suddenly Over the Past 1.5 Years: Technology, Money, and 996

• Three variables drove the acceleration. First was the evolution of VLMs: thinking models trade inference-time token budgets for answer accuracy, “so robots can think as well.” Second was attention from leadership: “once CEOs care, the resources and compute allocated to it grow.”
• Third was competitive intensity: “people used to think 996 was a Chinese tradition; now Silicon Valley is 996 too.” Nobody wants to lose this competition. If you are the world’s second-best team, the strongest people on your team will think, “I need to go to the world’s number-one team.” The inflection came 1.5-2 years after ChatGPT launched, when the field realized this technology could be used for robotics; then everyone started pushing extremely hard.

8. Data Is the Clearest Bottleneck: How to Build the Pyramid

• Language-model data is “free,” while robots live in unstructured environments and need enormous amounts of highly diverse data that simply does not exist today. The shortage is so severe that “the amount of data today cannot possibly saturate the model’s capacity, so we have not yet discovered what the second bottleneck is.”
• He endorses a data pyramid: internet data at the base, large but noisy; egocentric video above it, easy to collect with a pair of glasses and abundant on YouTube, though the gap between humans and robots makes it unclear how to use; simulation data above that, with a sim-to-real gap; and robot-specific teleoperation data at the apex. Pretraining only needs to teach an understanding of physics, where scale wins. The apex requires small amounts of high-quality, embodiment- and task-specific data. “The entire pyramid is extremely important.”

9. Gemini Robotics 1.5 Breakthrough One: Robots Think While They Act

• A thinking trace is now part of the VLA’s output. Earlier VLAs took an image and language as input and directly output motor angles. For a multi-step task such as sorting clothes by color, the model first outputs text—“this is red, so I should throw it into the red pile”—then an action, then thinks about the next step. “The previous generation could not think through things in this level of detail.”
• The second benefit is transparency: “the robot can tell people what it wants to do next and why,” improving both safety and human-robot interaction. “It may make people feel more at ease.” As for whether switching repeatedly between text and action modalities causes confusion, “we actually have not seen that problem”; start and stop tokens keep the modes distinct.

10. 0.5 Seconds vs. 20 Seconds: Two-Speed Models Are a Compute-Constrained Transition

• A robot’s thinking budget and an LLM’s are “two completely different worlds.” An LLM can think for 20 seconds before answering and everyone is happy as long as the answer is right. But if a robot takes 20 seconds to think at every step, “you’ll go crazy”—it may have only 0.5 seconds per step.
• The architecture therefore separates ER, or embodied reasoning, from the VLA. Packing luggage may require extra information: ER can search Google, check the calendar, and read the weather report, think for 20 seconds to formulate a plan, and then hand it to the VLA for execution. That is the meaning of the paper title “Bringing AI Agents into the Physical World”: an agent’s defining feature is tool use. In the demo, a robot sorting trash first checks Google Maps to confirm it is in San Francisco, then applies local rules.
• But he explicitly calls the two-model setup transitional. “Language is not a high-bandwidth communication method; it throws away a lot of information.” A unified model might be optimal, but the reasoning model would have to be extremely large, and “current compute is insufficient” to support a large model making 5-10 decisions per second. Autonomous driving is not comparable: its planning relies on maps and other conventional methods outside the model, so the model itself need not be large.

11. The Overthinking Trap and the Reward-Function Dead End

• The easiest failure mode after adding thinking is overfitting. If thinking annotations are too uniform, “when it encounters tasks it has never seen, its thinking becomes very strange and it does completely nonsensical things.”
• He only partly accepts the popular claim that embodied AI lacks verifiable feedback signals. Imitation learning has labels and therefore does not lack signals; reinforcement learning is the problem: how do you mathematically express whether “put this dish in the refrigerator” succeeded? Writing a reward function for thousands of manipulation tasks is “almost impossible.” That is why RL solved walking—“it moved forward and did not fall,” making the reward easy to define—but not manipulation. Motion Transfer relies mainly on imitation learning and avoids this dead end.

12. Breakthrough Two: Motion Transfer and a Qualitative Leap Across Embodiments

• First, he poured cold water on the premise. Cross-embodiment training has been discussed for years but consistently oversold. Google’s own RTX mixed data from more than 20 schools and claimed a benefit, but “it is actually very difficult to quantify.” Physical Intelligence said mixing five embodiments made the model more general; “that is also difficult to quantify.”
• This time the evidence is hard. Aloha is a tabletop bimanual robot trained entirely on planar data. Asked to take a book from the top shelf, “it would just keep fumbling around on the tabletop.” After adding data from an industrial bimanual robot operating at a vertical workbench and training with Motion Transfer, “Aloha suddenly could do that too.” His analogy: “Suppose you learned to drive, and I had never learned the task of driving, but I learned to drive too.” “This fundamentally solves the lack-of-data problem.”
• The boundary matters: a new robot does not become useful with zero training. “Its generalization is task-level generalization, not generalization across robot embodiments.” Tasks transfer only among embodiments represented in the training data. The three embodiments were Aloha, an industrial bimanual robot, and a humanoid whose name was unclear. As for the method: “It’s a secret sauce.” Asked twice, he replied, “It’s really hard to comment.”

13. The Ceiling on Transfer: Embodiment Gap and How Much Data You Have

• The first variable is the configuration gap. Moving from a single-arm gripper to a bimanual humanoid is “definitely an extremely difficult problem”; even moving from a gripper to a five-finger dexterous hand is harder than moving between similar grippers. “If the embodiment gap is very large, cross-configuration transfer becomes difficult.”
• The second variable is the target embodiment’s own data volume. For a robot that already has a lot of data, the positive delta from other embodiments “will be very small.” For a data-starved embodiment such as a humanoid, transferring from Aloha’s huge dataset can be highly effective.

14. The Data Road Map: The Probability of a Teleoperation Endgame Is “Very Low”

• He was unequivocal about the next priority: “data, data, data.” The direction is to use less and less teleoperation and more data that can be collected quickly—simulation, human video from YouTube, and even video generated by video models. “If we can make breakthroughs there, solving robotics will be within sight.” The current reality is that the field relies mainly on real data, “and the cost is extremely high.”
• On 王鹤’s arithmetic—a humanoid costs at least $100K to manufacture, 10,000 units cost $1B, two-shift teleoperators earn tens of thousands of RMB per month each, and maintaining 10,000 units costs hundreds of millions to $1B per month—he said: “I haven’t carefully checked whether the exact numbers are right, but this is definitely not a very scalable approach. I would be very surprised if the endgame is solving robotics through fully teleoperated data; the probability is very low.”
• The key clarification on real versus simulated data is worth retaining: generalization is caused by data coverage, not by whether the data is real or virtual. “If you have never collected data of cooking in a kitchen, it will be hard for your model to cook.”

15. Nobody Is Pretraining a Robot Foundation Model From Scratch

• The paradigms differ. LLM researchers study pretraining mixtures, post-training, and how to use RL; in robotics, “very, very few companies are actually pretraining a large robot model from scratch.” Pretraining may require tens of thousands of GPUs or several months, and there is nowhere near enough robot data, so most teams fine-tune pretrained models.
• 张小珺 speculated that those emphasizing an independent robot foundation model may come from startups that do not have a foundation LLM, while teams working on top of Gemini have a different experience. 谭杰 said this was “very possible,” citing Gemini’s vision encoder: it has seen vast amounts of internet data, while academics are still working to solve visual generalization across lighting and backgrounds. “We found that visual generalization comes for free; we did not need to do any research, and it was already particularly good.”

16. Video Generation Is the New Simulation: A 2030 Prediction That Aged Poorly

• The definition of simulation is shifting. It once meant solving physics equations with Bullet, MuJoCo, or Isaac Gym. Now “many people think simulation is simply generating a video, if the physics in that video looks correct.” His view is aggressive: “In the not-too-distant future, traditional physics-based simulation will gradually be replaced by simulation from generative models.” The field is only “just getting started.”
• The economics are not straightforward: generating video costs more compute, but it solves scene construction. Traditional simulation would require a designer to manually build 500 household scenes; to generate videos of 500 household scenes, “you only need 500 different prompts.”
• He preserved the story of being proven wrong. A few years ago, the director of The Wandering Earth came to Silicon Valley and asked whether the director’s profession would still exist when The Wandering Earth 3 premiered. He reassured him that there was no hurry because it would take a long time. At the time he thought usable video generation would arrive in 2030. “I definitely underestimated how fast video generation would develop.” His rough rating: Sora to Sora 2 was “definitely a qualitative leap,” but compared with a model that might be Veo 3, Sora 2 had “nothing fundamentally different or particularly breathtaking.”

17. The Test for a World Model: “You Can Drive a Dragon”

• The definition in one sentence: provide the previous frame and the robot’s action, and the model predicts the next frame. Most video-generation models cannot change the next frame according to the action, so “I would not call them world models yet.” But the field is not far from moving from existing video models to world models.
• The contrast is concrete: a system that might be called Veo is a video-generation model; one that might be called Genie looks more like a world model. “You can drive a dragon. You can say I want to turn left or right, and you see a completely different world.” He did not name a clear winner, saying only that Sora and OpenAI are doing well and many smaller companies are working on it. Progress in every video-generation model “makes robotics extremely excited, because this is a new simulation or a new world model.”

18. Trading Compute for Accuracy: Simulation vs. Real Data Is a Matter of Faith

• If simulation is not realistic enough, vary the simulation parameters continuously and generate enormous amounts of data. “On average, it may cover enough of the physics of the real world”—use compute to trade for accuracy. The current bottleneck is hallucination and nonphysical behavior. Researchers like to test models on gymnastics videos, but during a flip, “who knows how many legs will be generated.”
• Why did some teams at the Yunqi Conference distrust simulation? His explanation was candid: real data produces results quickly; add data and you immediately see an improvement. But “the final amount of data cannot be infinite. It must have a glass ceiling.” Simulation requires foundational research, and early on “many things may fall within the category of noise.” Once the research breaks through, it has no glass ceiling. “So this is a matter of faith. The overwhelming majority of research is a matter of faith.”
• His own position: “I believe in the scalable data camp”—simulation, people collecting data with cameras rather than teleoperating, and data generated by video models. “All of these can use compute to give you an endless supply of data.” One unresolved industry problem is that data quality has no standard definition. When a data foundry’s data underperforms, “we will definitely say the data was collected badly, and they will definitely say your model was not trained well.” The result is an inevitable blame game.

19. The Architecture Debate: He Thinks End-to-End May Win

• Responding to 王星星’s argument that the industry discusses data too much and overlooks architectural innovation, he agreed that architecture matters. Motion Transfer itself contains “a lot of architectural and algorithmic innovation.” But “you can’t make a silk purse out of a sow’s ear”: if data volume and coverage are insufficient, generalization will not work.
• On hierarchical systems versus end-to-end, he drew on his own experience. Early PPO was inefficient, so he took a shortcut: conventional control at the bottom and RL for parameter tuning on top. “The results were decent and immediate.” But “the historical trajectory did not go in the direction I imagined”; the mainstream later became fully end-to-end, with state in and action out. The lesson: “Perhaps if you invest directly in the end-to-end unified model you ultimately believe in, that model may win.” His team is now “basically all end-to-end”; the two-model setup is only a short-term proof of concept.
• Is VLA the final architecture? “VLA is the best-performing model today,” but language loses information when describing complex behavior. “If I’m spinning a pen, expressing in language what I should do next is very difficult.” That is why Silicon Valley increasingly sees world models as the next paradigm: VLA may evolve into V-L-V, with vision and language going in, the next frame coming out, and action generated from there. It would coexist with VLA rather than replace it. On the claim that VLA+RL is a “dumb architecture”: “It’s not dumb. If you build it, it’s extremely powerful.” VLA looks formulaic only because the field’s exploration of action representation converged over the past year.

20. Three Stages of Thinking About Tactile Sensing: Proven Wrong, Then Reversed

• Stage one was the intuitive belief that touch mattered: “you have skin sensing the world through touch every day.” Stage two came when Stanford’s Aloha paper showed a robot using pure-vision teleoperation to remove a credit card—a very thin object—from a wallet. It “hit me in the face hard,” leading him to hold the belief that vision could handle 95% of tasks and touch was needed only for the remaining 5%.
• Stage three reversed the view. In recent teleoperation of a five-finger dexterous hand using scissors, the operator’s fingers went into the loops but there was no tactile feedback. “I didn’t know when to open or close,” and the scissors were impossible to control. The methodological conclusion: “When you think touch is unimportant, perhaps the technology simply has not reached that stage yet.” Vision handling 95% was valid in the gripper era, which he thinks robotics is still in; tactile sensing will be indispensable in the dexterous-hand era, and adding modalities is inevitable.
• On Musk’s claim that Optimus can watch YouTube to learn tasks and needs only vision, he agreed completely with the first half: “I hope that one day our robots can sit in front of a TV, watch Netflix and YouTube, and learn all kinds of skills.” He disagreed with the second: “Learning things through vision and solving problems through vision are two different problems.” Autonomous driving can be vision-only; manipulation requires repeated interaction with the world.

21. If You Can Bet on Only One: Humanoid Embodiment, End-to-End Architecture

• What happens if future hardware is not compatible with today’s brain? “That shows why cross-embodiment is so important.” His personal judgment: humanoids “will not be the only form, but they will definitely be a mainstream form. If you can bet on only one embodiment, I would still bet on humanoids.”
• But he stressed that the answer depends on which chair you occupy: “It depends entirely on whether you are working at a large company or building a startup.” His goal is to “solve AGI in the physical world,” so he focuses on the end-state embodiment; “everything else may be a distraction.” Startups carry investor expectations for revenue and must land short-term applications to get the data flywheel turning.
• His analogy is Waymo versus Tesla. Waymo makes every decision for L4; Tesla sells L2 and L2.5 cars and builds the flywheel. “Neither path is right or wrong.” But he credits Waymo with one major achievement: “It really removed the driver from the car.” He is a Tesla user and thinks FSD is extremely good, but having the courage to remove the driver is a zero-to-one breakthrough.

22. Change Battlefields: Once High Schoolers Can Use PPO, It Is Time to Build Foundation Models

• His reason for leaving locomotion reflects a general rule: “when a technology becomes easier to use, it spreads very quickly.” In the MPC era, solving the equations required a PhD. Once RL became accessible, “many high school students could download a PPO package online, get Isaac Gym from NVIDIA, train it, and deploy it.” The field immediately became crowded. He extended the point: everyone can do prompt engineering, and “the emergence of large language models may soon solve many, many problems—not just in robotics, but across industries.”
• That led him toward robot foundation models. He referred verbally to “2012”—a timeline that remains uncertain in the original subtitles—and described a straight line of progress: SayCan used language models to give robots common sense and decompose complex tasks; RT-1 showed that Transformers could absorb large amounts of data; RT-2 added action training to a VLM and borrowed internet knowledge; then came Gemini Robotics.

23. Ten Years of Google Robotics: From 10 People to a 160-180-Person Author List

• When he joined nine years ago, the team had roughly 10 people—“I think I was the only Chinese person.” Management was loose: “every researcher who joined could operate independently; you could do whatever you liked.” It was enjoyable, but “your individual impact was extremely limited, like a very well-paid PhD.” Google eventually decided it did not want to maintain “an academic lab but very costly,” and used promotion, performance reviews, incentives, and other structures to push toward army-scale collaboration. The robotics team now has 150 people.
• The scale is visible in the author lists. The first Gemini Robotics release in March had roughly 120 authors; version 1.5 had 160-180. Some people worked on the data pipeline, others on research, evaluation, or data collection. “You really need a very large team and extremely strong compute to do something like this well.”
• The organizational principle is simple: “What is top-down, but how is often bottom-up.” Motion Transfer is the example: 3-4 small teams each had an idea, and the ideas were mutually compatible, “like assembling building blocks.” His precise formulation of brute force is: scale is necessary—“if you do not scale up, it is hard for miracles to happen”—but “just scaling up, without being smart enough, just piling on compute and data, is not sufficient.”

24. Big-Company Disease and “This Era Cannot Afford to Wait”

• One of his responsibilities is research iteration speed, essentially fighting big-company disease. A researcher may need weeks to clear due diligence and legal compliance before using a dataset. Purchasing a new robot can take “more than 30 business days” from expressing interest to issuing a purchase order. He negotiated with legal and operations: “This era really cannot afford to wait.” After 30 days, researchers with a strong sense of mission may already have left. DeepMind’s top-down focus on robotics has helped by creating many special cases to accelerate the process.
• Performance evaluation recognizes impact. The number, quality, and citations of papers are one dimension; the value created by integrating results into products is another. Google is “a little more patient than Meta” on underperformance: there is no clear-cut quota line. People first enter an improvement plan—“you will have a second chance to prove yourself.”

25. Meta’s $100M Talent Hunt: Three Explanations, but Retention Comes from the Driver’s Seat

• Meta “has driven up the price of AI talent, extremely high.” Is $100M worth it? He offered three explanations without choosing one: supply and demand—every large company is AI-first, but top talent is scarce; compute leverage—“compute is extremely expensive, so it may be better to spend money hiring people who can use it well; the value is far greater than their cost”; and poaching people is still cheaper than buying a comparable company. Packages are often spread over 4 years, but he did not know the details of that specific case. Meta’s motivation is straightforward: its models had fallen out of the first tier over the past 1-2 years, and building a frontier-model team from scratch is difficult, so hiring people away is the most direct route.
• His retention playbook as a manager starts with checking whether the competing offer is rational and whether matching it would damage internal fairness. The real lever is mission: top talent “doesn’t necessarily care most about money,” but wants to be in the driver’s seat when transformation happens. “Someone with a sense of mission will not tolerate saying, ‘I’m on a wrong ship.’” His record: “I can often retain people. Of course I have failures, but more successes.”
• The Chinese-talent discussion followed naturally. The team is now “50-60% Chinese”: “AI involves a lot of math, Chinese people are relatively good at math, and they are both talented and willing to work hard.” He thinks AI will change the old pattern in which Chinese professionals struggled to reach Silicon Valley’s management ranks, citing 永辉, a Google Chinese executive who recently returned to ByteDance. He also acknowledged Indian strengths: strong communication and a greater willingness to speak up. “The proportion of Indians in senior positions in the US is higher.”

26. The “Ox and Horse” Interview Story: A White PhD Candidate Asked Them to Hire More Chinese People

• During recruiting, an American white PhD candidate unexpectedly told him: “I really hope your team can hire more Chinese people,” because the Chinese AI and robotics researchers he had worked with were “extremely hardworking and productive.”
• The candidate had also learned a word: “牛马.” He said he especially liked being a 牛马 himself—working like an ox and a horse—and asked whether he had misunderstood the term. In his mind, it symbolized diligence. Was the interview recorded? “It was.” 张小珺 asked whether he had only said this because 谭杰 was present. “I looked at his earlier papers; he really did have many Chinese collaborators.”

27. The Road Map and the Safety Red Line: Stop If Capability Outruns Safety Understanding

• Over the next 2-3 years, there will be a GPT moment when “people genuinely realize generalist robots are coming,” bringing in more capital. Within 5 years, robots with generalization will first land in traditional sectors—manufacturing, logistics, and supermarkets—but their behavior will no longer be a fixed sequence of automated actions. Within 10 years, robots will begin entering homes at scale.
• His safety position was categorical: “AI safety and robot safety are not child’s play. When AI can self-iterate, or robots can self-iterate, I think humanity faces an existential problem.” DeepMind has a responsibility and safety council that reviews the social impact of every model. In the worst case, “if robot capability exceeds our understanding of AI safety, you should stop developing robot capability, let safety catch up, and then move forward together.”

28. China and the US: Hardware in China, Patience in Silicon Valley, but China’s Robotics Data Flywheel Is Not Turning

• His observation after returning to China: Chinese hardware companies are “sprouting like bamboo after rain, supplying cheap and high-quality robot hardware to the world”—Unitree, Zhipu, and Xinghaitu among them. The cultural difference is capital’s patience. In Silicon Valley, even without early results, if people believe in something, they are still willing to spend time, effort, and money on it for 10 years. In China, the expectation is short-term deployment and profitability: collect a dozen hours of data, show decent results, and ask for more investment. “That initial funding is nowhere near enough.” Robotics needs tens of thousands of hours of data; with only a few hundred hours, “you cannot even validate the scaling law.” Musk’s significance is that he accomplished enough to make Silicon Valley willing to believe in something that initially looked like a far-fetched faith.
• 张小珺 argued that Chinese companies have hardware and can collect data first, making them more like Tesla. He dismantled the analogy in one sentence: “The fundamental difference is that Tesla’s cars are useful,” so Tesla has a data flywheel. Chinese companies’ hardware has not yet cleared the usefulness bar, so the flywheel cannot turn and the comparison with Tesla is difficult. Setting geopolitics aside, “intelligence is developing faster in the US,” while China is already strong in supply chains, hardware manufacturing, embodiments, and some aspects of control. Better China-US cooperation would be an important milestone for humanity, especially robotics.

29. Vertical Applications Can Make Money, but Once Generalists Mature, “Specialists Will Struggle to Survive”

• The vertical example he found convincing was an online clothes-folding demo, perhaps called Dyna. Flexible-cloth manipulation is difficult to solve with traditional controllers and planning algorithms; “it really requires the value of AI.” With high US labor costs and real demand, the demo found “the intersection” where AI can add value and replace conventional algorithms, while the vertical market has enough volume and demand. “Finding that intersection overall is still quite difficult.”
• But this is not his career objective: “Solve AGI in the physical world.” Once a general-purpose brain matures, it will cover vertical applications. “When a generalist is truly formed, specialists will find it hard to survive—I can do your task, but I can also do 100 other things.” Taken to an extreme, even autonomous driving could be absorbed: a household humanoid could drive for you, so perhaps you would no longer need to buy a car with autonomous driving. He admits this is an extreme scenario and not necessarily necessary from a utility standpoint, but it is replaceable.
• His five-stage theory of robotics is: automation—factory robots programmed by automakers, already achieved; teleoperated robots—hardware is mature but there is no brain, and “in many scenarios Tesla is teleoperated; sometimes you cannot tell whether it is teleoperated or autonomous”; narrow-domain generalists such as autonomous driving; true generalists, with humanoids entering homes and becoming as capable as people; and superhuman robots, which through reinforcement learning and their non-biological storage and power density could exceed human intelligence and physical ability in many fields.

30. Surprise and Cold Water: From Being Unable to Grab a Christmas Stocking to Completing 10 of 25 Unfamiliar Tasks

• A year ago, the team wanted to make a Christmas demo by putting gifts into a stocking. “At the time I thought the robot could not even pick up the stocking.” This year at CoRL, a researcher who appeared to be from MSR brought a box covered with buttons, sliders, knobs, and headphone jacks. He proposed 25 tasks absent from the training data, and the robot completed 10. “Six months ago, this was unimaginable.” Asked whether currently useful humanoids are an empty desert, he said: “I basically agree. Humanoids that can do useful work are currently an empty desert.”
• But throughout the interview he repeated the same warning: outsiders overestimate robotics. “People are used to filming the best result and putting it in a video. It may simply mean I filmed the best one of ten attempts.” Investors are somewhat more clear-eyed, but they also often overestimate the industry’s development. Is there a bubble? “There may be a huge bubble,” depending on whether progress continues to accelerate and whether real deployment scenarios are found. A year ago he thought short-term deployment was impossible; now the industry appears to have found several potentially viable directions.
• He accepted 张小珺’s observation that AI has a clear main line while robotics is “a bit here and a bit there,” with nothing solved and everything just beginning. “Robotics is a composite problem made up of many, many problems layered together; solving one problem does not suddenly make it work.” If he could choose only one priority: “How to obtain high-quality data is the first problem we urgently need to solve.” His closing line was symmetrical: “Do not overestimate robots’ capabilities today, then underestimate how long it will take robots to become real.”

31. Raising Children and Building Robots: Manipulation Is Way Harder Than Locomotion

• Training robots and raising children—his older child is 12 and his younger one is 5 months—are “very similar.” Children learn in two ways: reinforcement learning, by exploring and receiving rewards, and imitation, by watching parents and peers. The mismatch is that “somehow manipulation happens earlier for kids, but for robots, manipulation is way harder than locomotion.”
• The current capability profile is extremely uneven. Locomotion has surpassed adults: “at the Beijing Robot Games, Unitree’s humanoid ran faster than me.” Gripper manipulation is “like a 2- or 3-year-old child”—unsteady, but it roughly understands what you want and can get it right after several attempts. Dexterous hands? “Nothing works.”
• Does a machine with intelligence still count as a machine? “At least with the current methods of doing AI and robotics, it may still be closer to a machine.” Today’s machine learning “simply finds a statistical pattern in big data,” which is very different from human consciousness. Could he accept his child falling in love with a robot? His unusually honest non-answer: “This is a question I have not thought through carefully. What exactly is consciousness? I do not have a better answer.”

32. Rapid-Fire Questions and the Final Bet: You Have to Believe in Synthetic Data

• A sketch of his life outside work: racing games—“real tracks are too expensive, so I bought a very good simulator at home”—piano, and gardening. He recommends Start with Why, which “completely changed my communication skills,” and The 7 Habits of Highly Effective People. The papers he sees as having shaped robotics are his own Sim-to-Real quadruped paper, SayCan, RT-1, RT-2, RTX, and Gemini Robotics. Favorite place: Shanghai. Silicon Valley restaurant recommendation: the Yunnan restaurant near his home, Pink’s Bistro.
• His most important bet based on what he knows today—and the endpoint of the episode—is: “You have to believe in the value of synthetic data. Real data alone cannot solve robotics.”

Verification Notes

• The original subtitles give the year he turned to robot foundation models as “2012,” which conflicts with other dates in the interview timeline and cannot be resolved from the original subtitles alone.