陈天奇

XGBoost, MXNet, TVM, and MLC LLM were not four separate attempts to chase the next hot trend, but one technical throughline in Chen Tianqi’s sustained effort to remove the bottlenecks to machine learning at scale. He is a problem-driven rather than method-driven researcher: tree models, deep learning frameworks, compilers, and even hardware instruction sets are all means to the same end—“letting fewer people build and deploy AI with fewer engineering resources.” For infrastructure observers, the implication is that value may not settle in any single model generation, but in the bring-up, optimization, and deployment friction that recurs across models and hardware.

XGBoost shows how a small team can build a durable moat through an exceptional single-point product, tight algorithm–systems integration, and steadily expanding community trust. At launch, its goal was to be the world’s fastest Gradient Boosting implementation, with missing-value handling built directly into the model; before recording, He Tairan saw that the paper’s citations were approaching 70,000. Chen attributes the success to “doing one thing extremely well,” focusing on a single algorithm, and building with the community—not to having forecast the market size in advance.

MXNet’s exit shows that leading performance and backing from large companies cannot substitute for user experience and community momentum. MXNet delivered excellent multi-GPU performance for years, with Amazon and NVIDIA both heavily involved, but the team initially tried to maximize performance and usability at the same time; Chen later concluded that “we should actually have chosen user experience first,” allowing PyTorch to capture stronger developer mindshare. The deeper lesson was that the engineering cost of manually chasing every GPU generation was unsustainable, directly giving rise to TVM.

Model convergence will not automatically eliminate machine learning compilation; hardware specialization instead makes model–chip co-design tighter and more frequent. Chen acknowledges that a small number of models accounting for most inference could compress the space for general-purpose compilation, but every chip generation brings new programming methods, model variants, and specialized operators that require fresh adaptation; “there is no single recipe that gets you the next 2x.” MLC LLM is therefore betting on unified inference across cloud and edge, while edge adoption ultimately depends on whether local models become “good enough” for specialized use cases, after which cost, privacy, and latency can amplify demand.

The gap between universities and frontier labs with 10,000 or 100,000 GPUs is unavoidable, but 100-GPU-scale resources, the latest hardware, and open-source collaboration are still enough to crack high-leverage vertical bottlenecks. Chen does not choose topics by paper-level return on compute; he looks for the module where the system is genuinely stuck. XGrammar, a structured-JSON generation project that has become one of the de facto standards in the open-source ecosystem, is an example of “making one small module the best it can be.” “Resource constraint is also an opportunity to make people more creative.”

OctoML demonstrates that open-source technology does not convert into commercial revenue in a straight line: open source is the starting point, and the final product will almost inevitably differ from the original idea. The company initially sold cross-hardware model optimization and deployment, then shifted about two years ago to offering endpoints and APIs for models such as Llama; He Tairan noted that it was ultimately acquired by NVIDIA at the end of 2024. Chen’s founder retrospective does not shy away from the importance of PMF, leadership, and coordination, but he still insists: “If we kept optimizing ResNet, we would lose terribly in the end.” The stack must be willing to reinvent itself.

The most important form of long-termism in this episode is not sticking to the original path, but remaining willing to choose a new problem, rewrite the system, and start again after failure. From spending 2 years pursuing the wrong method on the right ImageNet problem to investing 11 months late in his PhD to build TVM’s first working version, Chen’s risk tolerance comes from the belief that “failure is not that scary.” The more resources, reputation, and responsibility one carries, the harder it becomes to preserve one’s original intent; his reminder to himself is: “Courage may be a promise your past self made to you.”

🔗 Original source & video: Chen Tianqi: Machine Learning Systems, Long-Termism, Original Intent, XGBoost, MXNet, TVM, MLC LLM, OctoML, CMU, UW, ACM Class