Ep. 038 - Is GLM 5.3 a Mythos-Class Cyber Model From China? (InferenceX)

Summary

• GLM-5.3’s efficiency is not a straightforward bearish signal for memory demand. Kimbo’s distinction is that sparse attention eases bandwidth pressure but does not eliminate KV-cache capacity requirements: “you still have to keep all the tokens.” HiSparse moves cache between HBM and DRAM to support more concurrency; Jordan expects cheaper serving to drive “a lot more HBM and a lot more bandwidth” over time.
• AMD’s MI355 challenged NVIDIA’s serving economics within weeks of GLM-5.3’s launch. At 150 tokens per second per user, Wenyao reports costs roughly matching B200; Jordan also highlights MI355 throughput at 100 tokens per second per user. InferenceX showed “the red AMD curves overtake the green Nvidia curves,” before NVIDIA improved again—making inference-engine optimization a moving competitive variable.
• Ultra-low latency has different economics from maximum aggregate throughput. TailRT reaches up to 470 tokens per second at batch size 1 through a decode megakernel, but cannot serve large batches. Kimbo calls memory bandwidth “the most important part” in this regime, while Wenyao says he does not use ultra-fast modes himself: “It’s too expensive.”
• Post-training, not a larger base model, drove GLM-5.3’s reported capability jump. Wenyao reports more than 10% improvement in agentic benchmarks; Kimbo calls the cybersecurity change from GLM-5.2 a “night-and-day capability difference.” Jordan describes GLM-5.3 as a “frontier cybersecurity model,” based on his own experience of Anthropic and OpenAI refusals in CVP threat-teaming work and his fallback to GLM-5.3. Kimbo says it is “definitely not Mythos class,” though he tentatively calls it near Fable level.
• Z.ai’s edge may lie as much in research throughput as in new algorithms. Jordan’s pushback is that familiar post-training methods are not unique; the advantage can be running more, cheaper experiments on equivalent compute. Kimbo highlights critic-based SAO training and SLIME, “arguably one of the best-designed” frameworks, while cautioning that no detailed technical report has appeared since GLM-5.
• Chinese-chip deployment is a public claim; optimization for a particular supplier remains a hypothesis. The GLM-5 paper claimed day-one adaptation across seven domestic platforms, and Z.ai says OxAlpha/GLM-5.3 Flash ran entirely on Chinese hardware—Kimbo’s caveat is “assuming everyone’s honest.” His calculated arithmetic intensity around 120 roughly fits Moore Threads, but he explicitly says that proves neither optimization for those chips nor actual deployment.
• The next gains might come from scaling hybrid attention or post-training for specialized workflows. Kimbo calls GLM-5.3 Flash “almost like a Frankenstein model,” combining DSA with Kimi Delta Attention, and suspects scaling it to GLM-5.3’s size could yield further gains. Wenyao’s enterprise route is post-training on proprietary data rather than financing pre-training from scratch.

Deep dive

1. Sparse attention relieves bandwidth pressure, not cache capacity

• Kimbo Chen’s distinction: linear attention changes the attention algorithm; sparse attention preserves it while selecting the most relevant tokens. Since GLM-5, Z.ai has used DeepSeek Sparse Attention, introduced with DeepSeek-V3.2: a Lightning Indexer selects tokens for attention built on Multi-Head Latent Attention. Kimbo says dense attention is infeasible for million-token sessions; Jordan frames efficiency as important to serving them affordably.
• Kimbo rejects the idea that sparse attention solves KV-cache capacity: “you still have to keep all the tokens” in the relevant context. HiSparse, the sparse-attention version of HiCache, instead offloads cache from HBM to higher-capacity, lower-bandwidth DRAM and brings relevant portions back. The bandwidth saving and capacity workaround are different mechanisms.
• Wenyao Gao’s B200 example shows a substantial difference in host-memory percentage between concurrency levels 8 and 16. The effect is more pronounced on B200 than B300 because B200 has less memory per chip. Recalling the memory-stock selloff around DeepSeek V3 and R1, Jordan argues efficiency permits more users rather than automatically less memory demand. His Jevons paradox expectation is “a lot more HBM and a lot more bandwidth.”

2. AMD catches up as serving software changes the cost curve

• Wenyao reports NVIDIA led at launch, then AMD reached roughly B200’s cost at 150 tokens per second per user within weeks. Jordan corrects the chronology—GLM-5.3 arrived in early August, not October—and highlights MI355 throughput at 100 tokens per second per user. InferenceX’s history shows “the red AMD curves overtake the green Nvidia curves,” followed by NVIDIA improving again.
• Jordan points to peak efficiency around 45 million tokens per $1 of TCO. Wenyao’s explanation separates fixed chip capabilities from inference-engine development: AMD engineers are “working hard overnight,” repeatedly optimizing software and catching the cost per million tokens of many NVIDIA chips.
• Wenyao’s low-latency counterexample is TailRT: a decode megakernel reduces launch overhead and reaches up to 470 tokens per second at batch size 1, but cannot serve large batches. DRAM offloading adds latency, particularly in the tail. He contrasts Jane Street wanting answers within seconds with his own ordinary agent work: “I can just wait and maximize the output as long as the cost is affordable.”
• Jordan links that trade-off to Rubin Ultra’s discussed capacity reduction from 1 TB to 192 per GPU while preserving bandwidth. Kimbo interprets it as prioritizing bandwidth: in ultra-low latency, “you’re fighting against how fast you can load your model.” Capacity still matters, but caching, networking and overlapping transfers with computation offer ways around shortages.

3. Post-training delivers the capability jump, with limited technical disclosure

• Jordan calls GLM-5.3 a “frontier cybersecurity model,” based on his experience of Anthropic and OpenAI models refusing work he was doing in CVP threat-teaming; he says he falls back to GLM-5.3 and notes its weights are downloadable. Kimbo pushes back on the strongest comparison: “technically it’s definitely not Mythos class.” He tentatively says it is near Fable level, while unsure of Anthropic’s exact term.
• Wenyao calls the release a “wake-up call”: agentic benchmark results improved more than 10% through post-training rather than a fresh pre-training run. Kimbo describes GLM-5.2 versus GLM-5.3 in cybersecurity as a “night-and-day capability difference.” For them, this shows how much capability can be gained through post-training without scaling up the model.
• Kimbo describes synchronous GRPO first, prioritizing stability. Because agentic workloads have longer sequences, where reward signals can be diluted, the second stage switches to a more REINFORCE-like variant that tolerates policy staleness. That lets them use an older or most recent policy without extensive importance-ratio and policy-staleness calculations.
• Kimbo also highlights PPO-style training with a critic grading rollouts and co-training alongside the actor: Single-Rollout Asynchronous Optimization, or SAO. His caveat matters: GLM has published no detailed technical report since GLM-5; his account combines researcher discussions, announcements and published research.
• Jordan challenges whether those familiar algorithms amount to a distinctive advantage; Kimbo initially agrees they are not unique to Z.ai. Jordan then clarifies that efficiency itself enables more experiments and research jobs from equivalent compute. Kimbo calls SAO a clear innovation and SLIME “arguably one of the best-designed post-training frameworks, if not the best.”

4. Chinese-chip deployment is claimed; supplier-specific co-design is unproven

• Jordan reads the GLM-5 paper’s claim that “from the first day” it was “full-stack adapted to Chinese GPU ecosystems,” spanning kernels through inference frameworks across seven platforms: Huawei Ascend, Moore Threads, Hygon, Cambricon, Kunlunxin, MetaX and Enflame. His point: infrastructure optimization need not mean NVIDIA or AMD.
• Kimbo adds that Z.ai publicly claimed GLM-5.3 Flash, under the name OxAlpha, was fully served on Chinese hardware. He recalls tests suggesting OxAlpha served “100 trillion tokens or something”—a figure that led people to doubt a Chinese lab could serve it on GPUs. Jordan dates OxAlpha’s OpenRouter appearance to August 20. Kimbo’s condition is explicit—“assuming everyone’s honest”—but he thinks Z.ai has worked through kernel and system blockers sufficiently to serve efficiently.
• Kimbo calculates GLM-5.x decode arithmetic intensity around 120. DeepSeek had fitted its model for H800s; his estimate for GLM seems to fit Moore Threads GPUs. He says GLM’s configuration would not be optimal for B200 or H200: it may target something else or sacrifice hardware–software optimization for undisclosed reasons. He calls the Moore Threads idea a “conspiracy theory”: the calculation does not prove optimization for those chips or actual deployment.
• Kimbo also cites Z.ai’s account of models helping bring up and verify GPU kernels for unfamiliar, non-GPU hardware while monitoring jobs for system efficiency. Wenyao argues that with the right harness, tools and sufficient budget, agents can write faster kernels than humans. Asked what AMD’s TileRT megakernel cost in tokens, Kimbo answers: “I don’t have a good idea, honestly.”

5. The next gains may come from hybrids and proprietary data

• Kimbo guesses Z.ai will probably keep the architecture for further GLM-5.x releases, perhaps GLM-5.4, but sees another path in GLM-5.3 Flash, “almost like a Frankenstein model” mixing DSA with Kimi Delta Attention. He suspects Z.ai can scale this hybrid to GLM-5.3’s size and probably extract more performance or capability.
• Kimbo says GLM-5.3 arrived in early August and GLM-5.3 Flash in late August; he estimates Flash may have been trained in two or three months, with two months the aggressive-case estimate. He notes that betting on KDA was a major bet before Kimi K3 arrived in mid-July.
• Wenyao argues proprietary-data post-training can improve specific workflows without funding pre-training from scratch; he cites Harvey’s Kimi-based agent and Periodic Labs’ AI-for-science work. He also imagines large two- or three-trillion-parameter models alongside highly efficient, “good enough” models whose cost companies need not worry about.
• The unresolved input is human labor. Kimbo says Z.ai discussed “a lot of human effort” in GLM-5.2 and GLM-5.3, but will not clarify its nature. He is “pretty sure” it concerns labeling; what kind of labeling and effort remains unanswered.