E255 | Models Keep Getting Stronger—Why Don’t Users Feel It? A Return Visit with 张阔, President of Alibaba.com

Summary

• General-purpose models are already approaching 99/100 on many leaderboards, but Alibaba’s 107 tasks distilled from real operating data show that even frontier models achieve a pass rate of only “a little over 61%” with no human intervention. 张阔’s view is that gains in general capabilities such as Coding and mathematics do not necessarily translate proportionally into e-commerce task completion: “The fundamental capability level has not jumped that much every time a new model is released.” Vertical benchmarks therefore say more about commercial usability than launch-event scores.
• 张阔 estimates Coding Agent penetration at 99%, while commercial Agents remain at an early stage; the fundamental differences are digitization, verifiability, and feedback cycles. Whether code compiles and whether the output is correct can be checked immediately; business outcomes may take days, weeks, months, or even years to validate, and the same transaction can produce different answers depending on whether a company is optimizing for margin, market share, or speed.
• Accio has expanded from AI search into agentic trade and multi-platform operations, driving overall search growth of 20%, multimodal search growth of 130%, and compressing sourcing projects from a monthly process to roughly 24 hours. An Agent can turn an idea into a design pack, match factories, obtain quotes, calculate landed cost and margin, then handle logistics, HS codes, listings, and buyer feedback; 张阔 stresses that this is not a general-purpose model with a few prompts, but the product of Model × Harness × Context—“if any one of them is 0, the whole formula is 0.”
• The unit economics of commercial Agents depend on model routing, not on sending every task to the most expensive frontier model. In a comparison across 107 tasks, the host cited an estimated cost of $3.69 for Accio Work versus more than $9 for both Codex and Claude Code; Accio uses Pareto routing across small, specialized, and frontier models to bring costs down to roughly one-third. “An AI has to be affordable to be valuable.”
• AI’s impact on SaaS will not be uniform; one key distinction is whether a product owns the enterprise ontology and can turn existing workflows and context into assets an Agent can use. Simple feature-based SaaS may be weakened, while vertical systems in CRM, legal, and healthcare understand what the data means and whether a business is optimizing for margin or scale; 张阔 sees the more realistic path as two-way integration between SaaS and AI, with every technology inflection still creating new AI-native companies.
• Accio draws a deliberately narrow boundary around autonomy: AI can handle cross-platform reporting, product listings, customer-service reviews, and risk alerts, while humans retain control over payments, purchasing, budgets, inventory, advertising, and product taste. “A transaction can succeed and still ultimately make your business fail,” so when multiple Agents disagree, a single unified Agent can no longer eliminate the conflict; the owner must still make the final call based on targets such as gross margin, scale, or speed.
• Demand-side signals are emerging as merchants use Agents to start businesses, not merely to improve office productivity. CoCreate in Los Angeles estimated that 15,000 people registered and attended; a 15-year-old student in the UK used Accio Work to develop a towel-like product that cools people down and automatically cools pets, generating roughly £1M in annual sales, while MIT graduate Christian Reed sold measuring equipment to Schneider Electric and SpaceX and reportedly grew this year’s business to more than tens of millions of dollars. For investors, these cases point to the incremental opportunity of small teams using AI and global supply chains to expand their operating radius.

Deep dive

1. Coding Has Reached High Penetration; Commercial Agents Are Still Stuck in the Feedback Loop

• 张阔 estimates that AI penetration in Coding may already be 99%: “It’s hard to find a developer around me who writes code without using any AI,” and the vast majority of code “should be AI-generated.” Other verticals remain far from that level.

• Code is naturally suited to current models: the work is 100% digital, content in GitHub and various codebases is easy for AI to learn at scale, and compilation and correctness can be verified immediately. Copywriting and computation are therefore easier to automate than complex operating judgments.

• Commercial tasks lack a standardized process, draw on more varied data, and operate under different definitions of a “good result.” Whether a transaction serves margin or market share changes the budget, inventory, and advertising strategy, while the feedback cycle can stretch from a day to a year.

2. Accio Expands from AI Search to End-to-End Operations

• A year ago, Accio was still centered on Alibaba.com’s AI mode, where buyers used natural language or multimodal inputs to describe their sourcing needs. 张阔 says overall search has grown 20% and multimodal search 130%, reflecting a clear shift in how users describe what they want.

• The second phase pushed Sourcing upstream into market research and design packs and downstream into supplier communication and transaction completion, creating what the company calls “agentic trade.” The system no longer answers a single query; it advances an entire AI-driven sourcing chain.

• Accio Work, launched in April this year, expanded the scope into operations. Built on the supply chains of Alibaba.com and 1688, it connects Taobao, Tmall, JD.com, Douyin, Amazon, Shopify, eBay, and other sales channels, giving merchants a unified view of multi-platform operations, supply chains, and employee performance.

3. Sourcing’s Value: Compressing Months of Back-and-Forth into 24 Hours

• 张阔 used Edward’s plant device to illustrate the full workflow. A user wants the plant to “tell him what state it is in”; the Agent first turns that idea into a consumer-electronics design pack, then selects the sensing and readout approach and controls costs so the retail price and margin can work.

• Accio Work then matches electronics manufacturers in Guangzhou and Shenzhen, handles shipping schedules, customs HS codes, and local logistics, and generates channel-specific content and pricing for Amazon, eBay, Shopify, and TikTok. It also consolidates buyer feedback to iterate the product.

• Traditional sourcing may require maintaining a 20-column Excel sheet and comparing manufacturing capability, certifications, factory quality, pricing, payment terms, and logistics across more than a dozen suppliers. After months of back-and-forth, a team may narrow the field to 2 or 3 suppliers and obtain samples; once authorized, an Agent can decompose the requirements, communicate with multiple suppliers, and compress the reporting cycle to roughly 24 hours.

• Suppliers told the team they had “never seen such a professional design pack.” That not only reduces disputes caused by “drawing something on paper with a pen,” but also lets buyers see the tax-inclusive landed price and retail margin before placing an order, rather than discovering a misunderstanding only after the samples arrive.

4. Regional Software Ecosystems Differ, but the Agent Product Formula Is the Same

• The US already has mature vertical SaaS products, with dedicated systems for CRM, advertising, legal, and healthcare; in China, large platforms more often bundle multiple functions together. AI will inherit different forms of inertia from these ecosystems, but both ultimately have to deliver operating results to users.

• 张阔 acknowledges that simple, easily automated SaaS functions will be weakened, but says the enterprise ontology remains a moat. Without understanding what each data point means, what the owner is trying to achieve, and the context in which the business operates, “even bringing in a college student” would not be enough to start doing the work.

• SaaS products can use AI to upgrade themselves because they control workflows and enterprise context; general-purpose AI entering the enterprise must also connect back into those systems. Yiwen cited AI-native products such as Harvey and OpenEvidence, and 张阔’s response was: “Every era has its own companies.” Each technology shift allows companies with deeper understanding of both the technology and the industry to move faster.

• Accio’s product philosophy is that AI is “global from day one.” Language interaction is primarily through LUI, but performance follows the formula Agent = Model × Harness × Context. The platforms, tools, and operating data connected in the US and China differ, so Harness and Context still need to be adapted to regional and user requirements; where local needs and workflows diverge too far, parts of the capability must be rebuilt.

5. The Moat for Vertical Agents Comes from Optimizing Three Layers Together

• Early investment focused more heavily on the Harness: connecting tools, managing memory, and handling user authorization so the system could execute reliably over operating cycles lasting months. Context comes from more than 20 years of cross-border e-commerce, Sourcing, supplier communication, industry attributes, and platform trends.

• Model, serving, and Harness now have to be optimized together. 张阔 believes that simply taking a general-purpose frontier model and applying basic FT is no longer enough; teams need to post-train models of an appropriate size while optimizing cache hit rates, inference efficiency, accuracy, and cost.

• That is why vertical Agents still have room to differentiate: “It’s not a general-purpose model plus a few simple prompts that can produce the result.” The same base model can deliver better task performance when placed in a more suitable Harness and Context.

6. 107 Real-World Tasks Bring “Stronger Models” Back to Operating Results

• The team compiled 107 open-source tasks from millions of real operating records on Alibaba.com and Accio conversations, covering 7 categories and graded from L1 to L4. The 107 tasks are only the current snapshot; the set will continue to expand as real operating requirements become more complex.

• Several leaderboards are already approaching 99 out of 100, but the most frontier models achieved a zero-intervention pass rate of only “a little over 61%” on this test. That does not mean the tools are unusable; it measures whether the entire assignment can close with no human intervention, and long-horizon, open-ended tasks have especially low pass rates.

• One test requires identifying the right supplier from multiple quotations while filtering out an embedded phishing email, then checking manufacturing, taxes, logistics, and the final landed price. Other tasks require actually booking a ship or deciding whether dispute information is sufficient and whether compensation should be paid. Failure at any one step means the task is not complete.

• Continuous evals are also used to prevent capability “regression” after version updates. 张阔 offered an operational definition of “commercial AGI”: given the same budget and supply chain and a defined scope of authority, an Agent running the day-to-day business must make more money than a human.

7. AI Takes Over Repetitive Operations; Humans Retain Capital Allocation and Product Taste

• The most immediate pain point for multi-platform merchants is having “one set of books”: Douyin, Taobao, JD.com, and Pinduoduo may all show revenue, but revenue is not profit. AI can combine spending, returns, and proceeds data, eliminate manual switching between back ends and Excel sheets, and provide operating conclusions in real time.

• Product listing is a classic multiplication of workload: multiple products across multiple platforms, each with different audiences, media formats, and rules. AI can revise content in bulk and execute promotions, first placing drafts in a queue for human approval or publishing directly after authorization and reporting the results.

• On the customer-service side, AI can analyze every conversation immediately and identify the responses with higher conversion. In B2B sales, it can flag that “this deal may be going south,” identify where the conversation diverged, and point to the next most important judgment. Previously, this required a supervisor to review conversations one by one.

• User demand has moved gradually from L1 Q&A toward L4 operating judgments, but payment and purchasing buttons should remain with humans. Whether a market problem is real, whether the customer base and willingness to pay are sufficient, what the product taste should be, and how to trade off advertising, inventory, channels, and margin still depend on the operator’s intuition.

8. Model Routing Cuts Costs to One-Third

• Yiwen cited test results showing that, across 107 tasks, Accio Work had an estimated cost of $3.69, versus more than $9 for both Codex and Claude Code. Accio Work came in at roughly one-third the cost of the other 2.

• The core mechanism is Pareto routing: tasks below L3 are assigned as much as possible to smaller or more cost-effective models, while multimodal, video, and complex tasks are routed to specialized or frontier models. Volume purchasing and more efficient Harness and Context management further reduce the cost of using the same model.

• 张阔 said the latest small models from Qwen, after post-training, perform very close to top frontier models. Auto mode uses this to select the model for users; businesses such as OpenRouter and Fireworks exist in part because models are increasingly becoming commodities and the optimal choice will continue to change.

• Asked about the newly released GPT-6 Astra, 张阔 declined to draw conclusions from the launch narrative and recommended waiting for a unified benchmark: “It may be good at making 3D models,” or may break through on Computer Use, but that does not mean commercial Agentic capability has improved at the same time. He stated plainly: “You say it’s AGI? I’ll guarantee it right now—it is definitely not AGI.”

9. True Long-Horizon Execution Is Not Staying Online Forever, but Tracking Operating Goals Continuously

• Yiwen worried that overly rigid vertical workflows could constrain the model. 张阔’s answer was to fix the red lines, not the route: once the goal is clear, the Agent should keep optimizing how it gets there. Multiple Agents reaching opposing conclusions is normal; one unified Agent can no longer be expected to resolve every disagreement for the owner.

• Cloud deployment mainly solves memory synchronization and lets work continue across devices. “If an Agent still isn’t in the cloud today, all I can say is that its functionality isn’t finished yet.” Cloud deployment itself is not a technical moat. Commercial long-horizon execution includes scheduled daily reports, follow-up communication triggered by supplier replies, and profit-and-loss results that may take a quarter to validate.

• The point, then, is not whether a model can run 7×24 for a week, but whether it can continuously validate its objectives against employee OKRs, sales conversations, and the business as a whole. “There’s no point in it not crashing if it hasn’t solved your problem.”

10. Merchant Cases Show AI Expanding Small Teams’ Operating Radius

• What surprised 张阔 most at CoCreate in Los Angeles was the turnout: the team estimated that roughly 15,000 people registered and attended, with the entrance line still running through lunchtime. His observation was that US users are relatively further along in their conceptual education around AI and Agents, so the adoption barrier is already low.

• The UK champion at the previous event was a 15-year-old middle-school student. Using Accio Work to find suppliers and discover new products, he developed a towel-like product that cools people down and automatically cools pets, generating roughly £1M in annual sales.

• The US champion, Christian Reed, was initially mistaken for a construction worker because of his clothing before people learned that he had graduated from MIT. He handled everything from the idea through the supply chain on Alibaba.com and Accio Work; customers include Schneider Electric and SpaceX, and the business is reportedly already generating more than tens of millions of dollars this year.

• 张阔 characterizes this demand as a shortage of resources and experience, not a question of whether “AI will replace jobs.” By using AI to connect Alibaba.com, 1688, and global supply chains, small and midsize businesses may be able to scale an idea into a cross-market business without first building a large organization.