Overview
During the 2026 New-Domain New-Quality Innovation Contest, the China Center for Information Industry Development (CCID) released its “Top Ten Trends,” noting that by June 2026 China’s daily token call volume had surpassed 175 trillion — over 1,000 times the level at the start of 2024. Crucially, China’s AI inference data volume has exceeded training data volume for the first time, signaling that large-model applications have formally moved from a “training-led” to an “inference-led” phase. The report also shows top model iteration cycles have shortened from months to weeks or even days.
The inflection point means the structure of compute demand is fundamentally changing: concurrency, latency and cost on the inference side are replacing peak training compute as the new battleground.
Background & Interpretation
1. Technical Background and Evolution
Large-model development has moved through three stages: pretraining scaling, instruction tuning, and inference scale-up. From 2023–2024 competition centered on parameter scale and training clusters. Entering 2026, with open-source downloads and derivative apps surging, model capability truly reached application layers across industries, and inference requests exploded exponentially, shifting the data center of gravity from training to inference.
2. Core Drivers and Mechanisms
Inference overtaking training rests on three drivers: first, the normalization of agents, whose multi-step reasoning and tool calls vastly extend token consumption per task; second, improved edge/device models pushing inference from cloud to devices, multiplying call frequency; third, the user-scale expansion of AI-native apps (search, office, customer service, coding). Mechanically, inference’s demand for “high concurrency, low latency, low cost” forces compute architectures from training-first to training-inference integration and inference-specific designs.
3. Market Structure and Chain Effects
In an inference-led era, inference chips, inference engines, model compression and scheduling platforms gain prominence. Products like Alibaba’s Zhenwu M890 super-node and Iluvatar’s TY1200 edge terminal reflect “cloud-edge-device layered compute” becoming the mainstream paradigm. Meanwhile, export controls spreading downstream in the compute supply chain raise the need for multi-cloud and domestic-chip backup plans.
Implications & Outlook
Near term, inference demand stays buoyant, lifting investment in inference chips, high-speed interconnect and liquid cooling. Medium term, model cycles compressed to weeks accelerate application innovation and industry churn. Watch: the speed of inference token-cost declines, edge-model penetration, and AI-Agent monetization willingness.
Takeaways for Industry Participants
Compute and chip vendors should rebuild roadmaps around “high-concurrency inference, low latency, low power” and prioritize edge/device layouts; app developers can embed AI into core business flows as inference costs fall, not just chat entrances; enterprise IT should evaluate hybrid “cloud-train, edge-infer” deployment to balance data security and cost.
This column compiles industry information and shares technical perspectives; it does not constitute investment advice.