Overview
On November 2, Qualcomm announced its first-generation High-Bandwidth Computing (HBC) chip has passed hardware validation, successfully testing full memory read/write paths and near-memory compute capability. By placing compute close to memory, the technology cuts data movement to raise bandwidth and energy efficiency while lowering AI inference power and cost. A datacenter product has been taped out; the HBC-based Dragonfly AI250 is expected to sample in H1 2027.
In the same window, domestic dedicated inference silicon drew capital: on November 3, Mojin AI closed a nearly 1 billion yuan Series C+ round to upgrade sparse dedicated inference chips and pre-research next-gen parts. Xiaomi’s humanoid robots and Ant’s LingBot embodied agents also underscore inference-side demand.
Background & Interpretation
1. Technical Background and Evolution
Inference cost and energy have long been capped by the memory wall—frequent data shuttling between compute and storage. Near-memory computing and HBM are the two fronts against it. Qualcomm’s HBC validation marks near-memory architecture moving from papers to tape-out-ready engineering.
2. Core Drivers and Underlying Mechanism
Inference stresses per-token cost and latency more than training. As edge, on-device and embodied-AI real-time inference surges, dedicated inference chips (sparse, near-memory) become a new track after training GPUs, via superior efficiency. Mojin’s sparse route and Qualcomm’s near-memory route represent algorithm sparsification versus architecture near-memory cost-cut paths.
3. Market Structure and Industry Chain Impact
Inference-chip demand spills into the broader electronics chain: a November 6 report noted AI-driven tightness in power semiconductors and chips used in PV inverters and storage converters, lifting prices. AI compute expansion thus transmits through chip costs back into solar-storage equipment pricing—compute and energy couple at the silicon layer.
Implications & Outlook
If near-memory computing scales, it could sharply cut per-token inference cost and accelerate AI’s move from cloud to edge. Watch HBC sampling pace, sparse-chip yield, and HBM and advanced-packaging supply.
Takeaways for Industry Participants
For chip and hardware firms: build differentiated inference routes (near-memory, sparse) around efficiency, avoiding homogeneous training-GPU competition. For application firms: watch the edge-AI deployment window as inference costs fall. For investors: track inference chips moving from proof-of-concept to volume shipment.
This column compiles industry information and shares technical perspectives; it does not constitute investment advice.