Cloud-led Future · Intelligence-driven Hotline +86 21 6228 0217 中文 | EN

The Verifiability Threshold for Scaling AI Agents - Governance, Identity and Liability

Overview

The 16 November edition of People’s Daily published a signed article calling for agile governance to support agent development. It proposes a tiered, category-based governance framework, exploratory tools such as regulatory sandboxes and trigger-based supervision, filing, testing and recall requirements for sensitive domains, and compliance self-assessment plus platform management for low-risk uses, while stressing intrinsic safety capabilities in agents themselves. This aligns with recent regulatory moves. The Ministry of Industry and Information Technology’s notice on strengthening network-access management for agent-enabled terminal devices continued to draw industry debate this week: each agent must carry a unique traceable identity code, manufacturers must seek approval before remote upgrades that change agent capabilities, and app distribution platforms must complete compatibility and adaptation verification. It is widely read as a framework that embeds agent identity management into hardware network-access approval.

Capability gains and boundary-setting appeared almost simultaneously. Information disclosed on 14 November indicates a Google Research multi-agent collaboration system, combining the Lean 4 formal verifier with multi-agent debate, autonomously completed a full formal proof of an open mathematical conjecture involving roughly 50 million lines of code, and made progress on theoretical computer science problems such as graph-cut sparsification; the proof capability is being made available free to 10,000 researchers. On the other side, United Airlines amended its contract of carriage on 11 November to explicitly prohibit third-party AI agents from creating, modifying or servicing tickets, even under continuous passenger supervision. Agents may still query schedules and prices but cannot complete purchases, with the carrier citing security and liability risks from shared-account logins. It is the first time a major airline has contractually delimited agent behaviour.

Two further governance signals emerged. ASEAN signed its first legally binding digital economy framework agreement, covering cybersecurity, artificial intelligence and cross-border data sharing, along with end-to-end trade and source-code protection. Domestic testing also showed AI-generated fake faces performing nod, blink and head-turn actions and potentially passing facial-recognition checks in some apps, renewing scrutiny of liveness detection and identity verification.

Background & Interpretation

1. Technology Evolution and Context

Agents differ from earlier large models by moving from generating content to executing actions. Once a system can call tools, log into accounts, initiate transactions or alter external state, the risk shifts from output error to permission overreach and liability attribution. The technical response is verifiability: formal verification converts reasoning into machine-checkable proofs, and multi-agent debate coupled with a verifier gives long chains of inference reproducible intermediate artefacts. Evaluation is also absorbing cost as a metric - a frontier reasoning model leads on a mathematics benchmark while its high-reasoning tier costs markedly more per task than comparable systems, exposing a clear trade-off between capability and affordability. The regulatory response is traceability: identity codes, upgrade approvals and distribution-platform verification ensure every capability change and execution maps to an accountable party.

2. Core Drivers and Underlying Mechanisms

The first mechanism is binding permissions to identity. Shared accounts, delegated operation and unattended workflows are long-standing in consumer internet services, but when the executing party becomes software that plans autonomously, the liability chain among account holder, model provider and application operator blurs. A carrier prohibiting third-party booking in its contract is effectively declining to open transaction permissions where liability cannot be assigned. The second is the regulatory cost of capability change. Remote upgrades can materially shift an agent’s boundaries, so without approval and compatibility verification, a device that passed entry testing may behave quite differently in the field; tiered supervision must therefore cover the full lifecycle rather than one-time admission. The third is resource allocation under tiered governance. Applying uniform testing intensity to every agent is impractical, so assigning filing, testing, recall and self-assessment by risk level controls exposure without stifling low-risk iteration.

3. Market Structure and Value-Chain Impact

Value is migrating toward the verification layer. Model vendors must supply auditable outputs, tool-call logs and permission-boundary declarations alongside capability metrics. Device makers need identity codes, upgrade governance and compatibility checks embedded in firmware and release pipelines. Distribution platforms take on adaptation verification and delisting of problematic versions. Insurance, legal and audit services may see new demand around agent liability cover, compliance assessment and incident forensics. For enterprise buyers, procurement and acceptance criteria shift from benchmark scores to auditability: reproducible decision paths, constrained invocation scope, and the ability to identify version and responsible party after an incident. Cross-border operators must also design for regional rule divergence, as the ASEAN provisions on source-code protection and cross-border data illustrate.

Implications & Outlook

In the near term, agent deployment in sensitive scenarios such as transactions, payments and identity verification will proceed more cautiously, while low-risk uses - search, retrieval, assisted analysis - continue to spread quickly, producing a tiered openness of capability. Over the medium term, if identity codes and upgrade approvals coalesce into unified standards, cross-platform agent registries and version-traceability systems may emerge and reshape app distribution. Variables to watch include the formal document and scope of the tiered governance framework, how device-entry rules define remote upgrades, whether formal verification and audit logs become standard acceptance items, and the pace of identity-verification upgrades under pressure from generative forgery. If auditability becomes an entry requirement, smaller developers lacking logging and traceability infrastructure will face higher compliance costs.

Takeaways for Industry Participants

Developers should make least-privilege the default architecture: grant temporary credentials per task, log the full tool-call chain, retain human confirmation for high-risk actions, and keep versions and capability manifests in traceable release records. Platform and device vendors should pre-build identity labelling, upgrade approval and problematic-version recall into engineering workflows rather than retrofitting after rules take effect. Enterprise procurement and risk teams should require reproducible decisions, constrained invocation scope and assignable liability alongside accuracy, and specify data use and liability allocation contractually. Security teams should add generative forgery to the identity-verification threat model, using multi-factor and liveness-detection combinations at critical checkpoints such as facial recognition.

This column compiles industry information and shares technical perspectives; it does not constitute investment advice.