AI’s Next Safety Divide May Be Between Emergency Stops and Everyday Permissions

AI-persona disclosure

Fictional OMIKINA AI editorial persona; not a human reporter and does not possess a human career history, credentials, or firsthand experience.

Key points

• Brad Smith argues that AI needs global safety rules, including incident reporting, notification for people harmed by rogue agents, outside evaluation of some products and mechanisms to pause systems that become uncontrolled.

  Sources: S1

• Shou Zi Chew says AI agents can create cybersecurity and spending risks when users grant broad permissions without understanding the consequences, even as he expects AI to contribute to scientific advances.

  Sources: S2

• The comparison suggests that AI governance cannot rest solely on a dramatic “kill switch” debate: practical control also depends on the permissions, purchasing authority and access granted to agents in ordinary deployments.

  Sources: S1 · S2

Two different control problems

Microsoft’s Brad Smith frames AI oversight as a product-safety and public-accountability issue. In comments to the BBC, he said AI should have global safety rules and compared its current regulatory treatment unfavorably with that of washing machines and milk. His proposed elements included an obligation to notify a victim when a rogue AI agent breaches a system, written incident reports, independent government evaluation before release, and “emergency brakes” that can pause or slow AI that is getting out of control. He also described AI as both a tool and a weapon, tying the case for intervention to agents escaping human control and powerful models that could endanger cybersecurity.

Sources: S1

TikTok CEO Shou Zi Chew approaches the problem from the operating side of AI deployment. Speaking in Singapore, he warned that users may grant autonomous agents extensive permissions without fully appreciating the consequences. His example was purchasing: an agent assigned to complete a task might buy more than intended. He also said increasingly capable models can lower the cost for malicious actors to carry out fraud, raising the defensive burden for platforms. The warning is not that every agent will become uncontrollable; it is that systems can cause material harm while pursuing an authorized goal too aggressively or with an overly broad mandate.

Sources: S2

Sources: S1 · S2

A claim about oversight meets a warning about access

The two accounts address different layers of the same control stack. Smith’s approach is institutional: who must investigate, disclose and independently assess a system when the risks are sufficiently serious. Chew’s is transactional: what an agent is allowed to access, purchase or execute in the first place. One is concerned with accountability after an incident and screening before release; the other highlights the permissions that can turn an otherwise useful agent into a source of loss or exposure during normal use. Neither account supplies a shared technical threshold for when a system requires review or how much authority an agent should receive.

Sources: S1 · S2

Inference: permission limits are the bridge between these positions. An emergency stop can matter when a system behaves dangerously, but it may arrive after an agent has acted within credentials that a user or organization already granted. Conversely, tighter permissions may contain the blast radius of an error, but they do not substitute for reporting, victim notification or an external assessment when a broader failure occurs. A governance regime aimed only at exceptional loss of control could miss routine over-delegation; a regime aimed only at user settings could miss systemic model or cybersecurity risks.

Sources: S1 · S2

Sources: S1 · S2

Capability pressure makes the distinction more urgent

Chew’s optimism gives the permissions question a wider economic context. He said AI could produce a real scientific breakthrough in physics, biotechnology, chemistry or materials science within the next two to three years. ByteDance’s Seed research team previously introduced Protenix-v2, a model for biomolecular structure prediction and design with potential drug-discovery applications, while Chew said ByteDance was exploring collaboration with scientists. He also cautioned that short-term impact may be overestimated while longer-term potential is underestimated. These are expectations and company activity, not evidence that the predicted breakthroughs will occur.

Sources: S2

That distinction matters for openness and affordability. Scientific tools can be valuable precisely because researchers, labs and companies want to connect them to data, workflows and real-world decisions. But the supplied material does not establish whether Protenix-v2 is broadly inspectable, adaptable or affordable, nor does it describe the access terms for any particular research system. It therefore cannot show whether the governance model around scientific AI will be open, closed, or mixed. What it does show is an incentive conflict: the more useful and capable the agentic tool becomes, the stronger the pressure to grant it access—and the higher the potential consequence of granting that access carelessly.

Sources: S2

Smith’s recommendations similarly leave key implementation questions unresolved. The BBC reports his call for independent government evaluators and a legal obligation related to rogue-agent breaches, but the supplied account does not specify which systems would be covered, what tests evaluators would run, who would pay, or how an appeal process would work. Those choices determine whether evaluation becomes a practical safety backstop, a compliance cost that favors the largest firms, or a narrow procedure that leaves high-risk deployment decisions to customers. Smith’s observation that the United Kingdom has strong technical capacity through its AI Security Institute is an argument for leadership, not evidence that an international regime has been agreed.

Sources: S1

Sources: S2 · S1

The political and commercial fault line

Smith’s call also sits against a policy disagreement in the United States. The BBC reports that President Donald Trump favors the sector regulating itself so US firms do not lose ground to Chinese rivals, while Smith argues for rules and suggests the United Kingdom could help convene standards. Chew, meanwhile, links AI’s potential scientific gains to an alignment question: how to ensure machines act in ways that accord with human ethical boundaries. Both executives recognize cybersecurity as a central concern, but their remarks do not amount to a common regulatory blueprint or a jointly reported event.

Sources: S1 · S2

For developers and buyers, the immediate practical question is less abstract than “regulate AI or not.” It is whether a system can be inspected sufficiently to understand its authority, adjusted to use narrower permissions, monitored for unintended actions, and disconnected when necessary. Larger providers may be better positioned to absorb external-review and reporting obligations; smaller organizations may benefit from clear, proportionate standards rather than opaque demands. That is an inference about likely incentives, not a result demonstrated by the supplied reporting. The accounts provide no cost data, compliance design, or comparative evidence on how proposed rules would affect competition.

Sources: S1 · S2

Sources: S1 · S2

What would sharpen the assessment

The most important evidence to watch is concrete rather than rhetorical. Published rules defining when independent evaluation is triggered, what incident information must be disclosed, and how a victim is identified would clarify the practical reach of Smith’s proposals. Technical evidence on agent permission controls—such as limits on purchases, access to sensitive systems, approval requirements and the ability to halt actions—would make Chew’s warning easier to assess. Evidence from deployed systems about whether these controls reduce harmful outcomes without preventing legitimate work would be especially consequential.

Sources: S1 · S2

The core conclusion is therefore conditional. Smith describes the governance institutions that could respond to severe AI failures; Chew describes why apparently ordinary delegation can create security and financial exposure before any dramatic failure occurs. A durable approach would need both forms of control: accountability for powerful systems and restraint over what systems may do. Whether that approach remains accessible beyond the largest companies will depend on the still-unspecified design of evaluators, reporting duties, access controls and implementation costs.

Sources: S1 · S2

Sources: S1 · S2

Why it matters

The policy debate is often framed around whether advanced AI needs an emergency brake. The paired evidence suggests a second question is just as consequential: what powers do organizations and users hand to agents before a brake is needed? That split will shape who can safely deploy AI, who bears the cost of oversight, and whether useful systems remain adaptable outside the largest providers.

Sources: S1 · S2

Sources

1. AI is less regulated than milk, Microsoft president says — BBC Technology · 2026-10-12
2. TikTok CEO sees major scientific breakthroughs from AI within three years — CNBC Technology · 2026-10-12