When an AI Agent Acts Without Approval: Security Lessons for Enterprise Autonomy

An agent acting without approval is an authority failure

Public incidents in which an autonomous agent deletes, publishes, sends, purchases, changes, or executes more than its operator expected are often described as the model going rogue. That language is dramatic but operationally weak. The important fact is that a probabilistic component received enough access to create an effect without a dependable system boundary enforcing intent.

The model may have lost an instruction, misunderstood scope, followed malicious content, selected the wrong tool, repeated a call, or optimized the stated goal too literally. Those are different causes. None should be the final barrier between a suggestion and an irreversible action. A secure architecture assumes interpretation can vary and enforces authority again at the action boundary.

A prompt that says ask before acting is a behavioral request. An approval service that requires a valid human decision bound to exact parameters is a control. A prompt that says never reveal secrets is guidance. A tool gateway that never exposes secrets to the model and blocks unauthorized egress is a control. Prompts remain useful, but they cannot replace mechanisms outside the model's discretion.

Bizz cybersecurity engineering treats agents as operational workloads that read untrusted data, hold delegated authority, and produce real effects. The design objective is not perfect obedience. It is a small blast radius, visible decisions, safe failure, and recovery from known state.

• Intent failure: the system misunderstood or lost what the operator wanted.
• Authorization failure: the agent could invoke an effect outside verified authority.
• Approval failure: confirmation was absent, vague, stale, or not bound to parameters.
• Containment failure: one mistake reached too much data or too many systems.
• Recovery failure: operators could not stop, reconcile, reverse, or explain the effects.

Reconstruct an incident as a chain of trust decisions

A useful incident timeline begins before the visible action. How was the agent installed and registered? Which user and workload identities did it use? Which tools, skills, plugins, files, channels, and networks could it reach? Which content entered context? What plan was proposed? What policy and approval checks occurred? Which API accepted the effect? What evidence was retained?

Separate observations from theories. The fact that an agent continued after a stop message does not prove why it ignored that message. Context compression, race conditions, queued tasks, tool latency, duplicate execution, stale state, or a product defect may produce similar symptoms. Preserve traces, state, model and prompt versions, tool requests, receipts, and timestamps before drawing a root cause.

Identify the first preventable control failure and every later containment opportunity. A malicious document may initiate the chain, but a broad credential, unrestricted tool, missing approval, unsafe default, and absent egress restriction may each have allowed impact to grow. Fixing only the first prompt leaves the rest of the path exposed.

Map effects to business objects. Which messages were sent, records changed, files deleted, accounts accessed, credentials exposed, code executed, or external recipients contacted? Outcome-unknown operations need reconciliation with downstream systems. A model transcript is not proof that an action did or did not occur.

• Capture installation, identity, configuration, data, model, prompt, tools, and network context.
• Distinguish confirmed events, operator reports, system receipts, and hypotheses.
• Find every control that could have prevented or contained the effect.
• Reconcile downstream state using stable request and object identifiers.
• Assess people, privacy, legal, financial, customer, and operational impact.

Threat-model the combination of private data, untrusted input, and external effects

An agent becomes especially risky when it can read sensitive data, process content controlled by outsiders, and communicate or act outside its trust boundary. Email assistants, browser agents, coding agents, procurement agents, and support agents routinely combine these capabilities. An attacker can place instructions in the content the agent is expected to inspect.

List assets and trust boundaries. Assets include customer and employee data, credentials, source code, contracts, messages, cloud resources, money, reputation, and the integrity of business records. Trust boundaries include user to agent, content to model, model to tool, tool to network, plugin to host, memory to future session, and one agent to another.

Model misuse by authorized users as well as external attack. An employee may ask an agent to export data they can view but are not allowed to redistribute. A contractor may connect a personal agent to company systems. A service account may accumulate access beyond the human who initiated the task. Security must preserve policy through delegation.

The OWASP AI Agent Security Cheat Sheet provides a useful catalog including prompt injection, tool abuse, memory poisoning, identity and privilege abuse, and unbounded resource consumption. Use catalogs to seed scenarios, then adapt them to the actual agent topology, data, tools, and business consequences.

• Assets: data confidentiality, record integrity, availability, money, code, credentials, and reputation.
• Inputs: users, web pages, email, files, APIs, tools, memory, plugins, and peer agents.
• Effects: messages, publications, transactions, record changes, file operations, and code execution.
• Actors: external attackers, insiders, compromised vendors, malicious content authors, and mistaken operators.
• Boundaries: identity, tenant, device, host, network, data domain, tool, and action authority.

Inventory agents by capability, not by product name

A software inventory that records only an agent framework or vendor does not reveal the risk. Two installations of the same tool can differ completely: one summarizes public documents in a sandbox, while another runs continuously on an employee workstation with email, browser, shell, cloud, and credential access.

Create an agent registry with owner, business purpose, users, environment, data classes, model providers, memory, tools, plugins, credentials, network paths, action tiers, human decisions, logs, retention, tests, incidents, and lifecycle state. Discover unsanctioned agents through identity, OAuth grants, API keys, endpoint telemetry, browser extensions, package inventories, and network behavior.

Classify effective capability. Can the agent only read? Can it draft? Can it stage a change? Can it execute? Is the action reversible? What is the maximum object count, value, audience, or environment? Does it operate interactively, on a schedule, from events, or continuously? Broad background authority deserves a different review from one foreground task.

Bizz enterprise software engineering can connect this registry to identity governance, configuration management, service ownership, incident response, and change control. Inventory is valuable when it drives decisions, not when it becomes another static spreadsheet.

• Ownership and purpose: accountable business and technical owners plus approved outcome.
• Reach: users, tenants, data classes, systems, tools, networks, plugins, and providers.
• Authority: read, draft, stage, execute, approve, delegate, schedule, and publish.
• Controls: identity, sandbox, approval, limits, monitoring, tests, and recovery.
• Lifecycle: proposed, test, restricted, production, suspended, retired, or orphaned.

Least privilege must reach the final API

A user may have broad access for many purposes, but an agent should receive only the capability needed for the current task. Copying the user's session cookie, cloud credentials, or full OAuth grant into an autonomous runtime lets a reasoning error inherit the user's maximum reach.

Use separate workload identity and explicit delegation. The downstream API should know the human subject, agent workload, tenant, purpose, requested capability, resource scope, and assurance level. Credentials should be short lived, audience restricted, and issued at execution time. The model should never see reusable secrets.

Scope at the operation and object level. An email-cleanup assistant may retrieve message metadata from one folder and create a proposed batch. It should not automatically receive permission to delete every mailbox item. A deployment assistant can inspect one service and prepare a rollout without holding organization-wide administrator access.

Enforce policy in the tool gateway and business service. The agent framework's allowlist is useful but not sufficient if a direct API accepts broad credentials. Rate, value, count, recipient, environment, and time limits should remain effective even if the model calls the tool with valid syntax.

• Separate user identity, workload identity, and delegated authority.
• Issue short-lived capability access for one audience, purpose, and resource scope.
• Keep credentials out of prompts, memory, code, files, and tool output.
• Revalidate identity and policy inside every effectful service.
• Limit value, volume, recipients, environment, frequency, and duration.

Separate read, propose, stage, approve, and execute

A natural-language request often combines several operations: inspect records, decide what qualifies, prepare changes, and commit them. If one tool call performs all of those, there is no stable boundary where the system can validate the selection or ask the user what they intended.

Create distinct capabilities. Read operations return source-linked data. Proposal operations produce a typed change set with reasons and warnings. Staging reserves or quarantines without final effect. Approval binds an authorized decision to the exact change set. Execution validates current state and commits once. Verification reads downstream truth and records a receipt.

The model may help interpret and propose, but deterministic code should calculate counts, validate objects, enforce policy, and compare the approved payload with the execution request. A changed plan needs fresh approval. A generic yes from several turns earlier should not authorize newly added recipients, files, amounts, or commands.

This design also improves usability. People can inspect what will happen in a form, diff, list, or preview instead of interpreting a paragraph. They can remove one item, change scope, or choose a reversible option. Bizz API engineering turns broad system access into these narrow, typed capabilities.

• Read: retrieve permitted state without side effects.
• Propose: create a typed, source-linked change set.
• Stage: place changes in a reversible or isolated intermediate state.
• Approve: bind identity and intent to exact parameters and expiry.
• Execute and verify: commit idempotently, then read the authoritative result.

Approval is a security protocol, not a conversational courtesy

A secure approval identifies the approver, verifies their authority, presents the exact operation in understandable terms, captures a deliberate decision, binds that decision to an immutable payload, expires it, and prevents replay. The service verifies all of those properties before execution.

Show object count, representative details, total value, recipients, environment, reversibility, and important consequences. For large batches, provide downloadable or inspectable detail and a risk-based review path. Do not let the agent title the confirmation in a misleading way or hide dangerous parameters in collapsed text.

Protect the approval interface from model-controlled content. An attacker can attempt to forge or socially engineer a human-in-the-loop dialog through tool output or retrieved text. Use trusted UI chrome, separate untrusted content, sanitize rendering, and fetch final parameters from the workflow state rather than from the agent's prose.

Account for time-of-check to time-of-use. Revalidate current state immediately before execution. If recipients, amounts, permissions, policy, or selected objects changed, invalidate or narrow the approval. High-risk operations may require step-up authentication, dual control, waiting periods, or an independent role.

• Authenticate the approver and verify authority for the exact action.
• Present scope, objects, value, audience, environment, and reversibility clearly.
• Bind approval to an immutable payload hash, intent ID, and expiry.
• Render confirmation in a trusted interface outside model-controlled text.
• Recheck state and policy before commit; require fresh approval after material change.

Reversibility should be the default path for uncertain decisions

Many actions described as delete can be implemented as move to quarantine, mark for review, revoke visibility, or schedule deletion after a recovery window. Publishing can begin as a private draft. Code changes can begin on a branch. Infrastructure changes can begin as a plan and canary. Payment or access changes can use holds, limits, or staged approval where the domain permits.

Reversibility does not eliminate the need for authorization. Moving sensitive records to the wrong quarantine can still leak data, and a draft can still contain confidential content. It reduces consequence while the system gains evidence and gives operators a recovery path.

Define undo semantics before launch. What identifiers link the original and compensating action? How long is recovery available? Which metadata and contents survive? Who may restore? What happens if another process modifies the object? A button labeled undo is not reliable if the underlying service cannot reconstruct state.

Some effects cannot be reversed: an external message read by a recipient, a disclosed secret, a public publication copied elsewhere, physical equipment movement, or a financial market action. These require stronger prevention and may justify keeping AI at recommendation or preparation rather than execution.

• Prefer quarantine, draft, branch, preview, canary, hold, or delayed commit.
• Keep authorization and data boundaries in reversible paths.
• Record original and compensating action IDs with state snapshots.
• Test restoration after concurrent changes and partial failure.
• Reduce autonomy where effects are public, physical, financial, secret-bearing, or irreversible.

Safety policy must survive context truncation and model replacement

Instructions inside the model context can be truncated, summarized, contradicted, or interpreted differently after a model or prompt update. Critical policy should therefore exist in executable controls outside the context window. The orchestrator and action service know which state, capability, approval, and limits apply even if the model forgets the conversation.

Keep invariant rules in code or policy engines: prohibited tools, data boundaries, maximum batch size, allowed destinations, required approval, environment restrictions, and step-up conditions. The agent receives enough policy to plan and explain, but cannot waive the enforcement mechanism.

Store current workflow state structurally. Pending approval, selected objects, attempted actions, receipts, and cancellation should not rely on a rolling conversation summary. When a session resumes or context is compacted, rebuild the model view from canonical state and current permissions.

Test long contexts intentionally. Fill the session with unrelated content, trigger summarization, switch channels, resume after delay, and change model versions. Verify that control decisions remain identical even if generated wording changes. A policy that works only in short demonstrations is not a production boundary.

• Enforce critical policy outside model context and prompts.
• Persist task, approval, cancellation, action, and receipt state structurally.
• Rebuild context from canonical state after truncation or resume.
• Version models and prompts without changing authorization semantics.
• Test long sessions, summaries, channel changes, and model substitutions.

Untrusted content must remain data even when it sounds like an instruction

Agents routinely read email, web pages, tickets, documents, code comments, calendar invitations, and tool output. Any of these can contain text that tells the agent to ignore prior rules, disclose data, fetch another URL, invoke a command, or write a durable memory. The content's grammatical form does not grant authority.

Separate content channels and label trust. Tool definitions and system policy come from controlled configuration. Retrieved content arrives as quoted data with source and sensitivity. The model can summarize or classify it, but the runtime should not dynamically create tools, destinations, or credentials from those words.

Use allowlisted fetchers, content-size limits, MIME validation, parser isolation, URL and redirect policy, malware scanning, output encoding, and egress controls. For high-risk tasks, use a separate model or process to extract narrowly typed facts from untrusted content before the planning agent sees them.

Prompt-injection defenses are layered and imperfect. Detection can help triage, but attackers can vary language and encoding. The durable defense is that a compromised interpretation still reaches tools with narrow authority, mandatory validation, and visible approval for consequence.

• Label system instructions, user requests, retrieved evidence, and tool output distinctly.
• Treat instruction-like text in external content as untrusted data.
• Restrict fetchers, parsers, URLs, redirects, file types, sizes, and egress.
• Use typed extraction and isolation for high-risk content.
• Assume detection can fail and contain the resulting tool authority.

Memory can turn one malicious input into a delayed incident

An attacker may plant a preference, rule, credential location, false identity, malicious URL, or operational instruction that the agent retrieves in a later session. The later task may have higher privileges than the original interaction. This delayed influence makes memory poisoning a distinct operational risk.

Allow only defined memory schemas and sources. A user can explicitly author low-risk preferences. Business facts come from authoritative systems. Procedure changes follow release control. Agent extraction creates candidates with evidence and status, not immediate global truth. Shared memory requires stronger review than personal session continuity.

Filter memory by tenant, subject, purpose, application, trust, status, and time before semantic ranking. Keep untrusted content out of instruction priority. Log which memories entered context and which influenced an action. Support quarantine and downstream impact analysis through stable memory IDs.

Red-team across time: submit malicious content in one session, trigger background consolidation, then start a different task, channel, agent, or user. Test whether deletion removes derived embeddings, summaries, caches, and shared copies.

• Restrict memory writes by schema, source, subject, purpose, and trust.
• Stage inferred memories for validation rather than activating them immediately.
• Keep procedure and authorization outside user-controlled memory.
• Trace memory IDs into context, decisions, and effects.
• Test delayed, cross-session, cross-agent, and shared-memory poisoning.

Plugins and skills are executable supply-chain dependencies

Agent ecosystems make capability easy to install through skills, plugins, packages, tool servers, browser extensions, scripts, and templates. A description that says summarize files may hide code that reads environment variables, modifies prompts, contacts an external host, or executes a shell. Popularity and a polished listing are not security review.

Use an allowlisted internal catalog with verified publisher identity, source review, dependency scanning, signatures or checksums, pinned versions, declared permissions, provenance, and an owner. Install into a test environment first. Treat natural-language instructions bundled with a skill as code that can alter behavior.

Grant capabilities at runtime rather than at installation. A plugin may be present but unable to read files, network, secrets, or customer data until a policy-authorized task requires it. Run third-party code in a sandbox with read-only inputs, constrained output, resource limits, and a default-deny network.

Plan revocation. The registry should identify every agent and workflow using a vulnerable component. A kill switch can disable the skill, revoke credentials, quarantine outputs, and block its destinations while preserving evidence. Updating one package is incomplete if malicious memory or artifacts remain.

• Verify publisher, source, dependencies, version, integrity, permissions, and ownership.
• Pin and scan components; test them in an isolated environment before approval.
• Treat bundled prompts, templates, scripts, and configuration as executable behavior.
• Grant task-scoped runtime access instead of permanent installation privileges.
• Maintain dependency-to-agent lineage for rapid disablement and remediation.

Sandbox the runtime and broker every route to the outside world

An agent that can execute code on an employee workstation inherits files, network trust, developer credentials, SSH configuration, browser sessions, and local services unless isolation removes them. Containers help but are not a complete boundary if they mount sensitive directories, run privileged, share host sockets, or allow unrestricted egress.

Use workload isolation appropriate to consequence: dedicated containers, microVMs, disposable desktops, restricted browser profiles, or isolated cloud accounts. Mount only task inputs, prefer read-only filesystems, drop privileges, constrain system calls, limit CPU, memory, time, and storage, and destroy the environment after the run.

Broker network access through a policy-enforcing proxy. Allow required domains and methods, inspect redirects and DNS behavior, restrict private address ranges, cap response size, and log destinations without exposing sensitive payloads broadly. External communication should use typed services rather than arbitrary HTTP wherever possible.

Inject secrets only into the operation that needs them and keep them non-exportable where possible. Rotate credentials after suspected compromise and examine downstream use. Bizz cloud application engineering can build these runtime, network, secret, and tenancy controls into the product rather than relying on every employee's local configuration.

• Isolate compute, filesystem, browser, identity, cloud account, and network by task risk.
• Use read-only inputs, dropped privileges, resource limits, and disposable environments.
• Default-deny egress and broker approved destinations, protocols, and volumes.
• Inject short-lived secrets at the narrowest execution boundary.
• Test escape, host access, credential theft, internal scanning, and data exfiltration.

Observability must connect intent to every real-world effect

A transcript shows generated language, not necessarily system truth. Incident responders need the initiating identity and request, workflow state, model and prompt versions, retrieved sources, memory IDs, tool selection, authorization decisions, approval artifact, exact parameters, attempt IDs, downstream receipts, and verified final state.

Use distributed traces and an append-only action ledger. Every effect has a stable business-intent ID and idempotency key. Unknown outcomes remain visible until reconciled. Cancellation, human intervention, policy denial, retry, compensation, and correction are events, not overwritten status text.

Alert on abnormal behavior: new destinations, broad object enumeration, rising deletion or send volume, repeated denials, unusual tool combinations, long loops, budget spikes, cross-tenant attempts, memory writes to sensitive namespaces, or actions outside normal hours. Baselines should be role and task aware.

Protect telemetry. Prompts and tool output may contain secrets and personal data. Minimize, tokenize, redact, encrypt, restrict support access, and set retention by purpose. Keep enough structured evidence to investigate without turning observability into a second unrestricted copy of the enterprise.

• Trace subject, workload, task, context, policy, approval, tool, attempt, receipt, and outcome.
• Maintain an append-only action ledger with intent and idempotency identifiers.
• Detect unusual scope, volume, destinations, denials, loops, and costs.
• Reconcile unknown outcomes against downstream systems.
• Minimize and protect telemetry while preserving causal evidence.

AI incident response needs more than disabling the model endpoint

An agent incident may continue through queued tasks, scheduled jobs, plugins, stolen credentials, persisted memory, copied artifacts, or downstream automations even after model access stops. Containment must address the whole execution graph. Preserve evidence before volatile state disappears.

Prepare controls to pause a run, stop new scheduling, disable a tool, revoke a plugin, rotate credentials, block egress, quarantine memory, freeze a workspace, and switch the product to read-only or human-only mode. Test who can invoke these controls and whether they work during provider and identity outages.

Eradication removes the cause and durable influence: vulnerable code, unsafe configuration, malicious memory, untrusted artifacts, excessive grants, compromised tokens, and dependent deployments. Recovery reconciles every effect, restores from authoritative state, compensates where possible, validates policy, and returns traffic gradually.

Notification and remediation depend on impact. Security, privacy, legal, compliance, business, communications, HR, and customer support may need defined roles. A retrospective should identify technical and organizational control gaps, add regression tests, and reconsider the agent's authority rather than merely rewriting its prompt.

• Detect and triage through agent, identity, endpoint, network, data, and application signals.
• Contain runs, schedules, tools, credentials, egress, memory, and dependent automation.
• Eradicate vulnerable code, malicious state, unsafe grants, and compromised artifacts.
• Reconcile and remediate every actual or unknown business effect.
• Recover gradually with tests, monitoring, ownership, and an autonomy review.

Red-team the full action path and the human controls

Model-only testing can reveal direct prompt injection but misses identity, tools, plugins, network, memory, approval UI, retries, and downstream APIs. Build adversarial scenarios across the complete journey. Use realistic permissions and faithful test systems so the team can observe whether a harmful interpretation becomes a real effect.

Test hidden instructions in web pages, email, documents, images, code comments, tool output, and memory. Test data exfiltration through URLs, messages, logs, filenames, DNS, and error reporting. Test attempts to enumerate data, expand scope, install a skill, obtain secrets, forge approval, or convince another agent that authorization already occurred.

Inject operational failures: context truncation, model switch, delayed stop, duplicate messages, timeout after successful action, stale approval, changed permissions, partial batch success, crashed worker, and unavailable policy service. Verify default-deny behavior, idempotency, reconciliation, compensation, and operator visibility.

Include people. Can an employee understand the approval? Can an attacker make a trusted dialog appear to approve something else? Can support safely recover without bypassing controls? Bizz quality engineering can turn these tests into repeatable pre-release gates, continuous exercises, and incident drills.

• Direct and indirect prompt injection across every content channel.
• Tool misuse, privilege expansion, exfiltration, plugin abuse, and memory poisoning.
• Approval forgery, misleading summaries, stale consent, and human fatigue.
• Timeouts, duplicates, partial success, crashes, truncation, and provider changes.
• Containment, kill switch, reconciliation, compensation, and communication drills.

Governance should scale authority through evidence

A review committee cannot inspect every agent decision, and a platform checkbox cannot understand every business consequence. Governance should establish common controls and require each use case to define its identity, data, tools, actions, human decisions, tests, monitoring, owner, and recovery.

Tier actions by consequence and reversibility. Public information retrieval may need standard controls. Access to confidential records needs stronger identity and logging. Drafting an external message needs content and recipient policy. Sending, deleting, purchasing, changing access, executing code, or modifying production requires progressively stronger approval and containment.

Use evidence gates. A use case moves from read to propose, stage, and execute only after representative and adversarial evaluation, operational readiness, incident exercises, and a controlled production cohort meet thresholds. Expansion is a new decision when data, tools, user groups, volume, or effect changes.

Measure severe outcomes, not only adoption and containment. Track unauthorized or incorrect effects, blocked attacks, human corrections, reversals, unknown outcomes, permission leakage, incident detection and recovery time, and cost per verified outcome. A busy agent can be a security liability rather than proof of value.

• Common foundation: registry, identity, policy, tool gateway, sandbox, ledger, and incident controls.
• Use-case contract: purpose, data, tools, actions, owners, tests, monitoring, and recovery.
• Authority ladder: read, recommend, draft, stage, execute reversible, execute consequential.
• Evidence gates: evaluation, red team, operations, cohort outcomes, and risk acceptance.
• Continuous review after model, prompt, tool, plugin, policy, or scope changes.

A 90-day security program can reduce risk before replacing every agent

During days one through fifteen, discover agents, OAuth grants, API keys, plugins, scheduled jobs, and tool servers. Rank them by sensitive data, untrusted input, external effects, irreversibility, and unattended operation. Immediately suspend orphaned or clearly unsafe access through the organization's normal incident and change processes.

During days sixteen through forty-five, select one high-value workflow and separate read, proposal, approval, execution, and verification. Add workload identity, scoped credentials, action schemas, parameter-bound approval, idempotency, limits, reversible staging, egress policy, and an action ledger. Establish a safe degraded mode.

During days forty-six through seventy, threat-model and test the full path. Exercise injection, memory poisoning, plugin tampering, credential theft, cross-tenant access, forged approval, context loss, duplicate effects, unknown outcomes, kill switches, and restoration. Train operators and business owners on the incident playbook.

During days seventy-one through ninety, release to a narrow cohort or shadow mode. Monitor effects, denials, corrections, reversals, unusual destinations, budgets, and verified outcomes. Apply the learned controls to the common agent platform and prioritize the next registry risks.

Bizz custom software development can engineer this control plane around existing cloud, SaaS, identity, data, and security investments. The organization can use appropriate models and agent frameworks without allowing any one runtime to become the policy authority.

• Days 1-15: inventory, capability risk, ownership, and immediate containment.
• Days 16-45: narrow tools, scoped identity, approval protocol, staging, limits, and ledger.
• Days 46-70: end-to-end adversarial tests plus incident and recovery exercises.
• Days 71-90: restricted release, outcome monitoring, and common-control rollout.
• Reassess authority whenever scope, model, tool, plugin, user, or data changes.

Trust comes from bounded authority, not a promise of perfect alignment

No model, prompt, platform, or guardrail can guarantee that every ambiguous and adversarial situation will be interpreted as its operator intended. Enterprise security does not need that guarantee. It needs an architecture in which a wrong interpretation cannot silently become an unlimited action.

That architecture makes identity explicit, keeps secrets outside context, treats external content as untrusted, separates proposals from effects, binds approval to parameters, stages reversible changes, limits tools and egress, records receipts, detects abnormal behavior, and recovers from authoritative state.

It also preserves human agency. People can understand what will happen, decline or change it, reach an accountable owner, and receive remediation when the system fails. Human-in-the-loop is meaningful only when the interface and authority are designed as controls rather than as a sentence generated by the same agent being reviewed.

The public incident pattern is a useful warning because it makes invisible assumptions visible. The durable response is not fear of autonomy or faith in a safer prompt. It is disciplined software and security engineering that grants only the autonomy the organization can observe, contain, and answer for.

• Assume interpretation can vary and enforce policy at the action boundary.
• Reduce the blast radius of credentials, data, tools, networks, memory, and plugins.
• Make approval specific, trusted, expiring, and resistant to replay or forgery.
• Design reversible operation, kill switches, reconciliation, and remediation.
• Earn additional autonomy through evidence from real outcomes and failure exercises.