Capability-Adaptive CAR
a 7 part framework

Governing AI Agents as Their Capabilities Advance

Most AI safety remains reactive:

A new capability appears. A failure is discovered. A control is developed.

That sequence may become increasingly difficult to sustain as agentic AI becomes more capable, more autonomous, and more deeply connected to real business systems.

The immediate risk is not necessarily that an AI agent develops malicious intent. It may simply become extremely effective at pursuing a legitimate objective—including finding pathways, combinations of actions, or workarounds its designers never anticipated.

And the agent itself does not have to “go rogue.” A legitimate agent can also become dangerous if its credentials, command path, identity, or delegated authority are compromised.

That leads to a different control question:

How do we keep authority and consequence from expanding automatically as AI capability increases?

The Contained Agentic Runtime (CAR) starts with one principle:

Capability should never automatically equal authority.

Capability-Adaptive CAR adds another idea: the controls themselves must evolve as agents become more capable.

The Capability-Control Evolution Loop is an AI-assisted process for testing whether plausible future capabilities or attack paths could defeat today's controls before those conditions become operational.

The objective is to move from:

Capability appears → failure occurs → control follows

toward:

Plausible capability → control assumption challenged → protection tested → capability arrives

An agent may be technically capable of deploying code, transferring money, changing system permissions, communicating externally, or modifying another model.

That does not mean it should automatically be allowed to do those things.

A safer operating structure looks more like:

Principal → trusted instruction → delegated authority → identified agent → bounded environment → authorized action

The agent operates inside an authority envelope: the defined set of systems, resources, actions, and consequences it is permitted to affect under its current delegation.

Concepts That Must Stay Separate
  • Identity: Who or what is acting?
  • Delegation: Who gave the agent authority, and what authority was transferred?
  • Authorization: Is this particular action allowed within that delegation?
  • Principal intent: Did the legitimate person or organization actually request or approve this consequential action?
  • Acceptance (for agents working across organizations): Does the external system recognize that delegated authority?
A human's valid credential should not automatically mean “the agent can do everything the human could do.” Separating capability from authority therefore requires protecting not only what the agent does, but also the environment and mechanisms that determine what it is allowed to do.

CAR is more than a sandbox. Its controls fall broadly into three layers.

The Three Control Layers
  • Runtime controls restrict what the agent can reach or alter: network access, persistence, credentials, tools, and execution environments.
  • Authority and governance controls determine what it may do: consequence thresholds, approval requirements, and graduated autonomy.
  • Identity and accountability controls establish who is acting, on whose behalf, and how consequential actions are attributed.
One Additional Boundary Is Critical

An agent should not have unilateral authority to modify the mechanisms that define its own capability, identity, permissions, or oversight. That includes model configuration, system instructions, tools, authorization policies, monitoring, and delegation records.

Controlled research by AI-security firm Irregular has shown why this matters. In experimental environments where agents were given access to model weights, training tools, and deployment capabilities, agents sometimes chose model modification as a way to accomplish an ordinary assigned objective.

The lesson is not that agents will inevitably rewrite themselves. It is that if a powerful capability is available, a sufficiently capable agent may treat using it as simply another means of completing the task.

Any material change to the agent or its governing configuration should create, for control purposes, a new agent identity whose authority must be reassessed.

Every safeguard works because some assumption remains true.

Assumptions Worth Questioning
  • An egress allowlist assumes prohibited destinations can be identified.
  • Task-scoped credentials assume equivalent authority cannot be obtained through another route.
  • Human approval assumes the person understands what is being approved.
  • Action-level authorization assumes risk can be judged one action at a time.
  • A trusted endpoint assumes the instruction reaching the agent actually came from the intended user.
Capability-Adaptive CAR Asks

What new capability—or compromise—would cause that assumption to stop being true?

Then adversarial testing tries to prove the control inadequate. The better test is not “Can you break the rule?” It is “Can you achieve the prohibited outcome while technically remaining inside the rules?”

For example, imagine a procurement agent permitted to reorder from approved suppliers but prohibited from creating new suppliers. A capability scenario might ask: what if the agent becomes good enough to create several approved-looking subaccounts or route purchases through intermediary vendors?

If adversarial testing shows that the intended restriction can be bypassed without technically violating any single rule, the organization can redesign the control before that behavior appears in production.

That is what the Evolution Loop is intended to produce: a discovered weakness, a stronger control, and a new test the system must continue to pass.

As agents become more capable, controlling individual actions will not be enough. The unit of control must expand.

Six Dimensions of Control
  • Action control asks: What may this agent do now?
  • Trajectory control asks: What is this sequence of actions collectively accomplishing?
  • Multi-agent control asks: What are several agents producing together?
  • Control-plane integrity asks: Can the agent alter the mechanisms governing itself?
  • Delegated-authority control asks: Is the authority being exercised still the authority actually granted?
  • Principal-intent integrity asks: Is the agent still acting on the legitimate principal's actual instructions?
Why Cumulative Consequence Matters

Consider a financial agent authorized to transfer up to $5,000 without approval. One $4,900 transaction may be acceptable. Twenty $4,900 transactions may technically satisfy the rule while completely defeating its purpose. The control therefore has to evaluate the cumulative consequence, not merely each isolated action.

Recent developments around Meta's Muse illustrate two other ends of the authority chain. A client-side vulnerability showed that an agent's cloud environment can remain secure while the command path controlling the agent is compromised. Separately, Amazon's decision to block Muse raised a different question: even if a user authorizes an agent, does an external company have to accept that agent as an authorized actor on its systems?

Different problems, same underlying lesson: authority requires more than possession of credentials.

AI can help improve agent safety. It can generate adversarial scenarios, identify unusual combinations of behavior, challenge hidden assumptions, and propose stronger controls.

But three problems remain:

  • Shared blind spots: similar models may fail in similar ways.
  • Monitor gaming: a capable agent may learn what its monitor looks for and adjust its behavior.
  • Capability co-evolution: better AI may strengthen attackers and defenders simultaneously.
Heterogeneous Defense

The governing principle: AI should expand the search for failure. It should not be able to declare the system safe—or grant itself additional authority.

That requires combining:

  • AI-assisted monitoring and testing
  • Deterministic controls that models cannot negotiate with
  • Traditional cybersecurity and identity controls
  • Independent evaluation
  • Human governance over changes to authority

A Controlled Lifecycle

A new safeguard should not move directly into production because an AI system proposed it.

  1. Identify a plausible capability or threat scenario.
  2. Determine which control assumption it challenges.
  3. Adversarially test that assumption.
  4. Develop a mitigation.
  5. Independently attack and evaluate the mitigation.
  6. Approve changes through governance.
  7. Deploy in a limited environment and monitor the result.

The loop should run periodically and whenever meaningful changes occur: new models, greater autonomy, access to new systems, serious near misses, or material changes to the agent's runtime or authority.

When a Control Fails

No framework should assume prevention always succeeds.

  1. Detect the abnormal condition.
  2. Contain the agent or affected environment.
  3. Revoke relevant authority, sessions, and credentials.
  4. Preserve evidence.
  5. Determine the blast radius.
  6. Recover or roll back safely.
  7. Identify the failed assumption and add it to future testing.
An incident should not merely be repaired. It should become a test the system must thereafter survive.

“Human in the loop” sounds reassuring, but manually approving every routine agent action is neither scalable nor necessarily safe.

People become fatigued. They may not understand every technical action. And the mechanism carrying their approval can itself be compromised.

Human judgment is more valuable at the policy level.

Leadership Should Determine
  • What consequences are acceptable
  • Which actions remain high consequence or irreversible
  • What authority may be delegated
  • When autonomy may increase
  • When authority must be restricted or revoked
Machines can then enforce those policies at machine speed.
Four Principles Anchor the Framework
  1. No autonomous AI system should be able to grant itself greater capability, authority, or control over the mechanisms governing it.
  2. An agent's authority should be traceable to an explicit delegation from an identifiable principal and limited to a defined scope.
  3. Possession of credentials, tokens, technical capability, or an authenticated session is not by itself proof of authorization or current principal intent.
  4. No safety architecture should depend solely on AI judgment; consequential controls require independent, heterogeneous, and externally enforceable safeguards.

Capability-Adaptive CAR does not claim to solve AI alignment or predict every future failure. Its objective is more practical:

Reduce the probability that the next increase in AI capability also produces an uncontrolled increase in authority or consequence.

Capability will continue to advance. The more durable challenge is ensuring that authority, consequence, and control advance deliberately rather than accidentally.