First decide whether an agent is justified
Agentic behaviour is useful when a system must choose and sequence steps in response to changing information. It also introduces more possible paths, more external effects, and more state to control. A deterministic workflow is often the better design when the process is known, the decisions can be expressed as rules, or the cost of variation is high.
Ask what uncertainty requires model-directed planning. If the only variable is how to phrase a response, use a constrained generation step. If the system must select among approved read-only sources, a bounded router may be enough. Add write actions, durable memory, delegation, or open-ended planning only when each creates measurable value that simpler orchestration cannot provide.
Assign autonomy per capability
A single label such as ‘human in the loop’ is too vague. The same product may safely let an agent search approved documents without review, require confirmation before drafting a customer message, and prohibit it from sending money or changing an entitlement. Autonomy belongs to a capability and effect, not to the product as a whole.
Create a capability register. For each capability, record whether it is read-only or effectful, the data it can reach, the maximum scope, the policy decision point, the required approval, the audit evidence, and the fallback. This reveals where one broad credential or general-purpose tool has accidentally given the agent more authority than the business intended.
The six-field execution contract
Ardevant's bounded-autonomy method defines an execution envelope with six fields. It is used in the Agentic AI Architecture & Security course to turn an agent idea into a reviewable system contract.
- Goal: the concrete outcome the run is allowed to pursue and the success evidence it must produce.
- Scope: the records, users, systems, time window, and task boundary included in this run.
- Authority: the identity, permissions, tools, and effects granted to the agent—never inferred from the goal alone.
- Budgets: limits for time, cost, tokens, steps, retries, data volume, and downstream effects.
- Invariants: conditions that must remain true, such as separation of duties, data residency, or ‘no external message without approval.’
- Stop conditions: events that end or pause the run, including uncertainty, policy denial, conflicting evidence, repeated failure, budget exhaustion, or an unknown write outcome.
The contract should be machine-enforced where practical and human-readable everywhere. It must travel with durable execution state so a retry, resumed workflow, or delegated task cannot silently receive a broader envelope.
Put a deterministic shell around model discretion
The model can interpret a request, propose a plan, choose among allowed options, and draft parameters. Deterministic components should authenticate the subject, resolve policy, issue scoped capabilities, validate schemas, enforce budgets, bind approvals, execute effects, and record outcomes.
This separation matters because a prompt is not a security boundary. Instructions can be misunderstood, displaced by retrieved content, or bypassed by an unexpected path. A policy decision point should evaluate structured facts about the actor, capability, resource, purpose, and requested effect. The executor should accept only the resulting narrow grant.
Tool descriptions also need to be contracts rather than marketing prose. Define input and output schemas, side effects, idempotency behaviour, permission requirements, data classification, timeout and retry rules, rate or spend limits, and failure semantics. A tool that can perform several unrelated actions should usually be split into smaller capabilities.
Bind approval to the exact effect
A generic approval to ‘continue’ is fragile. The person approving should see the material effect: recipient, resource, amount, content, permission change, or other consequential parameters. The system should bind that approval to a canonical action object and reject execution if the action changes afterwards.
Approval also needs an expiry, approver identity, policy context, and one-use or bounded-use semantics. The final executor—not the model—verifies the binding. This reduces approval laundering, where a benign-looking preview is later converted into a different action.
Some capabilities should remain prohibited even with an approval click, especially where the business process requires a different role, contractual control, or independent check. Human involvement does not automatically make an unsafe authority model safe.
Design for retries and unknown outcomes
Agent workflows outlive individual model calls. Services time out, workers restart, approvals arrive later, and external APIs can complete an action without returning a response. Store the authoritative workflow state outside the model context. Record planned actions, policy decisions, approvals, attempts, provider references, and observed outcomes.
Every effectful command needs an idempotency strategy. If a payment, message, or permission change has an unknown outcome, reconcile with the system of record before retrying. Asking the model what probably happened is not reconciliation. The safe state may be ‘requires investigation,’ with further action blocked until evidence arrives.
Memory deserves the same care. Separate durable business state from conversational context and learned preferences. Record provenance, scope, retention, and confidence. Treat content written by users, tools, or other agents as potentially tainted; it must not become a new policy or credential merely because it was remembered.
Observe the path from goal to effect
Operators need more than model logs. A goal-to-effect trace should connect the initiating user and request to the selected plan, evidence, policy checks, tool calls, approvals, external effects, final result, and cost. Sensitive content can be minimized or referenced rather than copied, but the decision path must remain reconstructable.
- Detect repeated planning, tool loops, budget pressure, policy denials, and unusual capability combinations.
- Measure task success and harmful side effects, not only answer quality or model latency.
- Test adversarial documents, compromised tools, stale memory, delegated authority, and unavailable dependencies.
- Define how to pause runs, revoke capabilities, quarantine memory, disable a tool, and recover state during an incident.
Use the AI architecture review checklist to place these controls inside the wider production system, with accountable owners and release evidence.
Require evidence before increasing autonomy
A responsible autonomy roadmap moves one capability at a time. Begin with observation or suggestions, then a reversible effect with narrow scope, then broader execution only after measured evidence supports it. Each step should name the new authority, expected value, evaluation set, monitoring, rollback, and residual risk owner.
- Do we have stable success criteria and representative failure cases?
- Can policy enforce the proposed scope without relying on model compliance?
- Are approvals bound to exact effects where they are required?
- Can effects be made idempotent, reconciled, reversed, or contained?
- Will telemetry reveal both task success and unintended consequences?
- Can an operator stop the system and recover authoritative state?
If the evidence is weak, holding the current boundary is a sound architecture decision. Bounded autonomy is not a temporary limitation on the way to maximum autonomy; it is the mechanism that makes useful delegation governable.