Guide · Risk reviewer / domain owner
How to test an AI agent’s permissions
Review an agent’s permissions with a worked action register, evidence from the tool boundary, and named owners for authorisation, handoff, and unresolved risk.
You are reviewing whether a travel agent may change a customer’s booking. Ask engineering to show both an authorised change and a rejected request for another customer’s booking. Check the booking record to establish that the rejected request changed nothing. The tool’s authorisation check determines whether this action is permitted.
As the domain owner or risk reviewer, you will make an action register with product and engineering. Each row names an action, its required authority, the check to inspect and the accountable owner. You will also record the evidence still needed before allowing that action. The fictional Horizon Travel example below shows what to ask for. This is an engineering review aid, not a legal determination or a compliance certification.
Build an action register
The example below uses fictional Horizon Travel. Its agent can explain booking conditions and prepare a request. Staff handle exceptions. These are proposed boundaries for the example, not a description of a real business.
| Action | Required authority and data | Boundary to inspect | Accountable owner |
|---|---|---|---|
| Read a booking | Customer authorised for that booking | Lookup verifies access before returning details | Booking service owner |
| Explain a policy | Current policy associated with the booking | Answer traces to the retrieved source | Travel policy owner |
| Request staff review | Customer-approved summary | Receiving queue contains the agreed fields | Support lead |
| Commit a booking change | Verified authority and accepted conditions | Tool rejects missing or mismatched authorisation | Booking service owner |
| Export a document | Approved source state and recipient | Artifact and delivery match the approved state | Document process owner |
List the actual actions in your system, including those hidden behind a broad tool name such as “update_customer.” Ask engineering to expand the tool’s effects. A label can hide multiple permissions.
Request evidence at the point of action
The guarded-action tutorial illustrates why a tool boundary matters for an irreversible change. The derived-document tutorial explores keeping an artifact tied to source state.
For each action, ask for one allowed case and one denied case. Inspect the resulting system record, not just the agent’s narration. A denied case passes only if the prohibited change did not happen. Keep the input conditions and software version with the evidence so another reviewer can reproduce the check.
For a booking change, useful denied cases include a reference belonging to another customer, expired authorisation, and a request that changes after confirmation. Your team decides which controls implement these rules. The review records what must remain true before the action can proceed.
Do not confuse approval with unlimited scope
A customer approving one action does not approve every later action. In the fictional example, approval to send a fee question to support does not authorise a booking change. Write the scope into the action register and the conversation design.
The difficult case is a corrected address or recipient after the user has approved a document. Ask whether the confirmation is bound to the exact data used at execution. If evidence for that relationship is missing, mark it unresolved and hold that action or route it to a human with the relevant authority.
NIST’s AI RMF Core addresses documenting context, measurement, and management of risks. It does not make this checklist a substitute for your organisation’s policy or applicable obligations.
Leave with a decision someone owns
For each action, record:
- What is allowed and who grants that authority
- Evidence inspected and unresolved cases
- Decision, owner and review date
Use “unknown” when evidence is missing. Do not turn an assumption into an approval because a deadline is near.
A useful disposition can be narrow: permit policy lookup while holding booking changes. Product must ensure the customer-facing workflow reflects that limitation. Operations must be able to disable the action. Design must explain the handoff accurately.
Before the next release, choose the highest-impact action and ask engineering to demonstrate its denied case. Bring the register to a release decision review. This creates a concrete conversation between the domain owner and the builder, with evidence both can inspect.