Review what an agent is allowed to do
by Rasa team
Ask “what can this agent change?” before asking whether it sounds careful. A polite explanation does not establish authority to update a record, send a document, or commit a transaction.
This guide is for a domain owner or risk reviewer working with product and engineering. You will produce an action register and a set of evidence requests. It is an engineering review aid, not a legal determination or a compliance certification.
Build an action register
The example below uses fictional Horizon Travel. Its agent can explain booking conditions and prepare a request; staff handle exceptions. These are proposed boundaries for the example, not a description of a real business.
| Action | Required authority and data | Boundary to inspect | Accountable owner |
|---|---|---|---|
| Read a booking | Customer authorized for that booking | Lookup verifies access before returning details | Booking service owner |
| Explain a policy | Current policy associated with the booking | Answer traces to the retrieved source | Travel policy owner |
| Request staff review | Customer-approved summary | Receiving queue contains the agreed fields | Support lead |
| Commit a booking change | Verified authority and accepted conditions | Tool rejects missing or mismatched authorization | Booking service owner |
| Export a document | Approved source state and recipient | Artifact and delivery match the approved state | Document process owner |
List the actual actions in your system, including those hidden behind a broad tool name such as “update_customer.” Ask engineering to expand the tool’s effects. A label can hide multiple permissions.
Request evidence at the point of action
The guarded-action tutorial illustrates why a tool boundary matters for an irreversible change. The derived-document tutorial explores keeping an artifact tied to source state.
For each action, ask for one allowed case and one denied case. Inspect the resulting system record, not just the agent’s narration. A denied case passes only if the prohibited change did not happen. Keep the input conditions and software version with the evidence so another reviewer can reproduce the check.
For a booking change, useful denied cases include a reference belonging to another customer, expired authorization, and a request that changes after confirmation. Your team decides which controls implement these rules; the review makes the intended invariant explicit.
Do not confuse approval with unlimited scope
A customer approving one action does not approve every later action. In the fictional example, approval to send a fee question to support does not authorize a booking change. Write the scope into the action register and the conversation design.
The difficult case is a corrected address or recipient after the user has approved a document. Ask whether the confirmation is bound to the exact data used at execution. If the system cannot establish that relationship, mark it unresolved and hold that action or route it to a human with the relevant authority.
NIST’s AI RMF Core addresses documenting context, measurement, and management of risks. It does not make this checklist a substitute for your organization’s policy or applicable obligations.
Leave with a decision someone owns
For each row, record: allowed scope; source of authority; evidence inspected; unresolved cases; disposition; owner; review date. Use “unknown” when evidence is missing. Do not turn an assumption into an approval because a deadline is near.
A useful disposition can be narrow: permit policy lookup while holding booking changes. Product must ensure the user-facing workflow reflects that limitation; operations must be able to disable the affected action; design must provide an honest handoff.
Before the next release, choose the highest-impact action and ask engineering to demonstrate its denied case. Bring the register to a release decision review. This creates a concrete conversation between the domain owner and the builder, with evidence both can inspect.
