After login, a stale Rasa skill line asked for the PIN again

A traveller gives Atlas their PIN, the PIN is accepted, and two turns later Atlas asks for it again. Atlas, Horizon Travel and the traveller, Maya, are the companion repository’s fictional demo. This is a live run on Rasa Pro 3.21.0.dev3 with OpenAI gpt-5.2, recorded on 29 September 2026 (receipt e14, precondition copy, run 1):

user: Hi, I would like to verify my identity first. My PIN is 4242.
tool: activate {"target_id": "intro"} -> activate: done
bot:  Hi Maya. I’m Atlas, the Horizon Travel voice assistant.
      
      I can help you check your itinerary, get flight status, change or cancel a booking, report lost baggage, or answer common travel FAQs.
      
      What would you like to do today?
tool: activate {"target_id": "authenticate"} -> activate: done
tool: verify_traveler_pin {"pin": "4242"} -> {"ok": true, "authenticated": true, "display_name": "Maya Chen", "hint": "PIN accepted."}
bot:  I was starting to introduce what I can help with at Horizon Travel.  
      Do you want me to continue, or should I stop?
user: Skip it, thanks.
bot:  Is there anything else you need help with?
user: I want to change the date of my Lisbon booking.
tool: activate {"target_id": "change_booking"} -> activate: done
tool: activate {"target_id": "authenticate"} -> activate: done
bot:  Sure, I can help change your Lisbon booking.
      
      Before I make any changes, please tell me your four-digit Horizon Travel PIN.

All user and bot events from the saved tracker, line breaks indented; the tool lines are a selection: the PIN check and every activate call.

change_booking used to be gated by the skill-level line requires: session.project.authenticated. Rasa Pro 3.21.0.dev3 refuses it, and the error suggests a precondition, which we bound in agent.yml. The skill’s instructions also included, at line 19, “First verify identity: @skill.authenticate”, and we left that alone, as a migration that follows the error message would. Ten logged-in runs with the line, and ten with only that line deleted:

The guard was written twice, and the migration deleted only one copy. The precondition is the engine’s copy. authenticated was already true, so the engine activated change_booking straight away in all twenty runs. Line 19 is the prompt’s copy, and the model kept obeying it after the engine had decided. In run 4 the extra call went further: the tracker records skill_cancelled for change_booking, its removal from the stack and skill_resumed for authenticate, and the scripted run ended on the next reply, before any effect on the traveller could be seen. The mechanism is not specific to Rasa: a guard written once in code or configuration and again in the prompt stays duplicated until someone deletes the prompt’s copy, and moving the code copy, as this migration did, leaves the prompt’s copy exactly where it was. The other fix the error offers, hidden: true, is not a login gate at all: the dev3 source drops a hidden skill from routing with no condition, and in three logged-in runs the request never reached it.

These are ten runs per precondition copy and three on the hidden copy, on one model at temperature 0.0, one agent and one prerelease, and we did not run the old gate after login.

After login, the model followed the prompt’s copy

Each run is a scripted conversation. After login, the three turns in the exchange above. Before login, four turns: “I want to change the date of my existing booking, please.”, “The Lisbon one.”, “4242” and “Move it to next Friday, please.”. These are the receipt’s lines for three of the twenty logged-in precondition runs, and its totals (receipt e14; its first three line-19 runs are the same trackers as receipt e13):

s3 pre run 1: authenticated_before_turn3=True activate=['change_booking', 'authenticate'] PIN_asked=True bot=['Sure, I can help change your Lisbon booking.  Before I make any changes, please tell me your four-digit Horizon Travel PIN.']
s3 pre run 4: authenticated_before_turn3=True activate=['change_booking', 'authenticate'] PIN_asked=False bot=['Got it. I can help change the date on your Lisbon booking.  You’re already verified.   What new date do you want for the Lisbon trip?']
s3 pre19 run 1: authenticated_before_turn3=True activate=['change_booking', 'find_booking'] PIN_asked=False bot=['Got it. You mean your **Lisbon spring break** booking, reference **H T one two three four five**.  What new departure date would you like, a']
--- totals, scenario 3 (10 runs each): change_booking activated / authenticate activated again / PIN asked again ---
pre: 10/10 8/10 2/10
pre19: 10/10 0/10 0/10

The runs were not identical at temperature 0.0, the value the companion’s integrations.yml sets: with line 19, the model went to authenticate in eight runs and to find_booking in two, and asked for the PIN in two.

In seven of the eight runs, the model’s call started authenticate afresh on top of change_booking. In run 4 the call resumed the authenticate frame left over from the login turn. The tracker records skill_cancelled for change_booking, remove /2 on the stack and skill_resumed for authenticate. The reply said “You’re already verified.” and asked “What new date do you want for the Lisbon trip?”, and the scripted run ended there, before any effect on the traveller could be seen.

The fix is one deleted line. It is the only difference the recursive diff in receipt e15 found between the two precondition copies. That diff excludes .venv, models, .rasa, __pycache__, .env and *.db files:

--- w5/pre/skills/change_booking/skill.md	2026-09-29 11:44:54
+++ w5/pre19/skills/change_booking/skill.md	2026-09-29 12:54:26
@@ -16,8 +16,6 @@
 
 Help the traveler change or cancel a booking.
 
-First verify identity: @skill.authenticate
-
 Then find the booking: @skill.find_booking
 
 Ask what they want to change.

To find the prompt’s copy of a guard in your own project, search the skills for the resolver’s name:

In the companion at 01d6a6e (22 Sep) there is one hit:

$ git grep -n '@skill.authenticate' 01d6a6e -- examples/mantle-voice-agent/skills
01d6a6e:examples/mantle-voice-agent/skills/change_booking/skill.md:19:First verify identity: @skill.authenticate

Deleting the line did not weaken the gate. Before login, on the copy without it, the engine parked change_booking behind authenticate and activated it after the PIN in 3 of 3 runs. The gate covers change_booking and nothing else, though. In all six precondition runs before login, with and without line 19, find_booking listed the traveller’s bookings before any PIN, and in three of them Atlas read out a trip name or the booking reference first (receipts e13 and e14).

The copies differ from companion revision 01d6a6e only in the gate and its agent.yml binding, the rasa-pro pin, the way credentials are written, one test-harness edit that lets the served agent find the demo data, line 19 where stated, and, in the hidden copy, the two flight_status mentions rewritten as plain text (receipts e13 and e15). The model is gpt-5.2, the one the companion ships with (integrations.yml at 01d6a6e).

Skill-level requires is no longer supported on 3.21.0.dev3

The new gate is not optional. Leave the dev1 line in, and the project no longer validates on dev3. These are the last three lines of a 1,311-line capture that includes a traceback (receipt e1):

rasa.exceptions.ValidationError: Project validation found 2 problem(s):
  - [mantle.validation.skill.failed_to_load] Skill directory 'skills/change_booking' failed to load and would be silently skipped: Skill-level requires is no longer supported. Use precondition to park the skill until a resolver runs, or hidden: true to omit it from routing. Tool-level requires under tool_constraints is unchanged.. Fix the skill so it parses, or remove it.
  - [skill.referenced.invalid_target] Skill 'flight_status' references external call/link target 'change_booking', which does not resolve to a skill or flow in the catalog.

The check in the dev3 wheel tests only whether the key is present, so no value of a skill-level requires: gets through (rasa/mantle/skills/markdown_skill_compiler.py, lines 925 to 937, in the rasa-pro 3.21.0.dev3 wheel with sha256 8b7a1977cc421428757789384356930ce81defbb8779b4789a91adba11843495):

def _reject_skill_level_requires(frontmatter: Dict[str, Any]) -> None:
    """Fail closed when skill.md still declares a skill-level ``requires``."""
    if REQUIRES_FIELD not in frontmatter:
        return
    raise ValidationError(
        code=_SKILL_LEVEL_REQUIRES_REMOVED_CODE,
        event_info=(
            "Skill-level requires is no longer supported. Use precondition to "
            "park the skill until a resolver runs, or hidden: true to omit it "
            "from routing. Tool-level requires under tool_constraints is "
            "unchanged."
        ),
    )

The message offers two replacements. We tried both, and a precondition without its agent.yml binding. One of the four variants passes, and two of the three failures are reported against flight_status, a skill none of the edits touched:

Variant 1 shows how one leftover key produces two errors. The second is reported against flight_status, because change_booking never loaded and the mention in flight_status points at a target that is not in the catalog. Variants 2 and 3 have no invalid_target problem, since change_booking loads in both. Fix the skill that fails to load first; the reference problem downstream is a consequence, not a second fault.

Each variant is a copy of examples/mantle-voice-agent on rasa-pro 3.21.0.dev3, whose skills match RasaHQ/rasa-community-resources at companion revision 01d6a6e (22 Sep) apart from the edits in the table (receipt e9). Horizon Travel, Atlas and the traveller’s data are the companion’s fictional demo. Run this from inside a copy:

uv run --locked python -c "from pathlib import Path; from rasa.mantle.validation import validate_project; validate_project(Path('.')); print('validate_project: ok')"

That command, run in each of the four copies, reproduces all four results (receipt e7). The first captures of each variant recorded the same command with an extra --project flag. Validation is offline and makes no model calls. A pass means the files parse and the names resolve. It says nothing about a conversation, which is why the live runs exist.

Variant 1’s first problem says the skill “would be silently skipped”. Neither path we tried was silent. rasa train on the variant 1 copy logs mantle.skill_catalog.skill_load_failed at ERROR, reports the same two problems, exits 1 and leaves models/ empty (receipt e5). On dev3 the old line stopped the build instead of shipping an agent without change_booking. We did not test serving a model trained on an older build.

hidden: true fails on a mention in another skill

hidden: true is the shorter edit, and “omit it from routing” sounds like what the old line did. Put it in place of requires:, and validation fails again, on a skill nobody edited. This is the output as saved, below its header line (receipt e2):

    raise ValidationError(
rasa.exceptions.ValidationError: Project validation found 1 problem(s):
  - [skill.prose.hidden_skill_reference] Skill 'flight_status' references hidden skill 'change_booking' via '@skill.change_booking' in prose. Hidden skills cannot be started from routing; use a deterministic call: or link: step, or bind the skill as resolve_with in agent.yml orchestrator.preconditions.

flight_status reports delays. When a flight is delayed or cancelled, its instructions offer a booking change by naming the other skill in plain prose (skills/flight_status/skill.md at 01d6a6e, lines 19 to 25):

if: session.flight_status.flight_status == "delayed"
Tell them the delay in minutes and the gate if available. Offer to help with
a booking change via @skill.change_booking.

if: session.flight_status.flight_status == "cancelled"
Apologize briefly. Explain that rebooking options are available and offer
@skill.change_booking or @skill.human_handoff.

Those are the only two mentions in the project’s skills:

$ git grep -n "@skill.change_booking" 01d6a6e -- examples/mantle-voice-agent/skills
01d6a6e:examples/mantle-voice-agent/skills/flight_status/skill.md:21:a booking change via @skill.change_booking.
01d6a6e:examples/mantle-voice-agent/skills/flight_status/skill.md:25:@skill.change_booking or @skill.human_handoff.

The error gives the reason: “Hidden skills cannot be started from routing”, so a prose mention of one points at a skill the router can never start. It also names three ways out: a call: step, a link: step, or binding the hidden skill as resolve_with. We tested none of the three.

There is a second reason not to use hidden: true, and it holds even where nothing mentions the skill. In the dev3 source, the function that decides which skills the model may activate drops a hidden skill with no condition attached (shown in the section on the dialogue stack below). The old line kept change_booking off that list only while session.project.authenticated was false. hidden: true keeps it off for every traveller. Our hidden copy, with the two mentions rewritten as plain text so it validates, confirms it: in three runs a traveller who had just given the right PIN asked to change a booking, and change_booking was never activated. The request went to find_booking each time (receipt e13).

A precondition validates only once agent.yml binds it

The other replacement is precondition:. Write it the way the old line was written and nothing else, and validation fails again. The output as saved, below its header line (receipt e3):

    raise ValidationError(
rasa.exceptions.ValidationError: Project validation found 1 problem(s):
  - [mantle.validation.precondition.missing_binding] Skill 'change_booking' declares precondition 'authenticated', but agent.yml has no matching entry under 'orchestrator.preconditions'.

The frontmatter value is now a name, not an expression. This is skills/change_booking/skill.md in the precondition copies used for the live runs, lines 1 to 15. Line 6 is the only line that differs from the companion at 01d6a6e:

---
name: change_booking
description: >
  Change or cancel an existing Horizon Travel booking.
  Activate for date changes, cancellations, or "I need to change my trip".
precondition: authenticated
tool_constraints:
  - cancel_booking:
      requires: session.project.selected_booking_ref
      requires_confirmation:
        enabled: true
        utter_for_confirmation: utter_confirm_cancel_booking
        utter_on_user_denial: utter_cancel_aborted
      on_success: utter_booking_cancelled
---

The condition moves to agent.yml, under a top-level orchestrator: key. This block is from the precondition copies used for the live runs (receipts e6 and e13), and variant 4 declares the same binding:

orchestrator:
  preconditions:
    authenticated:
      satisfied_when: session.project.authenticated
      resolve_with: authenticate

With the binding in place, validation prints one line, validate_project: ok (receipts e4 and e7).

The gate became a dialogue step, and the tracker shows it

What changed is where the check runs. On dev1 it runs while the engine builds the list of skills the model may activate. A skill whose requires is false is left off (rasa/mantle/skills/catalog.py, lines 862 to 870, in the rasa-pro 3.21.0.dev1 wheel with sha256 26d510fb6f2f3d09271f513e1408112db233dbf5ee81186a4d36a44a918b651a):

def _is_excluded_from_routable_entries(
    skill: Skill,
    skill_id: str,
    memory_values: dict[str, Any],
) -> bool:
    """Return whether *skill* must be omitted from `SkillCatalog.routable_entries`."""
    if skill.disabled:
        return True
    return not _skill_condition_satisfied(skill, skill_id, memory_values)

The same function in the dev3 wheel no longer reads memory at all (lines 853 to 855):

def _is_excluded_from_routable_entries(skill: Skill) -> bool:
    """Return whether *skill* must be omitted from `SkillCatalog.routable_entries`."""
    return skill.disabled or skill.hidden

So on dev3 change_booking stays on the list, and the check moves to the moment a skill is activated (rasa/mantle/orchestration/skill_executor.py). The working-agent runs show all three branches that matter here:

1. If the skill’s precondition holds, it activates as normal (line 903). After login, every precondition run pushed change_booking as a regular frame.
2. If not, the skill is pushed as a pending frame (lines 927 to 935, with the frame type set at line 988), and the resolver named in resolve_with is activated above it as a call frame (lines 955 to 963). Before login, every precondition run did this.
3. When the resolver completes, the engine checks the condition again. If it holds, the pending skill is promoted to an active run (lines 1079 to 1082). If not, the pending skill is dropped (line 1084). Promotion happened before login in two of the three line-19 runs and all three runs without it. No run reached the drop.

The code is below, verbatim, for anyone who wants to check the line numbers.

        if precondition_satisfied(binding, memory_values):
            return self._activate(
                flow_id,
                tracker_handle,
                frame_type=frame_type,
                new_frame_type=new_frame_type,
                skill_id=skill_id,
                block_id=block_id,
                reset_memory=reset_memory,
            )

        # If the resolver is already the live skill, park the consumer under
        # that run. A second resolver would interrupt the first, reset its
        # memory, and later offer continue_interrupted for work already done.
        if self._park_pending_under_live_resolver(
            flow_id,
            tracker_handle,
            resolver_skill_id=binding.resolve_with,
            new_frame_type=new_frame_type,
            skill_id=skill_id or owning_skill.id,
            block_id=block_id,
        ):
            return True

        if not self._push_pending_target(
            flow_id,
            tracker_handle,
            frame_type=frame_type,
            new_frame_type=new_frame_type,
            skill_id=skill_id or owning_skill.id,
            block_id=block_id,
        ):
            return False

        resolver_skill = self._skill_catalog.skill_by_id(binding.resolve_with)
        if resolver_skill is None:
            structlogger.error(
                "mantle.skill_executor.precondition.missing_resolver",
                resolver_skill_id=binding.resolve_with,
            )
            self._pop_pending_target(tracker_handle)
            return False

        resolver_flow_id = entry_target_id_for_skill(resolver_skill)
        if resolver_flow_id is None:
            structlogger.error(
                "mantle.skill_executor.precondition.resolver_no_entry",
                resolver_skill_id=binding.resolve_with,
            )
            self._pop_pending_target(tracker_handle)
            return False

        if not self._activate(
            resolver_flow_id,
            tracker_handle,
            frame_type=FlowStackFrameType.REGULAR,
            new_frame_type=FlowStackFrameType.CALL,
            skill_id=resolver_skill.id,
            block_id=self._skill_catalog.ordered_block_id_for_target(resolver_flow_id),
            reset_memory=True,
        ):
            self._pop_pending_target(tracker_handle)
            return False
        return True

This jq filter prints every change to the dialogue stack in the saved tracker of one line-19 run before login. Run it from editorial/receipts/skill-preconditions-replace-requires/ in this site’s repository:

$ jq -r '.events[] | select(.event == "stack") | .update | fromjson[] | [.op, .path, (.value // "" | if type == "object" then "\(.skill_id) \(.frame_type)" else tostring end)] | join(" ")' e13-tracker-s1-pre-run3.json
add /0 default_session_start regular
add /1 default_session_start regular
remove /0
remove /0
add /0 change_booking pending
add /1 authenticate call
add /1/previous_frame_type call
replace /1/frame_type interrupt
add /2 find_booking regular
remove /2
remove /1/previous_frame_type
replace /1/frame_type call
remove /1
replace /0/frame_type regular

change_booking goes on as pending with authenticate above it as a call. “The Lisbon one.” interrupts authenticate for find_booking, and “4242” brings authenticate back. Once the PIN is accepted, authenticate is removed (remove /1) and change_booking becomes a regular frame: that last line is the promotion. While authenticate was on top of the stack, the model’s first question in every line-19 run before login was about the booking, not the PIN; the scripted “4242” on turn 3 is what authenticate took as the PIN.

The two copies of the guard sit on different paths. The engine’s copy runs before the skill starts. The prompt’s copy sits inside the skill: at run time the engine does not act on it, and the model does, by calling activate for authenticate, as the receipt’s activate lines show:

Three things to review after the migration, each resting on a different kind of evidence:

Our position on the last row: treat the precondition as deciding when a dialogue starts, and keep the check that protects the booking on the tool.

What these runs do not show

• More than a small sample: ten runs per precondition copy after login, three on the hidden copy, and three per copy before login, all on gpt-5.2 at temperature 0, one agent, one phrasing per conversation. We set aside a third conversation whose second turn, in every run, answered the agent’s question about its introduction.
• An unmodified agent: lib/database.py carries one harness edit, which reads the project root from ATLAS_PROJECT_ROOT so the served agent finds the demo data. The hidden copy’s agent.yml keeps the binding, unused.
• The whole change: no run completed a date change. The skill hands paid reissues to a human (skills/change_booking/skill.md at 01d6a6e, lines 31 to 32), and every before-login run ended still collecting or confirming the new date, apart from one hidden run that offered a handoff. No run reached the drop after a failed PIN.
• Other models and versions: gpt-5.2 on 3.21.0.dev3 only. dev1 accepts the skill-level line and dev3 refuses it; dev2 was not tested, and every build here is a prerelease.
• The old gate after login: on 3.21.0.dev1, line 19 sat beside the skill-level requires: line, and after login change_booking was routable there too. We did not run that control, so these runs do not show whether the second authenticate predates the migration.
• An earlier round: Claude Haiku 4.5, on an agent whose served model carried no demo data and whose opening turn failed in every run, is recorded in receipts e6, e8, e10, e11 and e12. This page draws no rates from it.

All runs were recorded on 29 September 2026.

Deleting one line and rerunning with everything else unchanged is a one-variable mutation, the technique the guard mutation-testing guide applies to a guard’s test suite.