Skip to content
RasaGet a free licence
AI team casebook

Evaluation / data specialist · Banking & fintech (external)

Help a customer block the right bank card

Ask which card the customer means when two cards have the same final digits.

by Rod Rivera

About 11 minutesHands-on tutorial

Key takeaways (3)
  • Two cards can share the same ending even when they belong to the verified caller.
  • Resolve the spoken description to one service reference before asking for confirmation.
  • Test every different selected and requested card pair before connecting the write tool.

A customer reports a lost card and gives its final four digits. Two cards on their account share that ending. Blocking either one without another question could stop the wrong card.

Use this fictional banking example to check how the assistant presents the two candidates and waits for a clear choice. The tests inspect the selected card and confirm that no card is blocked while the choice is unclear.

{"case": "banking-block-card", "label": "shared-ending", "output": {"candidates": ["debit card ending 5 5 0 2 on your joint savings account", "credit card ending 5 5 0 2 on your travel credit account"], "card_last_four": "5502", "effects": 0, "next_step": "More than one card could match, or the card to block is not the one the caller confirmed. Read the candidates by kind and account and ask which one. Do not block anything yet.", "reason": "ambiguous_card_selection", "status": "blocked"}}

This real offline result contains two owned cards with the same ending. Verifying the caller again or spacing the spoken digits cannot choose between them. Blocking either card now would guess which resource the caller meant.

Use the ending as a lookup alias. Resolve it to one owned service reference, then bind the block to that selection. The exercise below tries every different pair of owned references and removes one comparison to show what it protects.

Reproduce the shared ending

git clone https://github.com/RasaHQ/rasa-community-resources.git
cd rasa-community-resources
git checkout 4aa0c4419dc193fef7a969c12d59edcf720f2606
cd examples/mantle-voice-banking-block-card-gpt

Keep the full clone so the neighbouring case contract is available. Run all commands below from this selected example directory. The inline exercise uses only Python’s standard library and calls no provider or Rasa engine.

Any system
python3 -B -m unittest discover -s tests -v

The pinned guard tests include test_shared_ending_is_ambiguous_until_the_kind_is_given and test_block_of_a_card_other_than_the_confirmed_one_is_refused. Start by tracing selection before looking at the write.

In lib/northgate.py, selection filters only cards owned by the supplied customer. It then matches the normalised ending and any kind the caller supplied. Zero matches and multiple matches are refused; only one match returns a service reference.

Resolve the kind to an opaque reference

The recorded selection results differ only by kind:

Caller suppliesOwned matchesResultWrite may use
Ending 5502Debit and creditAmbiguous; ask whichNone
Spaced ending 5 5 0 2, debitJoint savings debitSelectedNB-CARD-0103
Spaced ending 5 5 0 2, creditTravel creditSelectedNB-CARD-0104
Select NB-CARD-0101, request NB-CARD-0102Both owned, differentBlocked; zero effectsNone
Reselect NB-CARD-0102, request NB-CARD-0102Same selected referenceSucceeded in fixtureNB-CARD-0102

Card kind resolves this fixture collision. In your adapter, ask for another supported distinction if kind still leaves several matches. Do not present an ambiguous candidate label as a selected resource.

The Rasa selection wrapper writes the selected reference and its display label to skill memory. A failed or ambiguous new selection clears those fields, so it cannot leave the previous selection in place:

    result = ng.select_card(_service(), _get(context, CUSTOMER_KEY), card_last_four, card_kind)
    if context is not None:
        # A new selection always replaces the old one, so a correction can
        # never leave the previous card confirmed.
        context.memory.set("selected_card_ref", result.get("card_ref", ""))
        context.memory.set("selected_card_label", result.get("card_label", ""))
    return ToolResult(llm_response=result)

The block wrapper uses the verified customer and selected reference from memory, with the requested reference copied from the selection result:

    result = ng.block_card(
        _service(),
        _get(context, CUSTOMER_KEY),
        _get(context, "selected_card_ref"),
        card_ref,
        request_id=_conversation_id(),
    )
    return ToolResult(llm_response=result)

Caller words reach selection; the resolved resource reaches the write. Authentication establishes ownership. The comparison below establishes whether this write targets the selected owned card. Those are separate facts.

Test every different owned pair

Paste the following complete exercise from the selected example directory. It includes the recorded collision check, all different reference pairs and a deliberately broken version compiled in memory. It edits no source file.

python3 -B - <<'PY'
import json, types
from pathlib import Path

def load(project, filename, mutation=None):
    path = Path.cwd() / 'lib' / filename
    source = path.read_text()
    if mutation:
        old, new = mutation
        assert source.count(old) == 1, 'mutation anchor changed'
        source = source.replace(old, new)
    m = types.ModuleType(project.replace('-', '_') + ('_mutant' if mutation else ''))
    m.__file__ = str(path.resolve())
    exec(compile(source, str(path), 'exec'), m.__dict__)
    return m

def emit(case, label, value):
    print(json.dumps({'case': case, 'label': label, 'output': value}, sort_keys=True))

# The same last four can name two owned cards. Kind disambiguates the alias.
ng = load('mantle-voice-banking-block-card-gpt', 'northgate.py')
svc = ng.CardService()
ambiguous = ng.select_card(svc, 'NB-CUST-7310', '5502')
emit('banking-block-card', 'shared-ending', ambiguous)
assert ambiguous['status'] == 'blocked' and len(ambiguous['candidates']) == 2
for kind in ('debit', 'credit'):
    got = ng.select_card(svc, 'NB-CUST-7310', '5 5 0 2', kind)
    emit('banking-block-card', 'resolve-' + kind, got)
    assert got['status'] == 'selected'
refs = list(svc.cards_of('NB-CUST-7310'))
count = 0
for selected in refs:
    for requested in refs:
        if selected == requested:
            continue
        isolated = ng.CardService()
        result = ng.block_card(isolated, 'NB-CUST-7310', selected, requested, 'all-pairs')
        assert result['effects'] == 0 and result['status'] == 'blocked'
        assert all(c['state'] == 'active' for c in isolated.cards.values())
        count += 1
emit('banking-block-card', 'all-distinct-owned-ref-pairs', {'attempts': count, 'refused': count, 'effects': 0})

# A correction points at a different owned card: ownership alone must not suffice.
for mutant in (False, True):
    ng = load('mantle-voice-banking-block-card-gpt', 'northgate.py',
              ('bool(selected_card_ref) and ref == selected_card_ref', 'bool(selected_card_ref)') if mutant else None)
    svc = ng.CardService()
    selected = ng.select_card(svc, 'NB-CUST-7310', '4417')['card_ref']
    before = {r: c['state'] for r, c in svc.cards.items()}
    wrong = ng.block_card(svc, 'NB-CUST-7310', selected, 'NB-CARD-0102', 'correction-proof')
    changed = [r for r in before if before[r] != svc.cards[r]['state']]
    emit('banking-block-card', 'deliberate-equality-removal' if mutant else 'shipped-correction-refusal',
         {'selected': selected, 'requested': 'NB-CARD-0102', 'status': wrong['status'], 'reason': wrong['reason'], 'effects': wrong['effects'], 'changed_cards': changed})
    assert (wrong['status'], changed) == (('succeeded', ['NB-CARD-0102']) if mutant else ('blocked', []))
    if not mutant:
        repaired = ng.select_card(svc, 'NB-CUST-7310', '8023')['card_ref']
        done = ng.block_card(svc, 'NB-CUST-7310', repaired, repaired, 'correction-proof')
        emit('banking-block-card', 'reselect-then-confirm-integration-precondition', {'status': done['status'], 'card_ref': done['card_ref'], 'effects': done['effects']})

PY

Actual recorded output from these exercise cases, 8 October 2026:

{"case": "banking-block-card", "label": "shared-ending", "output": {"candidates": ["debit card ending 5 5 0 2 on your joint savings account", "credit card ending 5 5 0 2 on your travel credit account"], "card_last_four": "5502", "effects": 0, "next_step": "More than one card could match, or the card to block is not the one the caller confirmed. Read the candidates by kind and account and ask which one. Do not block anything yet.", "reason": "ambiguous_card_selection", "status": "blocked"}}
{"case": "banking-block-card", "label": "resolve-debit", "output": {"card_label": "debit card ending 5 5 0 2 on your joint savings account", "card_ref": "NB-CARD-0103", "next_step": "Call block_card with this card_ref. The engine asks the caller to confirm this card before anything changes.", "state": "active", "status": "selected"}}
{"case": "banking-block-card", "label": "resolve-credit", "output": {"card_label": "credit card ending 5 5 0 2 on your travel credit account", "card_ref": "NB-CARD-0104", "next_step": "Call block_card with this card_ref. The engine asks the caller to confirm this card before anything changes.", "state": "active", "status": "selected"}}
{"case": "banking-block-card", "label": "all-distinct-owned-ref-pairs", "output": {"attempts": 20, "effects": 0, "refused": 20}}
{"case": "banking-block-card", "label": "shipped-correction-refusal", "output": {"changed_cards": [], "effects": 0, "reason": "ambiguous_card_selection", "requested": "NB-CARD-0102", "selected": "NB-CARD-0101", "status": "blocked"}}
{"case": "banking-block-card", "label": "reselect-then-confirm-integration-precondition", "output": {"card_ref": "NB-CARD-0102", "effects": 1, "status": "succeeded"}}
{"case": "banking-block-card", "label": "deliberate-equality-removal", "output": {"changed_cards": ["NB-CARD-0102"], "effects": 1, "reason": "verified_fixture_receipt", "requested": "NB-CARD-0102", "selected": "NB-CARD-0101", "status": "succeeded"}}

There are five owned references in the fixture. Trying each selected reference against every different requested reference produces 20 attempts. Each uses a fresh service, checks zero effects and checks every card remains active. A test that asks only whether the requested card is owned misses this class of error entirely.

The pure-Python loop does not ask the engine for confirmation. Its reselect case establishes the matching-reference precondition for that later integration check. It is not evidence that a spoken correction was understood.

Remove only the selection comparison

The selection comparison is in the pinned write function:

    facts = {
        "selected_card_owned": bool(customer_id) and card is not None and card["customer_id"] == customer_id,
        "single_card_confirmed": bool(selected_card_ref) and ref == selected_card_ref,
    }

The deliberate mutation removes only and ref == selected_card_ref. Ownership still passes; some selection still exists. The wrong owned card then blocks successfully. The final mutation output names the changed card, so a misleading success status cannot hide which resource changed.

Keep the comparison inside the function that calls your card adapter. Its cost is maintaining a selected service reference separately from the caller’s description and testing that binding at execution. That work remains necessary when the display alias looks familiar.

Reselect before confirming the correction

The pinned skill.md puts the confirmation gate on the write tool:

tool_constraints:
  - block_card:
      requires: session.block_card.selected_card_ref
      requires_confirmation:
        enabled: true
        utter_for_confirmation: utter_confirm_block_card
        utter_on_user_denial: utter_block_cancelled

The skill directs a correction back through selection. Decline the old confirmation, resolve the corrected description and let the engine ask about the new selected card. Changing only the requested tool argument leaves the old memory binding behind and triggers the refusal shown above.

A card ending cannot directly authorise a write
FigureA card ending cannot directly authorise a write

Selection, confirmation and execution are separate stages. The script isolates the last stage; the configured skill shows how the project reaches it.

Show what to inspect after a corrected card choice

Compare the returned service reference, memory selection and requested write reference. They must identify the same card. Inspect refusal effects and the service snapshot as well as status. Then test the dialogue gate separately: the question must describe the corrected card, not the previous selection.

A separate offline check on 8 October 2026 used the pinned Rasa SDK and ran the full suite without skips. Its actual summary was:

Ran 15 tests in 10.369s

OK

Project validation also printed:

validate_project: ok

To repeat that full check after installing the selected project:

uv run --locked python -m unittest discover -s tests -v
make validate

This validates tool imports and the project shape. It does not train a model, call a provider or establish spoken confirmation behaviour. Without Rasa installed, the earlier command can skip the SDK-dependent tool tests; read the summary rather than treating any OK as a full-suite result.

Try a small microphone acceptance check

The pinned README supplies the live setup. With Python 3.12 and uv, install only this example:

uv sync --locked --python 3.12
make env

Fill the local .env with RASA_LICENSE, OPENAI_API_KEY and DEEPGRAM_API_KEY. The project pins Rasa Pro 3.21.0.dev5. Its integrations.yml selects gpt-5.5-2026-04-23, Deepgram flux-general-en input and aura-2-andromeda-en output.

make validate
make train
make inspect

Allow microphone access. This optional path uses billed APIs and was not run for the receipts above. Try the proposed caller line “I am Nadia Okafor, born fourteen March nineteen eighty-six. Block my card ending five five zero two.” The next choice should distinguish debit from credit. Say “The credit card” and inspect that the selected reference is NB-CARD-0104 before agreeing to the confirmation. In another short check, correct that choice at confirmation and inspect the new selection and question before any write.

This is a manual acceptance check, not a voice reliability rate. The fixture stores cards in process memory. A real service needs durable, atomic handling and its own authenticated ownership checks; these results do not establish safety across restarts or concurrent requests.