AI product engineer · Banking & fintech (external)
One passing transfer refusal can hide a missing check
Replay a competing debit with each submission check removed, and see why a no-transfer assertion alone misses both missing guards.
Key takeaways (3)
- One refusal can conceal a missing check when another check fails too.
- Test stale revision and insufficient funds with separate inputs.
- The production ledger must own the reservation and resulting payment state.
The transfer is refused even after either of its two checks is removed. A test that checks only “nothing was sent” therefore misses both single-check deletions. Remove both and the fixture posts the transfer, leaving a negative balance.
These are actual results from four temporary versions of the fictional Northgate ledger’s submission function. No production ledger or money is involved:
{"available_after": "$340.00", "effects": 0, "reason": "stale_balance", "status": "blocked", "variant": "original"}
{"available_after": "$340.00", "effects": 0, "reason": "funds_not_reserved", "status": "blocked", "variant": "no_revision_check"}
{"available_after": "$340.00", "effects": 0, "reason": "stale_balance", "status": "blocked", "variant": "no_funds_check"}
{"available_after": "$-60.00", "effects": 1, "reason": null, "status": "submitted", "variant": "neither_check"}
Ask which refusal actually protected the account
Each arm starts with a fresh copy of the same fixture and prepares the same transfer. Removing the revision check changes the reason to insufficient funds. Removing the funds check leaves the revision refusal. Both arms still have no effect.
| Submission checks | Observed refusal | Transfer effect |
|---|---|---|
| Both present | Stale balance | None |
| Revision removed | Funds not reserved | None |
| Funds removed | Stale balance | None |
| Both removed | No refusal | Posted; fixture balance below zero |
Do not use the two-check mutation as a production fix. It is a way to examine the test’s meaning. Keep separate fixtures where the revision alone fails and where current funds alone fail. Otherwise either condition can cover for a missing check in this particular scenario.
Show the counterexperiment script
Run this from the pinned companion tests directory shown below: save it as counterexperiment.py, then run python3 counterexperiment.py. It changes no files and uses no model or provider.
"""Offline counterexperiment. Run from the pinned companion tests directory.
No source files are changed. Replacement functions exist only in this process.
"""
import inspect
import json
import textwrap
import test_guard as t
original = t.nb.submit_transfer
source = textwrap.dedent(inspect.getsource(original))
revision = 'account["revision"] == draft["revision"]'
funds = 'ledger.available(draft["from_account"]) >= draft["amount"]'
assert source.count(revision) == source.count(funds) == 1
for variant, drop_revision, drop_funds in (("original", False, False), ("no_revision_check", True, False), ("no_funds_check", False, True), ("neither_check", True, True)):
changed = source.replace(revision, 'True') if drop_revision else source
changed = changed.replace(funds, 'True') if drop_funds else changed
namespace = dict(original.__globals__)
exec(changed, namespace)
t.nb.submit_transfer = namespace["submit_transfer"]
flow = t.Flow()
flow.select("savings")
flow.prepare("bills", "400")
source_account = flow.ledger.drafts[flow.draft]["from_account"]
result = flow.submit()
print(json.dumps({"variant": variant, "status": result["status"], "reason": result.get("reason"), "effects": result["effects"], "available_after": t.nb.money(flow.ledger.available(source_account))}, sort_keys=True))
assert result["effects"] == (1 if drop_revision and drop_funds else 0)
t.nb.submit_transfer = originalRun two experiments on the same account
The first refusal concerns an old observation. The draft records the revision at which the balance was read. The fixture posts another debit after that read, so submission sees a different revision and refuses the draft.
The second refusal concerns available funds. Prepare the same amount again, without resetting the ledger. The revision is now current. The amount still exceeds the available balance, so the reservation fails. These are separate reasons to stop.
| Attempt in this fixture | Observation | Result |
|---|---|---|
| Original $400 draft | Old revision | Stale balance; no effect |
| Fresh $400 draft | Current revision, insufficient funds | Funds not reserved |
| Fresh $300 draft | Current revision and enough funds | Own-account transfer posted |
The smaller amount is a new request in the proof, not a suggestion the agent should make on the customer’s behalf. Keep control of the amount with the customer.
Make the ledger own both checks
Do not glue a balance read to an unconditional debit and call that atomic. A competing debit could land between them. The production ledger needs an endpoint that checks authority and reserves or posts the amount as one supported operation.
In the companion, the current revision and selected payee are checked before available funds. These lines are the decision at submission; they are not a production database transaction.
account = ledger.accounts[draft["from_account"]]
dest = ledger.destination(customer_id, draft["payee_ref"])
facts = {
"ledger_revision_current": account["revision"] == draft["revision"],
"payee_identity_confirmed": (
dest is not None and bool(selected_payee_ref) and draft["payee_ref"] == selected_payee_ref
),
}
if all(value is True for value in facts.values()):
# The reservation is the ledger's decision: available balance now, not
# as read. It is attempted only for a current draft to a confirmed payee.
facts["funds_reserved"] = ledger.available(draft["from_account"]) >= draft["amount"]
reason = evaluate(facts)
if reason:
return _blocked_submit(reason, draft, facts, ledger)
The refusal path also marks the draft void. Preparing a replacement needs a new draft and confirmation; an earlier yes cannot authorise new ledger state. The source’s blocked-submit function and the correction tests show that draft lifecycle.
Preserve the payment’s state after submission
The recovery tests distinguish a saved-payee transfer that is pending from an own-account transfer that has posted. A lost acknowledgement can be found by its original attempt identifier. Another fixture leaves the external scheme unknown and escalates to the payments ledger owner.
This costs a reconciliation path and a durable attempt record. The alternative is to let a timeout create a fresh transfer while the first may already exist. The fixture tests that distinction; it does not measure real settlement or concurrent account traffic.
Repeat the interleavings and payment recovery
Run these checks from the pinned companion project. They use Python’s standard library and make no model calls. The outputs below were recorded on 7 October 2026; elapsed times can differ.
git clone https://github.com/RasaHQ/rasa-community-resources.git
cd rasa-community-resources
git checkout 4aa0c4419dc193fef7a969c12d59edcf720f2606
cd examples/mantle-text-banking-transfer-gpt/tests
- Any system
python3 -m unittest test_guard.SubmissionTests test_guard.RecoveryTests test_guard.PayeeSelectionTests test_guard.WordingTests -q
----------------------------------------------------------------------
Ran 20 tests in 0.004s
OK
The companion tests contain the assertions for these cases. To print the additional fixture states yourself, run this script from the same directory:
Show the script that printed the fixture states
Paste it into a file such as trace.py, then run python3 trace.py. It prints selected fields from the actual tool results; it does not invent replies.
"""Run from the pinned companion project tests directory. Stdlib only."""
import json
import test_guard as t
flow = t.Flow()
flow.select("savings")
draft = flow.prepare("bills", "400")
print(json.dumps({"balance_read": draft["balance_as_read"]["available_balance"]}, sort_keys=True))
result = flow.submit()
print(json.dumps({"status": result["status"], "reason": result["reason"], "effects": result["effects"], "current_balance": result["current"]["available_balance"]}, sort_keys=True))
flow.prepare("bills", "400")
result = flow.submit()
print(json.dumps({"fresh_400_reason": result["reason"]}, sort_keys=True))
flow.prepare("bills", "300")
result = flow.submit()
print(json.dumps({"fresh_300_state": result["ledger_status"]}, sort_keys=True))Ask the payments team to reproduce these three attempts in its sandbox. Inspect ledger entries and original references, then have the conversation designer review the returned-state wording. Bind confirmation to the version read back when the customer changes a payment.
The complete fixture implementation defines the service state and remaining branches cited here.