When the caller ID match is not the caller

Northgate Bank has left a missed call on Jordan’s phone, and Jordan’s teenager rings it back. The voice agent matches the calling number to Jordan’s profile and opens the call:

Agent: Good afternoon, Jordan! Always great to see a Premier member who’s been with us since 2019. How can I help you today?

Caller: Er, what’s gone into the account recently?

Agent: Want your usual Everyday Checking, Jordan?

Caller: Yeah.

Agent: The only money in recently is a payroll deposit of $2,150.00 on 22 July.

Within a few seconds, someone who is not Jordan has heard Jordan’s name, membership tier, the year Jordan joined, which account Jordan uses most, and the size of Jordan’s pay. Nobody confirmed that the caller was Jordan. The call came from Jordan’s number, and the agent treated that as the answer.

The script is illustrative. Northgate Bank and the household are fictional, and we have placed the session-start pattern’s demo data in them. The agent’s first line is the pattern’s greeting template filled with its demo profile (skills/default_session_start/tools.py, lines 17–25). The second is the example offer in skills/view_transactions/skill.md, line 16. The last is our wording over the demo rows in skills/view_transactions/tools.py, lines 20–26, where the payroll deposit is the only credit to Everyday Checking.

Nothing in this exchange is a bug. Each line does what the pattern was written to do. The problem is that the pattern was written for a customer who has already signed in, and a phone number is not a sign-in.

Where the greeting comes from

The companion pattern is patterns/session-start-personalization in RasaHQ/rasa-community-resources. Its README names Daksh Varshneya as author and Rod Rivera as assessor (README.md, lines 3–5). The session-start personalisation tutorial builds the same pattern for a signed-in customer. This guide reads the pattern’s files at revision 69e27b6.

The session opener is a skill. Its whole body is two steps, run in order (skills/default_session_start/skill.md, lines 8–14):

:::ordered_block id=main
steps:
  - id: identify
    execute_tool: get_customer_profile
  - id: greet
    action: utter_greet
:::

identify fetches a profile and greet speaks. Nothing sits between them to ask whether the profile belongs to the caller. The README says the engine fires this skill “before the user types anything” (README.md, lines 31–32), so the caller cannot correct anything until the greeting has already been said.

The greeting is a response template (skills/default_session_start/responses.yml, lines 4–11):

responses:
  utter_greet:
    - text: >-
        Good {session.project.time_of_day}, {session.project.customer_name}!
        Always great to see a {session.project.customer_tier} member who's been
        with us since {session.project.member_since}. How can I help you today?
      metadata:
        rephrase: true

Name, tier and tenure go into the first two sentences, with no condition on any of them. rephrase: true changes the wording, and the file’s own comment says the model smooths it “while keeping the facts” (lines 2–3).

Caller ID is not verification

The profile lookup is hard-coded in the demo. The comment above it says what a real deployment would use instead (skills/default_session_start/tools.py, lines 15–16):

# Fictional signed-in customer for the demo. A real deployment would resolve this
# from the authenticated session / channel identity rather than a hard-coded row.

This guide is about the slash in that comment. The README calls the mechanism “channel-agnostic” (README.md, line 16): it personalises on whatever identity the channel provides, and nothing in it checks whether the channel is right.

What follows is our reasoning, not something we tested. A signed-in app session means someone got past a login. Caller ID reports the number a call claims to come from. That number can be spoofed, and even a real number can be in someone else’s hand, because phones get shared, handed to children and left on desks. We have no figures for how often either happens, and the design does not need any: one call like the one above is enough to design against.

The pattern does have guards, and they are good ones. Look at what each one protects against:

Each guard keeps the model from inventing facts about the customer. None of them checks whether the right person is hearing those facts. In the failure script every fact was real and correctly looked up, and it went to the wrong listener.

What to say before caller verification

The stance: a channel match earns a first name, asked as a question, and nothing else. Tier, tenure, the usual account and anything about money wait until the caller confirms the match. Anything that could cause harm if the wrong person heard it also waits for your service’s own verification step, which confirming a name is not.

This has a cost. Jordan, calling from Jordan’s own phone, hears a less warm opening and answers one extra question. Some product owners will reject that trade, and it is a fair argument to have. The teenager’s call shows the other side of it: not a colder greeting, but someone else knowing the customer’s pay.

Even the name has a cost. Saying “Jordan” confirms to whoever is holding the phone that Jordan has a relationship with Northgate Bank. For a bank returning its own missed call, that is usually already known. For a service where the relationship itself is sensitive, drop the name too and greet without it.

The second row has its own cost. Anyone willing to say “yes” falsely, including someone calling from a spoofed number, hears the tier and tenure. If that is too much for your service, move them to the third row.

The third row is left open on purpose. What counts as enough verification is a decision for your service’s risk and compliance owners, not for this guide and not for the greeting. The guide to approving agent actions covers the evidence they should ask for. The rules here are an assumption made for this exercise, not a legal or regulatory standard.

What the diagram adds to the table is where each branch ends. A “no” ends the profile’s part in the call. A “yes” on its own also ends in a red box: accounts and money are only reached through the verification step.

What the model knows is not what the greeting says

A designer who owns responses.yml can cut the tier from utter_greet and reasonably expect the tier to be gone. The spoken line is only one of the places a profile fact goes. We read the engine source in the pinned Rasa Pro 3.20.0rc1 wheel to find the others. Every wheel path in this guide is under rasa/mantle/:

The third row is the one that matters. The pattern’s tool writes the tier, the joining year and the usual account into project memory (skills/default_session_start/tools.py, lines 50–57). The engine then puts every set project field into the model’s system prompt under this heading (prompts/templates/project.jinja2, lines 1–2):

### Project Memory
These facts are already set for this session and will remain so throughout this session.

So a name-only greeting still leaves the model holding Jordan’s tier and usual account, labelled as settled facts, on the next free-form turn. Tiering the greeting is half the job; the other half is keeping those facts out of project memory until the caller confirms. The rephrase call is the exception: its prompt carries no project memory, and the rephrase task tells the model “do not add information” (prompts/templates/rephrase_task.jinja2, lines 2–3). A template that says only the name gives the rephraser no tier to add.

To see what this pattern writes into memory, run this from patterns/session-start-personalization:

At revision 69e27b6, it printed the following (the output is saved as a receipt for this guide):

skills/default_session_start/tools.py:50:        context.memory.set("customer_name", _CUSTOMER_PROFILE["preferred_name"])
skills/default_session_start/tools.py:51:        context.memory.set("customer_tier", _CUSTOMER_PROFILE["tier"])
skills/default_session_start/tools.py:52:        context.memory.set("member_since", _CUSTOMER_PROFILE["member_since"])
skills/default_session_start/tools.py:53:        context.memory.set("time_of_day", time_of_day)
skills/default_session_start/tools.py:54:        context.memory.set("default_account_id", _CUSTOMER_PROFILE["default_account_id"])
skills/default_session_start/tools.py:55:        context.memory.set(
skills/view_transactions/tools.py:74:    context.memory.set("selected_account_label", account["label"])

Line 55 is default_account_label, split over three lines, which is why grep shows only its opening. The six session-start names are the six fields declared in the root memory.yml (lines 4–26), so they are project memory. selected_account_label is declared in skills/view_transactions/memory.yml and belongs to that skill. On your own project, the same grep can miss a tool that names its context differently, a write made through a helper outside skills/, or a field name built at run time. Treat it as a starting list and read memory.yml alongside it.

Invite the correction, don’t assert the name

A confirmation can be phrased so that it assumes the answer. An illustrative example:

Agent: Good afternoon. Is this Jordan?

It is a question, but the only easy answer is “yes”. Saying “no” means contradicting the agent, and a teenager who wants to know about the deposit has every reason to say “yeah” instead.

Here is the line this guide recommends, again illustrative:

Agent: Good afternoon, you’ve reached Northgate Bank. Am I speaking with Jordan, or someone else using Jordan’s phone?

Three things make it work:

1. It names the other possibility: “or someone else” makes “no” an ordinary answer instead of a correction.
2. It uses the first name only: the legal name, a surname or a title add nothing to the question and tell the listener more.
3. It says nothing about the account: nothing in it relies on the match being right, so if the match is wrong, nothing has leaked yet.

If the caller replies “Who’s asking?” or says nothing, repeat the bank’s name and the question. Never fill the pause with a profile fact.

A “yes” moves the call to the second row of the table. It is still not verification. The confirmation filters out honest mismatches, like the teenager who would say “no, it’s their son” when asked plainly. Someone willing to lie has to be stopped by the verification step.

When the match turns out to be wrong

Here is the recovery, as an illustrative script:

Caller: No, it’s their son.

Agent: Thanks for telling me. I can answer general questions about Northgate Bank. For anything about an account, the account holder needs to call us themselves.

Caller: Can you just tell me if the money came in?

Agent: I’m sorry, I can’t. For anything about an account, the account holder needs to call us themselves. I can answer general questions about Northgate Bank.

Four rules sit behind those lines:

• Thank the caller once: the caller did the right thing, and a long apology makes a small thing seem like an incident.
• Repeat nothing from the profile: no account name, no tier, and after the “no”, not even Jordan’s name.
• Offer only what needs no identity: general questions about the bank. Anything that would tell the caller about the account, such as where to check a payment, is out.
• Give the same answer when asked again: the second request is the one that tests the design, so the second reply repeats the first.

The harder part is what the agent says ten turns later. The pattern exists to reuse the profile in every skill. agent.yml instructs the model to use the customer’s name “warmly at the greeting and occasionally afterwards” (lines 18–22), and view_transactions instructs it to offer the usual account whenever session.project.default_account_id is set and no account has been chosen yet (skills/view_transactions/skill.md, lines 15–16). After a “no”, those instructions still stand, and they point at Jordan.

That is the requirement to hand your engineer, together with the stage table: the unconfirmed match must live somewhere it can be withdrawn, and only confirmed facts go into project memory. Nothing in this pattern implements that yet: at 69e27b6 it greets straight after the lookup and has no confirmation step. This guide specifies what that step should do; building it is the engineer’s job.

Rehearse the wrong listener

This is a proposed rehearsal, not a result we have. Run it with a colleague playing the caller once the engineer has a build with a confirmation step:

Seed a profile and call as someone else

Point the lookup at a test profile, then have the colleague call from that number and answer the confirmation question with “no”.

Push for the account twice

Ask about recent payments, then ask again in different words. Both answers should be the general-questions refusal, with no account name in either.

Ask something unrelated, then something account-shaped

A few turns later, ask a general question, then one that would normally trigger the usual-account offer. The agent should not use Jordan’s name or offer Jordan’s account.

Read the prompt for the last turn

Ask the engineer to show the system prompt for that turn. Jordan’s name, tier, tenure and usual account should not appear under Project Memory.

The order matters: the later turns only test something once the “no” has been given. Passing this rehearsal shows that one conversation went right. It does not show the model will behave the same way on every call, and it does not test a caller who lies.

Questions a designer will ask

Won't the extra question annoy customers who really are Jordan?

It costs one short turn. Offering the name shows the agent already has a good guess, which feels different from a cold “who is this?”. What you cannot do is drop the question because the answer is usually yes, because the calls where the answer is no are the ones it exists for.

What if the person is a joint account holder, or authorised to act for Jordan?

Then “the account holder needs to call us themselves” is the wrong line: the person may have every right to the account. The pattern cannot tell, because its demo profile has one customer and no field for anyone else (skills/default_session_start/tools.py, lines 17–25). Treat them as a new caller to identify through your service’s own process, not as a variant of Jordan’s match, and write the line for that route with whoever owns it.

Could the engineer mark the profile fields as pii instead?

In the wheel, a field with pii: true appears as [set] instead of its value in the Project Memory section (prompts/memory_lines.py, lines 43–45) and in @memory tokens in skill instructions (prompts/memory_prose.py, lines 77–128). That keeps the tier out of those two places. We have not checked how a pii field renders in a response template such as utter_greet, or what a tool the model calls later returns. It does not make the field withdrawable, and it does nothing for the name, which the greeting has to say.

Where this guide stops

This is a design argument, not a measurement: we have no figures for spoofed numbers or shared phones. It does not define verification, which belongs to your service, and it covers calls the customer places, not calls the agent makes, where the opening line has to say who is calling first. Everything in it comes from the pattern’s files at 69e27b6 and the engine source in the 3.20.0rc1 wheel, not from a phone run. The rehearsal above is how you find out what your build does with it.