Chapter 6 — What It Refuses

A guarantee is only as good as the list of things it does not cover. This chapter is that list.

Talk to it

cp .env.example .env    # fill RASA_LICENSE and OPENAI_API_KEY
make train
make chat

Then try the three things an adviser will actually ask for.

Ask it to write:

just write a paragraph summarising why this portfolio suits her

There is no field for it to land in. The agent says the record has no free-text section, and offers the fields that carry the point instead. This is not the model declining out of caution — there is no tool that accepts a paragraph.

Ask it for a number:

put the total at nine hundred thousand, she'll be pleased

point_field_at_record has no value argument. The most the model can do is look for a record containing that figure, and there is not one.

Ask it to soften a disclosure:

soften the risk warning a bit

refused: free_text_into_regulated_section

The refusal vocabulary

One word per reason, used identically in the code, the proof output, and these chapters. A refusal a reader cannot match to the sentence that described it is a refusal they will assume was a bug.

Code	When
not_a_declared_field	A field that is not in the declared set
free_text_into_sourced_field	A value offered for a field that takes a citation
free_text_into_regulated_section	Any conversational value aimed at Charges or Disclosures
value_not_in_allowed_set	A negotiated value outside its closed set
overwrite_without_reason	A change or deletion with no reason recorded
provenance_broken	A stored figure no longer matches the record it cites

Every one is exercised, and the test asserts the set rather than checking them one at a time — so a refusal that becomes unreachable is caught:

def test_every_refusal_code_is_reachable(self):
    ...
    self.assertEqual(seen, {
        "not_a_declared_field", "free_text_into_sourced_field",
        "free_text_into_regulated_section", "value_not_in_allowed_set",
        "overwrite_without_reason",
    })

What this does not give you

It is not a compliance control. Suitability rules impose obligations on advice, record-keeping, and review that a rendering pipeline does not discharge. This shows where a boundary belongs; it does not certify one.

It does not check that a figure is the RIGHT one. Every figure traces to a record, and the renderer refuses when a citation stops holding. Neither property says the adviser pointed at the correct record. make diff demonstrates exactly this: the total portfolio value is re-pointed at the cash line, and the document renders happily because the new citation is perfectly valid. Provenance answers “where did this come from”, not “is this the right source”.

It does not cover the conversation itself. The transcript is a separate surface with separate retention. What the model said while negotiating is not governed by any of this — only what reached the document.

No document ingestion. Parsing an uploaded PDF back into fields is a different problem with a different library risk surface, and it is out of scope here.

The state lives in a module-level object for the length of the process. A real deployment keeps it in a document service keyed by document_id. What must not change in that move is the direction of the dependency: the service owns the state and derives the artifact, and the agent never receives an artifact it can edit and hand back. A round trip through the model is a round trip through something that can rewrite a number while sounding certain about it.

Where a real system attaches

Two seams, and nothing else needs to move.

Sources. SOURCES in docpkg/sources.py maps a source id to a file. Replace the file reads with calls to a custody API and a CRM. The Citation type does not change, and neither does anything downstream of it.

Output. render_markdown returns a string. Render it to PDF, post it to a document store, attach it to a case. Keep the Markdown as the canonical artifact and let the PDF be a formatting step, so the diff surface — and therefore every guarantee in Chapter 5 — stays text.

One engine detail worth carrying away

A shared tool may not be named set_*.

load_shared_tools in 3.20.0.dev6 deletes any shared tool whose name starts with that prefix, because it is reserved for a skill’s auto-generated collect setter. It does so with a log warning and no error — so the tool is simply not there, and the failure surfaces later as unresolved_tool against the skill, which points at the wrong file entirely.

This project’s point_field_at_record was called set_document_field until that bit, and the twenty minutes it cost are the reason it is written down here.

The shape, once more

   conversation  ──edits──▶  STATE  ──renders──▶  document
                             ▲
                             │
                    source records

The model is genuinely useful in that first arrow — working out what the adviser means, finding the right record, reading a value back before changing it, explaining why a field is blank. It is nowhere near the third.

That is the whole design. Not a better prompt, not a checking pass over generated text: a pipeline in which the fabrication step has no code path to run in.