Check the CRM instructions before and after an MCP move

Ora’s three skills identify a customer, check their support tickets and record an interaction. You are moving their tools from local Python calls to a server using Model Context Protocol (MCP), which lets the assistant discover and call remote tools. The service tasks stay the same, so you want to catch any accidental change to their instructions.

This chapter compares the instruction text directly. Tool bindings and schemas can change during the move; the comparison checks the skill bodies, rather than treating them as the whole model input.

Read the claim as an assertion

“The same three skills can keep their instructions” means: after the swap, the body of every skill.md is byte-identical to what it was before. Only the frontmatter may differ, and only in one key.

That is precise enough to check with difflib and a string comparison:

rest_body = read(f"skills/{skill}/skill.md").split("---\n", 2)[-1]
mcp_body = read(f"mcp_variant/skills/{skill}/skill.md").split("---\n", 2)[-1]
assert rest_body == mcp_body

Splitting on the second --- throws away the YAML frontmatter and keeps the instructions. If the two halves match, those skill instructions are unchanged. The model may still see different tool schemas and results, so this check does not establish identical behaviour.

Get it failing first

A proof that has never failed has not been tested. Before trusting the script, break the thing it checks and confirm it notices. Edit one line of prose in the MCP copy of check_tickets — add a clause to the first instruction:

Report the caller's tickets, and always list them even when not identified.

Then run the proof:

make mcp-prove

  ✗ check_tickets: only import_tools differs     prose changed: ["Report the
    caller's tickets.", "Report the caller's tickets, and always list them even
    when not identified."]
  ✗ check_tickets: instruction body byte-identical bodies differ

  2 check(s) failed:
    - check_tickets: only import_tools differs
    - check_tickets: instruction body byte-identical

Exit code 1. It prints both versions of the line, so the diff is visible without opening an editor. Undo the edit and it goes green again.

Do the same with the other two failure modes before moving on, because they are the ones you will actually hit:

Rename the server under mcp_servers: to hubspot_crm_TYPO:

  ✗ parse_mcp_servers accepts the file           servers: ['hubspot_crm_TYPO']
  ✗ server url is loopback http                  no server named 'hubspot_crm'
  ✗ no skill imports an unconfigured server      missing from integrations.yml:
    ['hubspot_crm']

Mistype a tool name in an import_tools: line:

  ✗ check_tickets imports list_open_tickets      reference not found in skill.md

Three ways to break it, three different red lines, each naming the file to open. That is what makes it a proof rather than a smoke test.

Why this runs with no credentials

The script starts both mock servers itself and tears them down afterwards:

for script, host, port in (
    ("scripts/mock_hubspot.py", CRM_HOST, CRM_PORT),
    ("scripts/mcp_crm_server.py", MCP_HOST, MCP_PORT),
):

The first is the stand-in CRM the REST tutorial already shipped. The second is a new MCP server that sits in front of it and speaks MCP to Mantle. Both bind to 127.0.0.1, so no packet leaves the machine and there is nothing to pay for and nothing to sign up to.

That matters more than convenience. A proof requiring an account is a proof most readers never run, and one that goes stale the first time the account expires.

What the script actually checks

Six groups, eighteen assertions. In order:

1. Skill prose is unchanged — the claim itself.
2. parse_mcp_servers accepts the new integrations.yml.
3. Each mcp/<server>:<tool> reference parses, and get_bare_tool_name returns the name the model will see.
4. Every imported server id is configured.
5. The MCP server really exposes the three tool names, over a live session.
6. A tool call over MCP returns the same customer the REST tool returned.

Groups 2 to 5 are the checks the engine performs at model load, run early. Group 6 is the one that says the wire works. Group 1 is the one this tutorial is about.