How to set the embedding model for Rasa Mantle references

You want your Rasa Mantle agent to embed its reference documents with a model you chose, perhaps a local one. The sample project has an embeddings model group in endpoints.yml, so you point it at a local model. This is an excerpt of that change in our run:

   - id: openai-embeddings
     models:
-      - provider: openai
-        model: text-embedding-3-large
-        api_key_env: OPENAI_API_KEY
+      - provider: ollama
+        model: all-minilm
+        api_base: http://localhost:11434

You retrain, and training passes. But Rasa still asks for the default model. We ran that training with the default provider’s address pointed at a local recorder, a small server that logs each request. This is our run harness’s summary of what the recorder received, trimmed to its first lines:

exit code: 0
requests the OpenAI stand-in received: 1
{"method": "POST", "path": "/v1/embeddings", "model": "text-embedding-3-large", "inputs": 3, "authorization_header": "present"}
    input head: "# Fixture data \u2014 entirely fictional\n\nEvery person, company, account, and event i"
    input head: "# Horizon Travel FAQ\n\n## Baggage\n- Economy includes one cabin bag up to 7 kg and"
    input head: "## Contact\n- Horizon Travel phone support is available daily from 7am to 10pm UK"

Training passed. It made one request for text-embedding-3-large, the default model, carrying all three chunks of reference text. That request went to our recorder at the default provider’s address. Nothing in the training output named an embedding provider.

The fix is to name the embedding model under references: in agent.yml, and declare that model group in integrations.yml. That matters if your reference documents must stay with a model you chose, or if your build host cannot reach the default provider.

Why the endpoints file setting changes nothing

A few terms first. Reference documents are the Markdown files in a references/ folder, either at the agent’s root or inside a skill. The sample uses a skill’s folder. The agent searches them to answer questions. When you run rasa train, Rasa splits them into chunks and turns each chunk into a vector with an embedding model. That builds the search index.

A model group is a named block of model settings: the provider, the model and the key. A Mantle agent keeps its own model groups in integrations.yml. Other Rasa features read model groups from endpoints.yml.

Mantle picks the embedding model from one setting: embeddings, under references: in agent.yml. The value is the id of a model group. Mantle looks that id up in integrations.yml only. When the setting is empty, Mantle uses its built-in default, which is OpenAI’s text-embedding-3-large.

The sample’s agent.yml leaves the setting empty. Its endpoints.yml declares a group called openai-embeddings. The heading above it says these groups are for features set in endpoints.yml, and nothing in the project refers to this one. It lists text-embedding-3-large too. So training the unedited sample looks as if the group works.

1. One setting in agent.yml decides the embedding model.
2. A named group is looked up in integrations.yml only.
3. The embeddings group in endpoints.yml plays no part in the reference index.

The function that chooses the model

This is the whole function from the Rasa Pro 3.20.0 package, with numbered markers added as comments:

def build_references_embedder(
    agent: AgentSpec, model_groups: list[dict[str, Any]]
) -> "EmbeddingClient":
    """Resolve the references embeddings model group into an embedding client.

    ``agent.references.embeddings`` names a ``model_groups`` id; empty → the
    OpenAI default (mirrors ``EnterpriseSearchPolicy``). The resolved config is
    handed to ``embedder_factory``, whose client ``FAISS_Store`` consumes
    directly.
    """
    group_id = agent.references.embeddings
    embeddings_config: Optional[dict[str, Any]] = None
    if group_id:
        # mantle resolves model groups only from the packaged integrations.yml.
        # resolve_model_client_config treats an empty model_groups list as falsy
        # and silently falls back to endpoints.yml, which would resolve against
        # the wrong (or missing) config. Fail closed with a clear message instead.
        if not model_groups:
            raise InvalidConfigException(
                f"references.embeddings names model group '{group_id}', but no "
                f"model_groups are defined in {V2_INTEGRATIONS_FILE}. Define the "
                f"model group there, or clear references.embeddings to use the "
                f"default embedder."
            )
        embeddings_config = resolve_model_client_config(
            {MODEL_GROUP_CONFIG_KEY: group_id},
            "references",
            model_groups=model_groups,
        )
    return embedder_factory(embeddings_config, DEFAULT_EMBEDDINGS_CONFIG)

The default is documented. The setting’s docstring in Rasa says that empty means the built-in OpenAI embeddings. What you do not get is a log line. The one line Rasa writes about the index gives its path and the document and chunk counts. It names no provider or model.

What Rasa checks, and what it cannot see

Rasa’s project validation checks the setting, but only when it has a value. We trained the unchanged sample and four variants of it. Two of the variants change two files, agent.yml and integrations.yml:

An unknown group name fails loudly, at validation. This is the first of the two ERROR lines from that run, unedited:

2026-09-29 11:21:50 ERROR    rasa.mantle.validation  - {"event_info": "'references.embeddings' in 'agent.yml' references model group 'no-such-group', which is not declared in 'integrations.yml'.", "docs_url": null, "event": "mantle.validation.config.unknown_model_group", "level": "error"}

How to name the embedding model

The change touches two files in your project’s root folder: agent.yml and integrations.yml. The diffs below are trimmed excerpts of our run against the sample.

         temperature: 0.0
 
+  - id: local-embeddings
+    models:
+      - provider: ollama
+        model: all-minilm
+        api_base: http://localhost:11434
+
 channels:

 references:
+  embeddings: local-embeddings
   instructions: |
     Answer from retrieved FAQ snippets only. Keep answers short for voice.

With these two edits, training passed and the recorder received no request. We did not record what the local model server received.

If you want the default provider, name a group for it in the same way. Then the choice is written down, and validation checks the name.

How to check the fix works

There are two checks. The first reads your settings and costs nothing. The second runs a training and records what is sent.

Read the setting

This one-line program prints the setting and any other keys under references:. Run it in your project folder. The backslashes join the lines, so it is the same program we ran:

uv run --frozen python -c "import yaml; \
r = yaml.safe_load(open('agent.yml')).get('references') or {}; \
print('references.embeddings:', r.get('embeddings') or '(empty: OpenAI default)'); \
print('other keys under references:', \
sorted(set(r) - {'instructions', 'embeddings', 'chunking'}) or 'none')"

Its output in the sample, in the misspelt copy and in the fixed copy, trimmed to remove uv’s install notices:

$ cd examples/mantle-voice-agent
references.embeddings: (empty: OpenAI default)
other keys under references: none

$ cd runs/c-misspelt-key
references.embeddings: (empty: OpenAI default)
other keys under references: ['embedding']

$ cd runs/d-named-local
references.embeddings: local-embeddings
other keys under references: none

The label “(empty: OpenAI default)” is ours, based on the function above. This check only reads YAML. To see what training sends, use the recorder.

Record what training sends

The recorder stands in for the OpenAI API on your machine. It logs the path, the model, the number of inputs and the first 80 characters of each input. It logs whether an Authorization header was present, never its value. It answers with fake vectors and forwards nothing.

Save the script below as openai_standin.py and start it on a free port. We use 8765 here; change it if that port is taken:

python openai_standin.py 8765 requests.jsonl

Then, in your project folder, train with a placeholder key and the provider’s address pointed at the recorder:

Rasa’s .env loaders in 3.20.0 do not replace a variable that is already set. So the placeholder and the recorder’s address win over any values in your .env.

Each line in requests.jsonl is one request, with a model field. If you named a group from another provider, as we did, any line for the embeddings path means your setting is not in use. If it stays empty and training passes, no embedding request went to the default provider’s address.

If you named an OpenAI group yourself, its requests may reach the recorder too. Then read the model field. To test the setting, name a model you can tell apart from the default.

"""Local stand-in for api.openai.com. Records each request (path, model, input count,
first 80 characters of each input, auth header present or not; never its value) and
returns deterministic fake embeddings. Nothing is forwarded anywhere."""
import http.server, json, sys, datetime, hashlib
PORT, LOG = int(sys.argv[1]), sys.argv[2]
class H(http.server.BaseHTTPRequestHandler):
    def do_POST(self):
        body = json.loads(self.rfile.read(int(self.headers.get('content-length', 0))) or b'{}')
        inputs = body.get('input', [])
        inputs = [inputs] if isinstance(inputs, str) else inputs
        rec = {'at': datetime.datetime.now().isoformat(timespec='seconds'), 'method': 'POST', 'path': self.path,
               'authorization_header': 'present' if self.headers.get('authorization') else 'absent',
               'model': body.get('model'), 'inputs': len(inputs),
               'input_heads': [str(i)[:80] for i in inputs]}
        with open(LOG, 'a') as f: f.write(json.dumps(rec) + '\n')
        if self.path.rstrip('/').endswith('/embeddings'):
            def vec(t):
                h = hashlib.sha256(str(t).encode()).digest()
                return [((h[i % 32] / 255.0) - 0.5) for i in range(1536)]
            out = {'object': 'list', 'model': body.get('model'), 'data': [{'object': 'embedding', 'index': i, 'embedding': vec(t)} for i, t in enumerate(inputs)],
                   'usage': {'prompt_tokens': 0, 'total_tokens': 0}}
            data, status = json.dumps(out).encode(), 200
        else:
            data, status = json.dumps({'error': {'message': 'stand-in serves /embeddings only'}}).encode(), 404
        self.send_response(status); self.send_header('content-type', 'application/json')
        self.send_header('content-length', str(len(data))); self.end_headers(); self.wfile.write(data)
    def do_GET(self): self.do_POST()
    def log_message(self, *a): pass
http.server.ThreadingHTTPServer(('127.0.0.1', PORT), H).serve_forever()

We used a recorder rather than an invalid key. A failed call only tells you that something was tried. The recorder lets training finish and keeps the request itself: the model, the number of chunks and the start of each one.

The same check in other stacks

The test does not depend on Rasa. Four conditions hid the problem here, and any stack can have them:

• An empty setting means a hosted default.
• Validation checks only values that are present.
• The settings schema ignores unknown keys, so a typo reads as empty.
• An unused config block names the same model as the default.

Point the provider’s address at a recorder and change the setting to a model you can tell apart from the default. Then train or index. If the recorder still sees the default model, the setting you changed is not the one in use.

Trade-offs

• You maintain one more model group in integrations.yml.
• Changing the embedding model means a retrain.
• By Rasa’s source, loading the index for a served agent calls the same function. So questions would be embedded by the same setting’s model. We did not serve the model to confirm this.

Limits

• One sample project on Rasa Pro 3.20.0. Other releases may differ.
• The recorder answered in place of OpenAI. No request reached OpenAI itself, and the fake vectors say nothing about search quality.
• We tested training only, not a served agent.
• In the fixed run, we recorded that no request reached the recorder and that training passed. We did not record what the local model server received.
• The run results were captured on 29 September 2026.