Skip to content
RasaGet Developer Edition

tutorial

Chapter 0 of 10

Build a Voice AI Agent with Rasa Skills

by Rod Rivera Published

Flagship tutorial for the Skills architecture and Deepgram voice — build Atlas, a Horizon Travel assistant, from first skill to progressive control.

What you will build

By the end of this tutorial you will have a speaking travel agent named Atlas that can:

  • answer trip FAQs from reference material
  • list itineraries from a demo database
  • look up flight status with hard tool gates
  • file a lost-baggage report with ordered steps and verbatim notices
  • change bookings by composing authentication and lookup skills

Speech in and speech out use Deepgram end to end (Flux ASR + Aura TTS) through the Rasa Inspector.

Who this is for

You do not need prior Rasa experience. You should be comfortable with:

  • a terminal and git
  • editing YAML and Markdown files
  • basic Python function signatures

Mental model: Skills, not flowcharts

Older Rasa assistants often centred on flow YAML and a large domain file. The Skills architecture is different:

IdeaMeaning
SkillA folder with skill.md instructions, plus optional tools and references
MantleThe orchestrator that activates skills from their descriptions
Progressive controlStart with prose; add runtime guarantees only where mistakes are expensive
MemoryShared state tools write and constraints read

There are no flowcharts required to start. The LLM interprets instructions. Where you need a guarantee, you add a frontmatter constraint or an ordered block — enforced by the runtime, not suggested to the model.

Those guarantees form a ladder, and the useful thing about it is how few skills climb it:

A ladder of runtime guarantees, each bought with one config key. At the bottom, prose only — name and description — where trip_faq, intro and goodbye stop. Above it: import_tools so answers come from data; tool_constraints with requires, which removes a tool from the schema until its data exists, where flight_status stops; an if marker so the wrong branch is never in context; utter with on activate for exact wording; requires_confirmation for a human check before an irreversible act; and at the top an ordered_block, which guarantees order itself. Only report_baggage and change_booking reach the top, and the rungs are cumulative, so those two carry nearly all of them at once. All four condition levers share one syntax: session, then a scope, then a field.

Start with prose. Add a rung only where a mistake is expensive — most skills never need one.

How voice fits

Traveler speaks
    → Deepgram ASR (speech to text)
    → Mantle + Skills
    → Deepgram TTS (text to speech)
    → Traveler hears Atlas

You configure Deepgram once in integrations.yml. Skills stay domain logic. That separation is intentional: the same skills can serve chat or voice.

Companion repository

All runnable code lives in:

github.com/RasaHQ/rasa-community-resources/tutorials/rasa-voice-agent-tutorial

Clone the monorepo, open that tutorial tree, run make install && make env && make verify, and keep two windows open: this tutorial and your editor.

Paste-ready chapter files are under tutorial/snippets/ in that tree.

Chapter map

ChTopic
1Scaffold a voice agent
2First skill (FAQ)
3First tool (itinerary)
4Tool constraints
5Scoped instructions
6Verbatim responses for voice
7Ordered blocks (baggage)
8Compose skills
9Voice deep dive
10Coding agents and the flywheel

Continue to Chapter 1 — Scaffold.