Documentation Index
Fetch the complete documentation index at: https://docs.superbryn.ai/docs/llms.txt
Use this file to discover all available pages before exploring further.
Projects
A project is the top-level container in SuperBryn. Each project has an associated industry (Healthcare, Finance, Insurance, etc.) and contains one or more agents. Use projects to organize agents by product line, team, or use case.
Agents
An agent represents the voice AI you’re testing. Each agent has:
| Field | Description |
|---|
| Name | Display name for the agent |
| Phone number | The number to call (E.164 format) |
| Type | inbound (agent receives calls) or outbound (agent makes calls) |
| Language & Gender | Used for voice matching in simulations |
| Provider | The platform hosting your agent (Vapi, Retell, LiveKit, etc.) |
| Call flow | A directed graph representing the conversation structure |
| Policy & Guardrails | Rules and constraints the agent must follow |
| Ring config | Which rings are enabled for testing |
Call Flows
A call flow is a directed graph (JSON) that models your agent’s conversation structure. It consists of:
- START nodes - entry points
- PROCESS nodes - actions the agent takes (greeting, collecting info, etc.)
- DECISION nodes - branching points based on user input
- END nodes - call termination points
Edges between nodes are labeled with the conditions or transitions that connect them. You can build flows visually in the Flow Editor, import from JSON, or auto-generate from a text description.
Paths
A path is a specific traversal through the call flow graph - for example, “happy path where user places an order successfully” vs. “error path where user cancels mid-flow.” Scenarios are generated per-path, ensuring coverage across all branches.
The Ring System
SuperBryn’s core testing framework. Nine concentric rings of increasing difficulty, each targeting a different dimension of voice agent quality:
| Ring | Name | Tests |
|---|
| 0 | Setup & Wiring | Structural integrity and configuration |
| 1 | Does the Job | Functional correctness under expected inputs |
| 2 | Plays by the Rules | Policy and compliance adherence |
| 3 | Hard to Trick | Adversarial robustness (jailbreaks, prompt injection) |
| 4 | Speech Variations | Accents, slang, typos, ASR errors |
| 5 | Handles Real People | Interruptions, emotion, timing, disfluency |
| 6 | Works in Real World | Background noise, bad networks |
| 7 | Handles Chaos | Multiple failure modes combined |
| 8 | Stays Good Over Time | Regression and temporal stability |
Scenarios
A scenario is an individual test case. Each scenario contains:
| Field | Description |
|---|
| Ring ID | Which ring this scenario tests |
| Intent | What the simulated caller wants (from the agent’s perspective) |
| User Perspective | How the caller describes their situation (second person) |
| Expected Outcome | Step-by-step expected agent behavior |
| Path | Which call flow path this scenario exercises |
| Variant Type | (Ring 2) Which policy rule is being tested |
| Behavior Modifiers | Optional adjustments: language, interruption timing, noise, etc. |
Evaluators
An evaluator is a container that groups scenarios for a given agent. Each agent has a default evaluator. When you generate or create scenarios, they’re stored under the agent’s evaluator.
Simulations
A simulation run is a single execution of a scenario - one phone call between SuperBryn’s AI caller and your voice agent. Each run tracks:
- Status: pending → running → completed / failed
- Recording: Full audio of the call
- Transcript: Turn-by-turn conversation text
- Metrics: Latency, interruptions, silence, words per minute, speaking ratios
- Evaluation: LLM-powered step-by-step pass/fail against the expected outcome
Batch Requests
When you click Run Calls, SuperBryn creates a batch request that expands your selected scenarios × runs-per-scenario into individual simulation jobs. Jobs are processed concurrently through a queue system.
Knowledge Base
Knowledge base documents are reference materials you upload for your agent - product documentation, FAQs, menu data, policy documents. These are injected into the scenario generation prompts so the AI generates contextually accurate test cases.
Agent Versions
SuperBryn tracks agent configuration changes through versions. When you make material changes (call flow, policies, LLM provider, etc.), you can fork a new version. This creates an immutable snapshot, enabling you to compare test results across configuration changes over time.
Production monitoring
Production monitoring (also called observability) lets you ingest real production calls from your voice AI provider and analyze them automatically. SuperBryn supports ingestion via webhooks, API key sync, SDK, audio upload, and trace upload. See Production monitoring for details.
Labels
Labels are color-coded tags you attach to monitored calls. Use them to organize calls by campaign, time period, agent version, or any other dimension. Labels also scope monitor reports to specific subsets of calls.
Verdicts
A verdict classifies a monitored call using a confusion-matrix approach based on user satisfaction and agent performance:
| Verdict | Label | Meaning |
|---|
| TP | All Clear | User satisfied, agent performed well |
| FP | Blind Spot | User satisfied, but agent performed poorly |
| FN | False Alarm | User unsatisfied, but agent performed well |
| TN | Red Alert | User unsatisfied, agent performed poorly |
Custom Metrics
Custom metrics are user-defined evaluation criteria. You write a natural-language prompt, choose an output type (pass/fail, numeric score, or text), and SuperBryn evaluates every call against your criteria. See Custom metrics for details.
Roles
Each organization member has a role that controls their access level:
| Role | Access |
|---|
| Owner | Full control including member management |
| Member | Full access to agents and simulations |
| Viewer | Read-only access to all data |
