Overview
Simulations are the core of SuperBryn’s testing. Each simulation is a real phone call between SuperBryn’s AI caller and your voice agent. The AI caller follows the scenario’s user perspective, while the system evaluates whether the agent meets the expected outcome.How to run
- Go to the Evaluators page
- Select one or more scenarios using the checkboxes
- Click Run Calls
- Configure the run:
- Runs per scenario: How many times to repeat each scenario (useful for consistency testing)
- Click Start to launch
What happens behind the scenes
Batch creation
SuperBryn creates a batch request that expands scenarios × runs-per-scenario into individual jobs.
Simulation context generation
For each job, the AI engine generates a simulation context: the adversarial caller prompt, TTS voice profile, noise settings (for Ring 6), SIP trunk routing, and call parameters.
Call dispatch
Jobs are enqueued and processed concurrently. For inbound agents, SuperBryn places a call to your agent’s phone number via LiveKit. For outbound agents, it triggers your agent’s outbound URL and answers the incoming call.
Call execution
The AI caller speaks with your agent following the scenario. The full conversation is recorded and transcribed.
Concurrency
Simulations run concurrently up to the agent’s configured concurrency limit. This prevents overloading your agent with more simultaneous calls than it can handle. Jobs beyond the limit are queued and processed as slots become available.Job statuses
| Status | Meaning |
|---|---|
| Pending | Queued, waiting to be processed |
| Running | Call is in progress |
| Completed | Call finished, results available |
| Failed | Call failed (agent unreachable, timeout, etc.) |