Documentation Index
Fetch the complete documentation index at: https://docs.superbryn.ai/docs/llms.txt
Use this file to discover all available pages before exploring further.
What are custom metrics?
Custom metrics let you define evaluation criteria specific to your use case. You write a natural-language prompt describing what to evaluate, choose an output format, and SuperBryn’s LLM evaluates every call against your criteria automatically.
Metrics apply to both simulation calls and production monitoring calls.
Metric types
| Type | Output | Example |
|---|
| Pass/Fail | pass, fail, or N/A | ”Did the agent verify the caller’s identity before sharing account details?” |
| Numeric Score | Score out of 10 | ”Rate the agent’s empathy on a scale of 1-10” |
| Text Response | Free-form text | ”Summarize the key topics discussed in this call” |
Creating a metric
- Go to Custom Metrics in the sidebar
- Click Create Metric
- Fill in the form:
| Field | Description |
|---|
| Name | A short label for the metric (e.g., “Greeting compliance”) |
| Evaluation prompt | Natural-language instructions for the LLM evaluator |
| Transcript scope | Which part of the conversation to evaluate |
| Response type | Pass/Fail, Numeric Score, or Text Response |
- Assign the metric to one or more agents
- Click Save
Improve with AI
Click Improve with AI to have the LLM refine your evaluation prompt. This rewrites your prompt to be more specific and produce more consistent results.
Transcript scoping
Control which portion of the transcript the evaluator sees:
| Scope | Description |
|---|
| Full transcript | Both agent and customer turns |
| Agent turns only | Only what the agent said |
| Customer turns only | Only what the customer said |
Assigning metrics to agents
Each metric can be assigned to one or more agents. Only assigned agents have their calls evaluated against that metric.
- Assign agents during metric creation or from the metric detail view
- Add or remove agents at any time without affecting historical results
- The same metric can be shared across agents in the same project
Analytics
The metric detail view shows aggregate statistics:
| Stat | Description |
|---|
| Total evaluations | Number of calls evaluated |
| Pass rate | Percentage of calls that passed |
| Passed | Total pass count |
| Failed | Total fail count |
Viewing results in call analysis
Custom metric results appear in the call analysis view for both simulation and monitoring calls. Each metric shows:
- The metric name
- The result (pass/fail, score, or text)
- An explanation of why the evaluator reached that conclusion
