> ## Documentation Index > Fetch the complete documentation index at: https://docs.tracks.alpic.ai/llms.txt > Use this file to discover all available pages before exploring further. # Reading results > What verdicts, scores and pass rates mean, why some runs are not evaluated, and whether agents used your tools Once an execution's runs end, each one is judged on its own: one yes-or-no answer per evaluation of its scenario, combined into a score and a verdict. The execution then adds them up into a pass rate. ## Verdict and score Each evaluation of a run is **passed** or **failed**. From those: * The **score** is the weighted share of evaluations passed, out of 100: the weights of the passed evaluations divided by the weights of all of them. * The **verdict** is **passed** or **failed**, by the scenario's pass rule: every evaluation passed (the default), or a score at least equal to its passing score. For example, a scenario with a passing score of 80 and three evaluations: | Evaluation | Weight | Result | | - | - | - | | The app builds | 2 | Passed | | The toggle switches themes | 2 | Passed | | The choice survives reload | 1 | Failed | The score is (2 + 2) / (2 + 2 + 1) = 80, so the run **passed**. Under "every evaluation passed", the same run **failed**. The score is never rounded before the pass rule is applied. The web app shows it only for scenarios with a passing score. ## Pass rate The pass rate is the runs that passed out of the runs judged, as a whole percent. > [!NOTE] > > Only judged runs count. A run that was **not evaluated** says nothing about the agent's work, so it counts neither as > passed nor as failed. Say an execution has 10 runs: 6 passed, 2 failed, and 2 reached their time limit. Its pass rate is 6 out of 8, 75%. The 2 others are listed apart: if many runs end the same way, fix the cause, here a time limit too short, and launch again. ## Not evaluated A run is not evaluated when it can't be judged. Only runs that ended `completed` are judged. | Reason in the API | Shown in the web app as | What to do | | - | - | - | | `never_started` | The slot's state, such as Refused or Canceled | See [refused and not started](#refused-and-not-started). | | `no_evaluations` | No evaluations | Add evaluations to the scenario, then launch again. | | `run_not_completed` | How the run ended, such as Time limit reached or Failed | Open the run: raise the time limit or budget, or fix the setup. | | `agent_failed` | Agent error | The agent stopped with an error, for example when its model provider couldn't be reached. | | `evidence_too_long` | Too much to judge | The run recorded more than the judge can read. Ask for less, or split the scenario. | | `too_many_attempts` | Judging failed | Tracks retried judging on its own and gave up after 20 failed attempts. | ## Refused and not started A slot whose run hasn't started yet is **not started**. If its execution is canceled first, it never starts. A slot is **refused** when its run couldn't start when its turn came. It is never retried. The web app says why: * your organization's balance was \$0, or your organization is blocked; * the agent's model is no longer offered; * one of its MCP servers was disconnected, lost its bearer token, or its connection can't be used. Fix the cause, then launch a new execution. ## Why repetitions matter The same agent, on the same prompt, does different things from one run to the next. With one run per combination, a single lucky or unlucky run decides the result. Set **Repetitions** to 3 or more when you compare agents or variants. Each one adds a run per scenario, variant and agent, so check the maximum spend before launching. Treat a gap of one or two runs between groups as noise, not as a winner. ## In the web app Open an experiment, then one of its executions. **1. Read the summary** The top shows the **Pass rate** with how many judged runs passed and why others were not judged, the **Runs** ended so far, and the **Total spend** against the maximum. Below it, bars compare pass rates **By agent tools**, **By agent**, **By repository** and **By scenario**, for each of them that has more than one value. **2. Group the results** In **Results**, pick what goes in **Rows** and **Columns**: scenario, agents, repository or agent tools. Each cell shows a pass rate and how many judged runs passed. Narrow it down with **Filters**. **3. Open the runs** Click a cell to list its runs, filtered by **Passed**, **Failed**, **Pending** or **Not evaluated**. Open a run to read its **Conversation**, the result of each evaluation in **Evals**, and the tools it used in **Agent Tools**. **Pending** means the run hasn't ended yet, or is waiting for its evaluation results. ## Agent tool use Agent tool use answers the question behind most experiments: **did the agent actually use your MCP server, skill or command-line tool?** A low pass rate with a tool nobody called is a discovery problem, not a quality problem. For each skill, MCP server, command-line tool and browser a run was given, Tracks counts how many of the agent's tool calls touched it, and how many of those failed: | Item | Counted as used when the agent | | - | - | | **Skill** | loads it, or reads or runs one of its files | | **MCP server** | calls one of its tools | | **Browser** | calls one of its tools | | **Command-line tool** | runs a shell command with the tool's name | It is recorded for every run once it ends, however it ended, so even a run that was not evaluated has it. It is free. Where to read it: * **Execution page**: under **Results**, an **Agent tool use** section shows, for each item and each column, how many of the runs given it used it, such as "4 of 6 runs used it", and how many calls failed. A dash means that group was never given the item. * **Run page**: the **Agent Tools** tab lists each tool the run was given, with its number of calls and failed calls, or **Not used**. > [!WARNING] > > Agent tool use is shown only in the web app. The API, CLI and MCP server don't return it yet. ## With the CLI or API ```bash theme={null} tracks executions get ``` ```json theme={null} { "id": "…", "experiment_id": "…", "number": 3, "pending": false, "slots": { "total": 12, "running": 0, "judging": 0, "judged": 10, "passed": 7 }, "pass_rate": 70, "cost_usd": 4.82 } ``` `pending` is `false` once every run has ended and been judged: the pass rate is then final. `pass_rate` is absent until a run has a verdict. ```bash theme={null} tracks executions slots ``` Each slot gives its `scenario_id`, `repository_id`, `agent_tools_id`, `agent` and `repetition`, then: | Field | Meaning | | - | - | | `outcome` | `running`, `judging`, `judged` or `not_evaluated`. | | `verdict` | `passed` or `failed`, once judged. | | `score` | Out of 100, once every evaluation result is in. | | `evaluation_results` | Each evaluation judged so far, and whether the run passed it. | | `not_evaluated_reason` | Why it won't be judged. See [Not evaluated](#not-evaluated). | | `refusal` | Why its run couldn't start. | | `run_id` | The run, once started. Read what the agent did with `tracks runs events `. | This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.