> ## Documentation Index > Fetch the complete documentation index at: https://docs.tracks.alpic.ai/llms.txt > Use this file to discover all available pages before exploring further. # Create an experiment > Creates an experiment from its scenarios, agents, variants and run settings. Launch it to start its runs. ## OpenAPI ````yaml https://tracks.alpic.ai/api/openapi.json post /experiments openapi: 3.1.0 info: title: Tracks API version: 0.0.1 description: >- Run coding agents such as Claude Code in isolated sandboxes with an API key of your organization. Create a run with a prompt, then follow its output and what it costs, or launch an experiment to compare agents on your scenarios. servers: - url: https://tracks.alpic.ai/api security: [] tags: - name: Runs description: Start coding agents and follow their work. - name: Files description: Upload files to place in a run's sandbox. - name: Connections description: >- Let agents use MCP servers as one of their users, without their credentials entering the sandbox. - name: Experiments description: >- Define scenarios to give several agents in several variants, judged on what they did. - name: Executions description: Launch experiments and read their results. - name: Library description: What your experiments can give their agents. - name: System paths: /experiments: post: tags: - Experiments summary: Create an experiment description: >- Creates an experiment from its scenarios, agents, variants and run settings. Launch it to start its runs. operationId: experiments.create parameters: [] requestBody: content: application/json: schema: $ref: '#/components/schemas/ExperimentInput' required: true responses: '201': description: >- Scenarios given to several agents in several variants, each run judged on the scenario's evaluations. content: application/json: schema: $ref: '#/components/schemas/Experiment' '400': description: >- The request is invalid: a value is missing, has the wrong type or is out of range. Fix it before sending the request again. content: application/json: schema: $ref: '#/components/schemas/InvalidRequestEncoded' '401': description: >- The API key is missing, malformed or revoked, or its organization is closed. content: application/json: schema: $ref: '#/components/schemas/ApiKeyRefusedEncoded' '422': description: >- An input file is missing, expired, or inaccessible to your organization. content: application/json: schema: $ref: '#/components/schemas/FileUnavailableEncoded' security: - bearer: [] components: schemas: ExperimentInput: type: object properties: name: type: string description: Name of the experiment. scenarios: type: array items: $ref: '#/components/schemas/ScenarioInput' agents: type: array items: $ref: '#/components/schemas/ExperimentAgent' description: Agents every scenario is given to, each once. repositories: type: array items: type: string description: >- Ids of repositories from your library, as listed by `GET /library`, each once, tried in turn with every agent tool configuration. Defaults to none: runs start without a repository. uniqueItems: true agent_tools: type: array items: $ref: '#/components/schemas/AgentTools' excluded_variants: type: array items: $ref: '#/components/schemas/ExcludedVariant' description: >- Combinations of repository and agent tools not to run. Every other combination is a variant, run for each scenario and agent. Defaults to none. run_settings: $ref: '#/components/schemas/RunSettingsInput' required: - name - scenarios - agents - agent_tools - run_settings additionalProperties: false Experiment: type: object properties: id: $ref: '#/components/schemas/ExperimentId' name: type: string description: Name of the experiment. scenarios: type: array items: $ref: '#/components/schemas/Scenario' agents: type: array items: $ref: '#/components/schemas/ExperimentAgent' description: Agents every scenario is given to, each once. repositories: type: array items: type: string description: >- Ids of repositories from your library, as listed by `GET /library`, each once, tried in turn with every agent tool configuration. Defaults to none: runs start without a repository. agent_tools: type: array items: $ref: '#/components/schemas/AgentTools' excluded_variants: type: array items: $ref: '#/components/schemas/ExcludedVariant_1' description: >- Combinations of repository and agent tools not to run. Every other combination is a variant, run for each scenario and agent. Defaults to none. run_settings: $ref: '#/components/schemas/RunSettings' required: - id - name - scenarios - agents - repositories - agent_tools - excluded_variants - run_settings additionalProperties: false description: >- Scenarios given to several agents in several variants, each run judged on the scenario's evaluations. InvalidRequestEncoded: type: object properties: _tag: type: string enum: - InvalidRequest message: type: string description: Every problem found, in one sentence. location: type: string enum: - path - query - header - body description: The part of the request that is wrong. issues: type: array items: $ref: '#/components/schemas/RequestIssue' description: Each problem found, with where it is. required: - _tag - message - location - issues additionalProperties: false ApiKeyRefusedEncoded: type: object properties: _tag: type: string enum: - ApiKeyRefused message: type: string required: - _tag - message additionalProperties: false FileUnavailableEncoded: type: object properties: _tag: type: string enum: - FileUnavailable message: type: string required: - _tag - message additionalProperties: false ScenarioInput: type: object properties: id: type: string minLength: 1 maxLength: 128 description: >- Identifier you choose for the scenario, unique in the experiment, up to 128 characters, such as `checkout`. Results refer to it. name: type: string description: Short name of the scenario. prompt: type: string description: What the agent is asked to do. input_files: type: array items: type: string description: Reference returned when requesting a file upload. maxItems: 20 description: >- Up to 20 uploaded file ids, given to each of the scenario's runs. Kept for good once saved. Defaults to none. evaluations: type: array items: $ref: '#/components/schemas/EvaluationInput' passing_score: type: integer minimum: 0 maximum: 100 description: >- Score out of 100, weighted by the evaluations' weights, a run needs to pass. Omit it so a run passes only when it passes every evaluation. required: - id - name - prompt - evaluations additionalProperties: false ExperimentAgent: type: object properties: harness: $ref: '#/components/schemas/Harness' model: type: string description: Model the agent uses, such as `claude-sonnet-5-5`. required: - harness - model additionalProperties: false description: A coding agent and the model it uses. AgentTools: type: object properties: id: type: string minLength: 1 maxLength: 128 description: >- Identifier you choose for the agent tool configuration, unique in the experiment, up to 128 characters, such as `with-docs-mcp`. Results refer to it. name: type: string description: >- Short name of the agent tools, such as `No agent tools` for the baseline. skills: type: array items: type: string description: Ids of skills from your library, as listed by `GET /library`. mcp_servers: type: array items: type: string description: Ids of MCP servers from your library, as listed by `GET /library`. command_line_tools: type: array items: type: string description: >- Ids of command-line tools from your library, as listed by `GET /library`. browser: type: boolean required: - id - name - skills - mcp_servers - command_line_tools additionalProperties: false description: >- Skills, MCP servers, command-line tools and a browser given to the agent. Keep one with none as the baseline the others are compared to. ExcludedVariant: type: object properties: repository_id: type: string description: >- Id of one of the experiment's repositories. Omit it for no repository. agent_tools_id: type: string description: Id of one of the experiment's agent tool configurations. required: - agent_tools_id additionalProperties: false description: A combination of repository and agent tools not to run. RunSettingsInput: type: object properties: repetitions: type: integer minimum: 1 maximum: 9007199254740991 description: >- Identical runs started for each scenario, variant and agent, to compare results beyond chance. time_limit_seconds: type: number exclusiveMinimum: 0 maximum: 43200 description: Longest each run may last, in seconds, above 0 and at most 43200. budget_usd: type: number exclusiveMinimum: 0 maximum: 10000 description: Most each run may spend, in US dollars, above 0 and at most 10000. setup: type: string minLength: 1 maxLength: 32768 description: >- Shell script run in each run's sandbox after checkout and before the prompt. At most 32768 characters. teardown: type: string minLength: 1 maxLength: 32768 description: >- Shell script run in each run's sandbox after the run ends. At most 32768 characters. variables: type: array items: type: object properties: name: type: string value: type: string required: - name - value additionalProperties: false description: >- Environment variables set in every process of each run's agent, each name once. Defaults to none. required: - repetitions - time_limit_seconds - budget_usd additionalProperties: false ExperimentId: type: string description: Unique identifier of the experiment. Scenario: type: object properties: id: type: string minLength: 1 maxLength: 128 description: >- Identifier you choose for the scenario, unique in the experiment, up to 128 characters, such as `checkout`. Results refer to it. name: type: string description: Short name of the scenario. prompt: type: string description: What the agent is asked to do. input_files: type: array items: type: string description: Reference returned when requesting a file upload. maxItems: 20 description: >- Up to 20 uploaded file ids, given to each of the scenario's runs. Kept for good once saved. Defaults to none. evaluations: type: array items: $ref: '#/components/schemas/Evaluation' passing_score: type: integer minimum: 0 maximum: 100 description: >- Score out of 100, weighted by the evaluations' weights, a run needs to pass. Omit it so a run passes only when it passes every evaluation. required: - id - name - prompt - input_files - evaluations additionalProperties: false description: One task given to the agents. ExcludedVariant_1: type: object properties: repository_id: type: string description: >- Id of one of the experiment's repositories. Omit it for no repository. agent_tools_id: type: string description: Id of one of the experiment's agent tool configurations. required: - agent_tools_id additionalProperties: false description: A combination of repository and agent tools not to run. RunSettings: type: object properties: repetitions: type: integer minimum: 1 maximum: 9007199254740991 description: >- Identical runs started for each scenario, variant and agent, to compare results beyond chance. time_limit_seconds: type: number exclusiveMinimum: 0 maximum: 43200 description: Longest each run may last, in seconds, above 0 and at most 43200. budget_usd: type: number exclusiveMinimum: 0 maximum: 10000 description: Most each run may spend, in US dollars, above 0 and at most 10000. setup: type: string minLength: 1 maxLength: 32768 description: >- Shell script run in each run's sandbox after checkout and before the prompt. At most 32768 characters. teardown: type: string minLength: 1 maxLength: 32768 description: >- Shell script run in each run's sandbox after the run ends. At most 32768 characters. variables: type: array items: type: object properties: name: type: string value: type: string required: - name - value additionalProperties: false description: >- Environment variables set in every process of each run's agent, each name once. Defaults to none. required: - repetitions - time_limit_seconds - budget_usd - variables additionalProperties: false description: What every run of the experiment shares. RequestIssue: type: object properties: path: type: array items: anyOf: - type: string - type: number - type: string enum: - Infinity - '-Infinity' - NaN description: >- Keys leading to the wrong value inside that part of the request, such as `["agent", "model"]`. Empty when the problem concerns that part as a whole. message: type: string description: What is wrong with the value. required: - path - message additionalProperties: false description: One problem found in the request. EvaluationInput: type: object properties: id: type: string minLength: 1 maxLength: 128 description: >- Identifier you choose for the evaluation, unique in its scenario, up to 128 characters, such as `tests-pass`. Results refer to it. name: type: string description: Short name of what is checked. judging_instructions: type: string description: >- The yes-or-no question a judge answers from what the agent did, such as `Do the tests pass?`. weight: type: integer minimum: 1 maximum: 9007199254740991 description: How much it counts in the run's score. Defaults to 1. required: - id - name - judging_instructions additionalProperties: false Harness: type: string enum: - claude-code - claude-desktop - codex - codex-app - cursor description: >- The coding agent to run: `claude-code` (Claude Code), `claude-desktop` (Cowork, the agent workspace of Anthropic's Claude desktop app, for documents, research and longer tasks), `codex` (OpenAI's Codex CLI), `codex-app` (the Codex mode of OpenAI's ChatGPT desktop app, with its browser, computer-use and document tools) or `cursor` (Cursor's CLI). Evaluation: type: object properties: id: type: string minLength: 1 maxLength: 128 description: >- Identifier you choose for the evaluation, unique in its scenario, up to 128 characters, such as `tests-pass`. Results refer to it. name: type: string description: Short name of what is checked. judging_instructions: type: string description: >- The yes-or-no question a judge answers from what the agent did, such as `Do the tests pass?`. weight: type: integer minimum: 1 maximum: 9007199254740991 description: How much it counts in the run's score. Defaults to 1. required: - id - name - judging_instructions - weight additionalProperties: false description: One criterion every run of the scenario is judged on. securitySchemes: bearer: description: >- An API key of your organization, starting with `trk_`, sent as `Authorization: Bearer `. type: http scheme: bearer ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.