<!-- Source: https://docs.biosimulant.com/how-to/run-an-experiment -->

# Run an experiment

An experiment in Biosimulant is a group of runs that answer one question. You name the question, describe each setup you want compared, approve the whole thing once, and read the arms against each other when they finish.

There is no separate experiment engine. Every arm is an ordinary run pinned to an exact revision, so anything you can do with a run — open it, fetch its artifacts, get its Passport — you can still do with an arm.

## What an arm is

An **arm** is one setup in the study. Two runs belong to the same arm when the three things that decide what actually executes are identical: the workspace revision, the parameter overrides, and the simulation config.

That has a useful consequence. Running the same arm twice gives you a **replicate**, not a second arm, and the study counts it as such without you declaring anything.

Arms are free to differ in any of those three dimensions, or all of them:

| What you change between arms | What you learn |
| --- | --- |
| A parameter | How sensitive the result is to that parameter |
| The input data | Whether the model holds across datasets |
| The revision | Whether your change improved anything |
| The lab or model entirely | Which approach answers the question better |

Nothing requires the arms of a study to share a lab.

## Plan one with an agent

This is the path the system is built around. From Codex or Claude connected over the [Agent Gateway](/how-to/agent-gateway):

> Compare the glycolysis lab and the central-metabolism lab on the same glucose input, and tell me which one reproduces the measured ATP yield.

Your assistant calls `experiment_prepare` with one arm per setup. That preflights **every** arm exactly as a lone run is preflighted — contract checks, graph resolution, compute limits — and returns a single plan digest plus a per-arm list of findings and blockers.

You then see the whole study before anything runs: each arm, what it changes, and how much compute the approval covers in total. Approving that one digest calls `experiment_create`, which opens the study and queues exactly those arms.

**Info:**

  The approval is the boundary. One digest authorizes precisely the arms you
  were shown — not "some runs in this workspace". If a revision or the compute
  configuration moves between preparing and approving, the whole study is
  refused rather than run in part.

Arms of one study run in parallel. A four-arm sweep is four runs going at once, not a queue.

### Reading it back

> How did that experiment turn out?

`experiment_get` returns the arms, the runs behind each one, and a comparison table across them. While runs are still going it says so instead of implying a result.

Only completed runs contribute values. An arm whose runs all failed reports no metrics rather than a zero.

### Tool reference

| Tool | What it does |
| --- | --- |
| `experiment_prepare` | Preflights every arm and returns one digest. Queues nothing. |
| `experiment_create` | Opens the study and queues its arms, under that one approval. |
| `experiment_get` | Arms, their runs, and the comparison across them. |

A study may propose at most **12 arms**. A wider study is really several studies, and splitting it keeps each approval something a person can actually read.

## Group runs you already have

If the runs exist, you don't need to re-run anything. In Studio, open **Experiments** and group them; each run can carry an arm label so results can be told apart.

Over the API:

```bash
curl -X POST https://api.biosimulant.com/api/run-groups \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "ATP yield across labs",
    "goal": "Which lab reproduces the measured yield?",
    "run_ids": ["<run-a>", "<run-b>"],
    "arm_labels": {"<run-a>": "glycolysis", "<run-b>": "central-metabolism"}
  }'
```

Adding a run to a study changes nothing about the run itself. Removing one, or closing the study, leaves every run in place — they stay listed under **Runs** as they always were.

## Read the comparison honestly

The comparison table is built from what the runs actually produced, and it is deliberately conservative about what it claims.

- **Scalars** compare directly.
- **A series** has no single value, so the table reports where it ended and how many points it had. It does not compute a mean or an area under the curve, because that choice is yours to make, not ours to guess.
- **Outputs with nothing comparable in them** are left out rather than rendered as `—` in every row.
- **The latest completed run** speaks for its arm. Earlier runs of the same arm are replicates and stay visible underneath.

A parameter that is identical in every arm is part of the setup, not part of the experiment, so the table leaves it out. What remains in the columns is what actually varied.

**Warning:**

  A finished experiment tells you what these models did under these inputs. It
  is not experimental evidence, and a difference between two arms is not a
  finding until you have checked that the arms differ only in the way you think
  they do.

For a deeper side-by-side, open **Compare runs** from the study — it carries each arm's representative run into Studio's run comparison view.

## Caveats

- Every arm is a managed run and is subject to the same rate and runtime limits.
- Runs under free beta come back with `user_charge: 0`; the study reports the total compute its approval covers.
- A study is scoped to the account that opened it. Nothing about grouping makes a run visible to anyone else.

## Next steps

- [Use Biosimulant from Codex or Claude](/how-to/agent-gateway): connecting and approving work
- [Compose Hub labs locally](/how-to/compose-hub-labs-locally): building the arms you want to compare
- [How-to guides](/how-to): other task guides