Run an experiment
An experiment in Biosimulant is a group of runs that answer one question. You name the question, describe each setup you want compared, approve the whole thing once, and read the arms against each other when they finish.
There is no separate experiment engine. Every arm is an ordinary run pinned to an exact revision, so anything you can do with a run — open it, fetch its artifacts, get its Passport — you can still do with an arm.
What an arm is
An arm is one setup in the study. Two runs belong to the same arm when the three things that decide what actually executes are identical: the workspace revision, the parameter overrides, and the simulation config.
That has a useful consequence. Running the same arm twice gives you a replicate, not a second arm, and the study counts it as such without you declaring anything.
Arms are free to differ in any of those three dimensions, or all of them:
| What you change between arms | What you learn |
|---|---|
| A parameter | How sensitive the result is to that parameter |
| The input data | Whether the model holds across datasets |
| The revision | Whether your change improved anything |
| The lab or model entirely | Which approach answers the question better |
Nothing requires the arms of a study to share a lab.
Plan one with an agent
This is the path the system is built around. From Codex or Claude connected over the Agent Gateway:
Compare the glycolysis lab and the central-metabolism lab on the same glucose input, and tell me which one reproduces the measured ATP yield.
Your assistant calls experiment_prepare with one arm per setup. That preflights every arm exactly as a lone run is preflighted — contract checks, graph resolution, compute limits — and returns a single plan digest plus a per-arm list of findings and blockers.
You then see the whole study before anything runs: each arm, what it changes, and how much compute the approval covers in total. Approving that one digest calls experiment_create, which opens the study and queues exactly those arms.
The approval is the boundary. One digest authorizes precisely the arms you were shown — not “some runs in this workspace”. If a revision or the compute configuration moves between preparing and approving, the whole study is refused rather than run in part.
Arms of one study run in parallel. A four-arm sweep is four runs going at once, not a queue.
Reading it back
How did that experiment turn out?
experiment_get returns the arms, the runs behind each one, and a comparison table across them. While runs are still going it says so instead of implying a result.
Only completed runs contribute values. An arm whose runs all failed reports no metrics rather than a zero.
Tool reference
| Tool | What it does |
|---|---|
experiment_prepare | Preflights every arm and returns one digest. Queues nothing. |
experiment_create | Opens the study and queues its arms, under that one approval. |
experiment_get | Arms, their runs, and the comparison across them. |
A study may propose at most 12 arms. A wider study is really several studies, and splitting it keeps each approval something a person can actually read.
Group runs you already have
If the runs exist, you don’t need to re-run anything. In Studio, open Experiments and group them; each run can carry an arm label so results can be told apart.
Over the API:
curl -X POST https://api.biosimulant.com/api/run-groups \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{
"name": "ATP yield across labs",
"goal": "Which lab reproduces the measured yield?",
"run_ids": ["<run-a>", "<run-b>"],
"arm_labels": {"<run-a>": "glycolysis", "<run-b>": "central-metabolism"}
}'Adding a run to a study changes nothing about the run itself. Removing one, or closing the study, leaves every run in place — they stay listed under Runs as they always were.
Read the comparison honestly
The comparison table is built from what the runs actually produced, and it is deliberately conservative about what it claims.
- Scalars compare directly.
- A series has no single value, so the table reports where it ended and how many points it had. It does not compute a mean or an area under the curve, because that choice is yours to make, not ours to guess.
- Outputs with nothing comparable in them are left out rather than rendered as
—in every row. - The latest completed run speaks for its arm. Earlier runs of the same arm are replicates and stay visible underneath.
A parameter that is identical in every arm is part of the setup, not part of the experiment, so the table leaves it out. What remains in the columns is what actually varied.
A finished experiment tells you what these models did under these inputs. It is not experimental evidence, and a difference between two arms is not a finding until you have checked that the arms differ only in the way you think they do.
For a deeper side-by-side, open Compare runs from the study — it carries each arm’s representative run into Studio’s run comparison view.
Caveats
- Every arm is a managed run and is subject to the same rate and runtime limits.
- Runs under free beta come back with
user_charge: 0; the study reports the total compute its approval covers. - A study is scoped to the account that opened it. Nothing about grouping makes a run visible to anyone else.
Next steps
- Use Biosimulant from Codex or Claude: connecting and approving work
- Compose Hub labs locally: building the arms you want to compare
- How-to guides: other task guides