Scientific interfaces and output acceptance

Implemented for the unreleased Biosimulant 0.0.36 runtime. This is an opt-in output contract. It does not establish biological accuracy, calibration, experimental binding, or fitness for an intended scientific use.

Declare checks before execution

Add scientific_validation to the root lab.yaml:

scientific_validation:
  schema_version: '1'
  required_profiles:
    - module: prediction
      direction: output
      port: binding_probability
      profile: boltz.binding-probability/v1
    - module: analysis
      direction: input
      port: binding_probability
      profile: boltz.binding-probability/v1
  checks:
    - id: binding_probability
      kind: profile
      source:
        module: prediction
        port: binding_probability
      profile: boltz.binding-probability/v1
    - id: analysis_finished
      kind: equals
      source:
        module: analysis
        port: completed
      expected: true

Required ports must declare the exact profile. Every connection touching those ports must have matching profiles on both ends. Other ports retain the ordinary compatibility policy. Static source validation checks these declarations without executing Python. Runtime preparation repeats the check before model setup. Package-backed child declarations are checked after dependency resolution.

By default a required port needs at least one successful live value check. Set coverage: {min_count: 10} or coverage: {exact_count: 10} on a required port when a specific count is necessary. Every observed check must succeed. These are counts of profiled value validations, not a guarantee of every time point or biological replicate; declare series_coverage/v1 or identity checks for those. Zero coverage must be explicitly requested with min_count: 0.

Embedded and version-pinned package child Labs contribute their acceptance requirements automatically. Child check IDs are namespaced as child/check_id and selectors use the actual flattened module names, including exposed IO mappings. A parent cannot remove child checks or weaken their required coverage. Conflicting profiles, conflicting exact counts, unresolved children, cycles and aggregate limit violations fail closed. Parent and child requirements together are the effective specification, saved as acceptance_spec in results. The backend independently reconstructs this specification from immutable workspace files and pinned child packages before issuing a Passport.

Built-in checks

Numerical closeness means abs(actual - expected) <= atol + rtol * abs(expected). Both tolerances default to zero; no scientific threshold is hidden in the runtime. NaN, infinity and booleans are rejected as numeric observations. Arrays must be nonempty. Series coverage checks sampling coverage, not the correctness of values.

same_ids/v2 defaults to unique composite IDs and nonempty rows. Use multiplicity: exact to compare repeated IDs with their counts, or include a replicate/time field in id_fields. allow_empty: true explicitly permits two empty tables. Numeric and string identities remain distinct, including their JSON numeric types; no silent coercion is performed.

Each source is {module, port, direction?, path?}. Direction defaults to output; input selects the latest supplied/consumed value of that input. path contains record field names or nonnegative array indices, not executable code. A serialized BioSignal is decoded once; nested records retain their fields. A schema: {payload: json} record uses path: [payload, status].

Only input ports selected by acceptance checks are retained; ordinary runs do not copy their input data for acceptance. Results save these selected snapshots as acceptance_inputs. This is not complete input history: publish explicit observation tables when requirements concern a trajectory or multiple batches. ID equality alone cannot detect deliberately mislabelled molecular content.

Unknown schema versions, rules, fields and profiles are rejected. A specification requires 1–100 checks and permits up to 100 required ports. Check IDs must be unique and cannot start with profile:. There is no manifest-supplied Python execution. Missing/invalid values fail; evaluator exceptions produce error, never a pass. No result is accepted just because a test file exists.

Run and inspect

biosimulant labs validate ./lab --json
biosimulant labs run ./lab --results-file result.json --require-acceptance --json

The optional --require-acceptance flag exits 3 if acceptance is missing or does not pass, after saving results. Without the flag, execution can finish while acceptance fails. Existing execution and compatibility exit codes remain intact. Execution exceptions do not create a successful acceptance record.

CLI and local Lab serving use the same evaluator. Results contain acceptance with status (not_declared, passed, failed), per-check evidence, counts, the canonical effective-specification SHA-256, evaluator implementation SHA-256 and runtime version. File-backed profiles are identified through catalogue representation metadata, not biological profile names. Successful file checks record the artifact SHA-256. Other checked values are not copied into the acceptance record. These hashes identify what was checked; they are not independent proof that model authors made correct claims.

Results are scoped to their exact packaged Lab revision and output artifact. Preserve both alongside the acceptance record. Re-evaluation can use biosimulant.acceptance.evaluate(result["acceptance_spec"], result["outputs"], result["compatibility"], inputs=result["acceptance_inputs"]); file-backed checks also require the original referenced artifact bytes/paths. This does not repeat inference. It is not yet a portable artifact-path resolver.

Hosted execution and Passports

The backend preserves bounded acceptance evidence in workspace and MCP summaries. Runs with failed or invalid acceptance cannot contribute comparison metrics; their run records and exclusion counts remain visible. Older unprofiled runs keep their legacy comparison behavior and do not acquire an acceptance pass.

Select research-ready@3 for the new Passport check. It binds the recorded spec hash and exact expected check IDs to the immutable Lab manifest. Older profile versions retain their meanings. Version 3 uses Passport schema 1.2. A successful check means declared output requirements passed, not that the science is valid.

Runtime support is negotiated from explicit capabilities, not inferred from a minimum version number. On the exact worker installation run:

biosimulant labs acceptance-capabilities --json

Register that descriptor in the backend’s BIOSIMULANT_RUNTIME_CAPABILITIES configuration, keyed by the exact BIOSIMULANT_CLI_VERSION; the descriptor’s runtime_version must match. This allows development builds while refusing unadvertised future releases. The descriptor lists schemas, versioned validators, features and resource limits. Regenerate it when changing installed validator plugins. An empty or insufficient descriptor blocks acceptance-dependent runs before compute submission. Deployment configuration is operator-owned; the runtime still validates the actual specification before execution.

Extending validation without editing the runtime

Trusted application startup can call register_validator("organisation.rule/v1", Validator(...)). A validator declares required/optional fields, a specification validator, an evaluation function and an implementation SHA-256 covering its installed implementation/dependencies. Duplicate references cannot replace existing semantics. Errors produce explicit failed/error evidence, never passes.

For packaged CLI workers, an installed Python distribution can expose a biosimulant.acceptance entry point whose callable accepts (register_validator, Validator). Enable exact entry-point names in BIOSIMULANT_ACCEPTANCE_PLUGINS=name1,name2. Only explicitly enabled installed plugins load, including during capability discovery. Labs cannot provide import paths or install/enable validators through their manifests. Plugins are trusted host code; they are not a sandbox for untrusted Python. Pin and review their packages as part of the worker image. Their digests appear in capabilities and per-check evidence and contribute to the evaluator digest.

The five original unversioned check names remain supported with their original semantics. New check implementations use explicit /vN references. Resource bounds and supported report-schema/evaluator versions remain explicit protocol constraints, rather than assumptions about a particular model domain.

Verification scope

CPU fixtures cover missing/duplicate/swapped IDs, partial results, invalid values, unused/partial interfaces, validator errors and serialization through actual Lab packaging. The Boltz example’s synthetic-executable tests are software evidence, not Boltz inference or biological evidence. Real GPU qualification, coordinate identity/stereochemistry validation and independent-user reuse studies remain separate release/evaluation tasks.

The accompanying CPU microbial-growth example declares an analytical starvation solution before execution, checks the expected integration time grid, and rejects a changed death rate. It establishes reuse across a temporal model and the Boltz workflow; it does not establish universal model coverage or experimental validity.