<!-- Source: https://docs.biosimulant.com/standards/model-compatibility/conformance -->

# Conformance

Conformance tests check that a tool implements the standard correctly, meaning it gives the same results as the reference implementations for the same input. Run them if you build a validator, a comparison tool, or anything else that reads port contracts.

To conform, an implementation must:

1. Use the exact released bundle of schemas and profiles.
2. Load references only from that bundle, and check the sha256 of each one.
3. Normalize contracts using the documented rules.
4. Serialize JSON with [RFC 8785](https://www.rfc-editor.org/rfc/rfc8785) before hashing.
5. Return the same statuses and reason codes as the reference implementations.

## Run the tests

```bash
pip install 'biosimulant[compatibility]'
biosimulant compatibility conformance
```

The command runs three test cases for every profile in the release:

- a valid contract, which must pass validation
- an invalid contract, which must be rejected with a specific reason code
- a comparison with a missing field, which must return `UNKNOWN`

On success it prints a summary and exits with status 0:

```json
{
  "bundle_sha256": "sha256:…",
  "profile_fixtures_passed": 1950,
  "profiles": 650,
  "valid": true
}
```

`bundle_sha256` matches the bundle sha256 shown on the [overview](/standards/model-compatibility). If a test fails, the command stops at the first failure, prints an error that names the profile and the case that failed, and exits with a non-zero status.

To run the full test suites for the Python and TypeScript reference implementations, clone the [standard repository](https://github.com/Biosimulant/model-compatibility-standard) and run:

```bash
python3 scripts/build_standard.py --check
python3 -m pip install -e '.[test]'
python3 -m pytest python/tests

npm install
npm test
```

## What the standard tests

| Level | What it checks |
|---|---|
| Document | Each schema accepts valid examples and rejects invalid ones |
| Profile | Each profile enforces its required fields and its rules for missing values |
| Comparison | Implementations return the same status and findings for the shared test cases |
| Hashing | Normalized JSON and sha256 hashes match byte for byte |
| Package | A lock is the same every time you build, and building doesn't change your source files |
| Runtime | If a value sent during a run doesn't match its port's contract, the run records it |

`biosimulant compatibility conformance` covers the Profile level: three cases per profile, 1,950 in total. Shared cases for comparison statuses and hashing are in [comparison-statuses.json](/standards/model-compatibility/v0.1/fixtures/golden/comparison-statuses.json) and [canonicalization.json](/standards/model-compatibility/v0.1/fixtures/golden/canonicalization.json).

**Info:**

  Passing these tests shows that a tool behaves correctly. It says nothing about whether the profiles themselves are scientifically right. See [review status](/standards/model-compatibility/catalogue#review).

## Before a stable release

Version 0.1.0-alpha.5 is an early release. Before a stable release, every profile needs:

- requirements written for its specific kind of data
- published scientific sources that support those requirements
- a named scientific reviewer who didn't write the profile
- a different named reviewer for its schema and rules
- a recorded domain owner, intended use, limitations and sourced decisions
- test cases for conversions, where the profile allows them

Each completed review is stored as a checked JSON record in the standard repository. The bundle changes `ga_ready` to `true` only after all 650 profiles pass that gate. Until then the standard stays in alpha, even when every automated test passes.