StandardsModel CompatibilityConformance

Conformance

Conformance tests check that a tool implements the standard correctly, meaning it gives the same results as the reference implementations for the same input. Run them if you build a validator, a comparison tool, or anything else that reads port contracts.

To conform, an implementation must:

  1. Use the exact released bundle of schemas and profiles.
  2. Load references only from that bundle, and check the sha256 of each one.
  3. Normalize contracts using the documented rules.
  4. Serialize JSON with RFC 8785 before hashing.
  5. Return the same statuses and reason codes as the reference implementations.

Run the tests

pip install 'biosimulant[compatibility]'
biosimulant compatibility conformance

The command runs three test cases for every profile in the release:

  • a valid contract, which must pass validation
  • an invalid contract, which must be rejected with a specific reason code
  • a comparison with a missing field, which must return UNKNOWN

On success it prints a summary and exits with status 0:

{
  "bundle_sha256": "sha256:…",
  "profile_fixtures_passed": 1950,
  "profiles": 650,
  "valid": true
}

bundle_sha256 matches the bundle sha256 shown on the overview. If a test fails, the command stops at the first failure, prints an error that names the profile and the case that failed, and exits with a non-zero status.

To run the full test suites for the Python and TypeScript reference implementations, clone the standard repository and run:

python3 scripts/build_standard.py --check
python3 -m pip install -e '.[test]'
python3 -m pytest python/tests
 
npm install
npm test

What the standard tests

LevelWhat it checks
DocumentEach schema accepts valid examples and rejects invalid ones
ProfileEach profile enforces its required fields and its rules for missing values
ComparisonImplementations return the same status and findings for the shared test cases
HashingNormalized JSON and sha256 hashes match byte for byte
PackageA lock is the same every time you build, and building doesn’t change your source files
RuntimeIf a value sent during a run doesn’t match its port’s contract, the run records it

biosimulant compatibility conformance covers the Profile level: three cases per profile, 1,950 in total. Shared cases for comparison statuses and hashing are in comparison-statuses.json and canonicalization.json.

Passing these tests shows that a tool behaves correctly. It says nothing about whether the profiles themselves are scientifically right. See review status.

Before a stable release

Version 0.1.0-alpha.5 is an early release. Before a stable release, every profile needs:

  • requirements written for its specific kind of data
  • published scientific sources that support those requirements
  • a named scientific reviewer who didn’t write the profile
  • a different named reviewer for its schema and rules
  • a recorded domain owner, intended use, limitations and sourced decisions
  • test cases for conversions, where the profile allows them

Each completed review is stored as a checked JSON record in the standard repository. The bundle changes ga_ready to true only after all 650 profiles pass that gate. Until then the standard stays in alpha, even when every automated test passes.

Adapters & InferenceAPI & CLI