Introduction

Veredictum is a conformance instrument for openEHR clinical data repositories. You point it at a running CDR and it tells you, with citations, which parts of the openEHR specification that server implements.

It executes a machine-readable catalogue of 1103 spec-cited test cases against the server’s own wire, records every exchange, and computes verdicts as pure functions over what it recorded. Functional conformance, measured performance and step-load stress come from the one tool.

The released openEHR specifications are the only authority the instrument accepts. Every expectation in the catalogue names the section it comes from, so it can be refuted by a better reading of the specification and by nothing else.

Who this is for

• Evaluators and procurement teams. You need a claim about a CDR that someone other than its vendor can check. The instrument produces a conformance report, a statement of claims and a certificate, all three derived from a recorded run rather than asserted.
• CDR vendors and maintainers. You want to know where your server diverges from the released text before a customer finds out. A failing row arrives with the exchange that produced it and the specification section it violates.
• The openEHR community. The catalogue is an openEHR conformance test suite in machine-readable form. Where the specification is silent or contradicts itself, the divergence is recorded in an ambiguity register and reported upstream, never resolved privately.

What is in the box

Three commands, three stages

The pipeline splits into stages on purpose. Nothing is computed at a stage that could hide what an earlier stage recorded.

1. veredictum validate checks the catalogue itself. Zero findings is the only passing result, and it runs before any server is involved.
2. veredictum run drives the catalogue against your endpoints and writes results.json, a record of exchanges and not yet a judgement.
3. veredictum verdicts reads that record together with your statement of claims and writes the report, the statement and the certificate.

Installation covers getting the command. Running the instrument walks the three stages against a live server.

Coverage is machine-checked

A green run over a thin catalogue proves nothing. A coverage gate enumerates the wire surface from the released sources alone, the Service Model’s platform interfaces crossed with their ITS-REST branches, and fails on any operation, status-code branch, header rule, negotiation variant or error family that has neither a covering case nor a cited exception. A behaviour the specification defines and the catalogue misses is a gap to close or an honest boundary in the register.

Cases are added. They are never removed to make a run go green.

Note

openEHR® is the registered trademark of the openEHR Foundation. Veredictum is an independent, community-driven conformance instrument: it names openEHR descriptively, to say what is being tested, and it is not an official openEHR Foundation product, not the Foundation’s CNF program, and not endorsed by or affiliated with the Foundation.

Installation

There are four ways to get the command, and all four end at the same place: a veredictum you can point at a catalogue. Pick by what you already have installed. Whichever you pick, you also want a clone of the repository, and the first section says why.

• What you need, and why a clone is part of it
• With cargo
• With Docker: the web console
• From a release binary
• From source
• Checking that the install works

What you need, and why a clone is part of it

The instrument reads three things at run time: the catalogue, the vendored specification oracle, and your own declaration of the deployment under test. It reads all three as paths you pass on the command line, so the code and the data travel separately.

That split is deliberate. The catalogue and the specification oracle together are over 300 MB of data, which no package registry accepts, and a party may legitimately want to point the instrument at a catalogue of their own. So the published crate and the published image carry the code, and a clone of the repository is where the data lives:

git clone https://github.com/rubentalstra/Veredictum
cd Veredictum

Pick one of the three ways of getting the command below. All of them then run the same subcommands against that clone.

With cargo

The binary is on crates.io. Take this path if you want the command on your PATH.

cargo install veredictum --locked --version 0.1.0-alpha.6
veredictum validate --root artifacts --specs specs/openehr

Two flags are worth understanding rather than copying:

• --version is required while the crate is on pre-release versions. cargo install ignores a pre-release unless you name it, so without the flag it finds no version to install.
• --locked builds against the Cargo.lock the release was tested with. Leave it off and cargo resolves fresh versions of every dependency, which is a different build from the one the project’s gates ran.

The library target is published with the binary, so you can consume the typed artifact model and the published JSON Schemas directly rather than reimplementing the format.

With Docker: the web console

The container image is the web console: a browser frontend over the same instrument, served by its own binary. The CLI is deliberately not distributed as an image — a static binary needs no container, and the release binaries below are its no-toolchain path. Start the console against a clone and it serves on port 3000:

docker run --rm -p 127.0.0.1:3000:3000 -v "$PWD:/work" \
    ghcr.io/rubentalstra/veredictum:<tag>

Substitute a published tag from the package page for <tag>. The image is multi-architecture and is pushed by digest, with its tags applied only after a smoke run and a vulnerability scan of that digest have passed.

The catalogue and the specification oracle are not baked into the image. That is the same over 300 MB reason as above, and it means the data you grade against is the data you can see in your own checkout. The console has no login, so the publish flag above binds it to loopback; exposing it further is the operator’s decision, behind their own gate.

Note

Every image tag published so far predates the console’s first release and still carries the CLI as the payload, invoked as docker run --rm -v "$PWD:/work" ghcr.io/rubentalstra/veredictum:<tag> validate --root /work/artifacts --specs /work/specs/openehr. The console serves from its first release tag onward; the console chapter shows what it does today.

From a release binary

Prebuilt binaries for x86_64 and aarch64 Linux are attached to every release. Each tarball ships with a sha256sum, a CycloneDX dependency SBOM and a Sigstore bundle.

Verify the bundle before you run the binary. The check that matters is not just “this file is signed” but “this file was built by the workflow in this repository”, which is what --signer-workflow asserts:

gh attestation verify veredictum-<tag>-<target>.tar.gz \
    -R rubentalstra/Veredictum \
    --signer-workflow rubentalstra/Veredictum/.github/workflows/release-build.yml

A release is created as a draft and published only once every expected asset is attached, so you never meet a release whose binaries are still uploading. Its tag is signed, and a repository rule refuses to delete one, so a tag is never re-pointed at different code. The recovery path for a bad cut is the next version.

From source

The toolchain pins itself from rust-toolchain.toml, so you do not choose a Rust version:

cargo run -- validate --root artifacts --specs specs/openehr

The declared minimum supported Rust version is 1.96, verified in CI against that floor rather than assumed. The only extra tool is cargo-nextest, and only if you intend to run the project’s own test suite.

Checking that the install works

validate is the check to run first, because it needs no server:

veredictum validate --root artifacts --specs specs/openehr

A working install over an intact clone prints one line and exits zero:

1103 case(s), 247 binding(s), 2 party statement(s), 0 finding(s)

Any finding count above zero is a failure of the catalogue, not of your setup, and the findings printed above that line say which artifact is at fault. A missing --specs tree is the common first-run mistake: the citation and Service-Model gates are skipped without it, so the case count is reported while the checks that make the count meaningful never run.

Running the instrument

• 1. Check the catalogue
• 2. Declare your deployment
• 3. Drive the catalogue
• 4. Compute the verdicts
• Measured performance and stress
• Where to look when something goes wrong

A conformance campaign is four steps. Each writes a file the next one reads, so you can stop after any of them, inspect what was produced, and resume.

1. Check the catalogue

Before a server is involved, check the catalogue you are about to grade with:

veredictum validate --root artifacts --specs specs/openehr

This is every machine check over the artifact tree: identifier uniqueness, citation resolution against the vendored specification text, binding completeness, coverage of the enumerated wire surface, and claim completeness against the committed party statements. Zero findings is the only passing result, and the command exits 1 when there is even one.

Pass --specs every time. Without it the citation and Service-Model resolution gates do not run, and the case count still prints, which looks like a pass over a catalogue that was never fully checked.

2. Declare your deployment

The instrument needs to know where your server is and how to authenticate to it. That declaration is the IXIT file, and copying an example is the fastest way to a correct one:

cp -r party/ehrbase party/mine

The directory holds two files, and the split between them matters:

• ixit.json describes the deployment. Endpoints per instance, the authentication mode, and for a measured run an environment block naming the hardware and topology the numbers were produced on. Credentials are named, never carried: the file holds the names of the environment variables the instrument reads the user and password from, so no secret ever enters an artifact you might publish.
• statement.json is your declaration of claims. It names the product and version, the specification versions it targets, and the capabilities it claims to implement. The verdict machinery reads it as the thing being tested against the record.

A typical ixit.json declares three instances, because a full run needs to speak to the server as three different callers: an ordinary clinical user, an administrator, and no one at all. The unauthenticated instance is what lets the security cases check that a route refuses an anonymous request.

3. Drive the catalogue

veredictum run --root artifacts --ixit party/mine/ixit.json --out out/ \
    --sut-name my-cdr --sut-version 1.2.3 --statement party/mine/statement.json

The command drives every applicable case against your endpoints and writes out/results.json, a record of what was sent, what came back, and how it was classified. It also writes out/run-exceptions.json, which lists the cases the interpreter could not drive at all.

results.json is a record, not a judgement. That separation is the point: the recorded exchange stays available for anyone to re-read, and the judgement is computed from it in a separate step that touches no network.

Passing --statement here changes what runs. A case gated on an option your statement does not declare is recorded as not-applicable at drive time rather than driven, which is the test-selection discipline ISO/IEC 9646 describes. Omit the flag and everything is driven, which is what you want when you are exploring an unfamiliar server rather than grading a declared one.

--filter takes a substring matched against case identifiers, which is how you re-drive one chapter while working on a fix. The resulting results.json holds only the cases that ran, so a filtered run is a working tool and never the record you submit.

The command exits 1 if any case failed or errored, so a shell script can gate on it.

Note

A functional run never re-measures. If a results.json already exists at the --out path, its measurement records are carried forward, so running the functional catalogue again after a measured run does not discard the performance evidence. A file that is present but unreadable stops the run instead, because carrying zero measurements past it would drop that evidence silently.

4. Compute the verdicts

veredictum verdicts --root artifacts --statement party/mine/statement.json \
    --results out/results.json --out out/

This step is a pure function of the statement, the recorded results, the catalogue and the catalogue’s capability matrix. Run it twice on the same files, on any machine, and you get the same bytes out. It writes:

Nothing in these documents is asserted by hand. A number that appears in them was computed from the record in the same run that printed it.

Measured performance and stress

Those four steps cover functional conformance. Two more instruments produce the other kinds of evidence, and both need the environment block in your IXIT file filled in, because a throughput number without the deployment described says nothing.

• veredictum perf earns a volumetric class (POC, S, L or R) with an open-loop measured run and merges the measurement record into an existing results.json. The normative window is one hour; --hours extends it to 2, 4, 6, 8 or 12, which is a stricter demonstration. Nothing shorter than the case exists.
• veredictum stress climbs geometric load steps to find where the deployment breaks and writes stress.json. It is exploration only and is never a conformance record, which is why it is class-free by design.

Both want an idle machine and a deployment whose resource limits match the envelope you are claiming. A measured run on a laptop that is also running a browser measures the browser.

Where to look when something goes wrong

A red row is not presumptive evidence of a bug in your server. It is evidence that the specification, the catalogue and the server do not all three agree, and which one is wrong is a question that gets answered before anything is changed. The conformance method sets out how, and what evidence an attribution has to carry.

The command reference lists every subcommand with its real flags.

Command reference

• validate
• run
• verdicts
• verify-record
• perf
• stress
• aql-probe
• stress-compare
• perf-assets
• conformance-assets
• emit-schemas

Every flag below is the one the binary declares. veredictum <command> --help prints the same list from the build you have installed, and that output is the authority if the two ever disagree.

Three commands make the conformance record (validate, run, verdicts), two measure (perf, stress), verify-record checks a sealed bundle, and the rest render or explore.

validate

Validate one artifact tree through every machine gate.

veredictum validate --root <ROOT> [--specs <SPECS>] [--write-report]

Every machine check over the catalogue: identifier uniqueness, citation resolution, binding completeness, coverage of the enumerated wire surface, and claim completeness against the committed party statements. It prints one line per finding and a summary line, and exits 1 if the finding count is not zero.

--write-report is off by default on purpose. A check verb that rewrites a file on every run is a trap for read-only invocations, so the pipelines that publish the coverage report ask for it explicitly.

run

Execute the catalogue against a live SUT and emit results.json plus the run report.

veredictum run --root <ROOT> --ixit <IXIT> --out <OUT> \
    [--sut-name <NAME>] [--sut-version <VERSION>] \
    [--filter <SUBSTRING>] [--statement <STATEMENT>] \
    [--sign-key <KEY>]

Drives every applicable case and records the exchange. Exits 1 if any case failed or errored.

verdicts

Compute the verdicts from a statement and a results record against an artifact tree, and write the rendered submission documents.

veredictum verdicts --statement <STATEMENT> --results <RESULTS> \
    --root <ROOT> --out <OUT> [--sign-key <KEY>]

The pure step. It reaches no network and reads nothing but its inputs, which is what makes a published verdict re-derivable by anyone who has the same four files.

verify-record

Verify a sealed bundle: recompute every digest its record manifest names, and check the detached signature over that manifest.

veredictum verify-record --record <DIR> --key <KEY>

Prints the signer fingerprint, the signing time, and one line per file with its digest verdict. Zero findings is the only passing result: a digest mismatch, a file the manifest names but the bundle does not carry, or a signature no component of the supplied key verifies, each exits 1 naming what failed.

The bundle is ordinary files, so the check does not depend on this tool. gpg --verify record-manifest.json.asc record-manifest.json establishes the same signature, and sha256sum re-derives the same digests.

A verified bundle is one link in the chain and not the whole of it. A valid signature proves integrity and origin since signing, and says nothing about the conditions the run executed under. The published instrument, the verification pack and the citation-carrying record are the rest, which is why that sentence prints on every verification.

perf

Execute the performance schedule’s open-loop measured run against a live SUT and merge the measurement records into an existing results.json.

veredictum perf --root <ROOT> --ixit <IXIT> --results <RESULTS> --class <CLASS> \
    [--seed-workers <N>] [--hours <H>]

Conformance by measurement. Latency is measured from the planned arrival instant under open-loop offered load, so a stall shows up as latency instead of disappearing into a slowed-down request rate. A longer --hours window is a stricter demonstration and persists like any measured run; nothing shorter than the case exists.

The environment block is mandatory rather than optional because a latency number is a claim about a deployment, and a claim with no deployment described cannot be checked or reproduced.

stress

Run the step-load stress instrument: geometric load steps up to the maximum sustainable throughput.

veredictum stress --root <ROOT> --ixit <IXIT> --out <OUT> \
    [--corpus-class <CLASS>] [--seed-workers <N>] \
    [--step-secs <S>] [--bisections <N>] [--max-rate <R>]

Exploration only, and class-free by design: no class floor enters the stress report or its chart. A stress.json is never a conformance record, and quoting one as if it were is a misuse of the tool.

aql-probe

Run the AQL optimization probe against a live, freshly seeded SUT.

veredictum aql-probe --root <ROOT> --ixit <IXIT> --out <OUT> \
    [--corpus-class <CLASS>] [--seed-workers <N>] [--requests <N>]

Fires the measurement machinery’s AQL vocabulary, records wire percentiles per probe, and attributes the database-side cost through pg_stat_statements. This is evidence for someone optimizing a server, and it is never a conformance record.

stress-compare

Render the cross-SUT stress overlay from two committed stress reports.

veredictum stress-compare --left <LEFT> --left-label <LABEL> \
    --right <RIGHT> --right-label <LABEL> --out <OUT>

Deterministic, and both directions on equal footing: the two curves are drawn by the same code from the same kind of file, so neither side gets a rendering advantage.

perf-assets

Render the published performance SVG assets from a committed results.json.

veredictum perf-assets --root <ROOT> --results <RESULTS> --out <OUT> \
    [--summary <PATH>] [--stress <STRESS>]

conformance-assets

Render the capability heat grid and the per-chapter outcome bars from committed party artifacts.

veredictum conformance-assets --root <ROOT> --results <RESULTS> \
    --verdicts <VERDICTS> --out <OUT> [--suffix <SUFFIX>]

Both renderers are deterministic over files already committed, so a build job can regenerate them and diff the result. A hand-drawn number in a published chart is a build failure rather than a review comment.

emit-schemas

Write the published JSON-Schema set.

veredictum emit-schemas --out <OUT>

Byte-deterministic. The schemas in the repository’s schemas/ directory are this command’s output, drift-tested against it, which is how the published format and the code that reads it stay one thing. Author against these if you are writing your own catalogue or your own harness.

The web console

The console is the instrument with a browser in front of it. It ships as its own container image, reads the catalogue and the vendored specification text from paths you mount, and reaches the engine only through the published veredictum crate: every number it shows is one the command line prints too.

Every screenshot below is captured by the console’s own browser journeys (scripts/ui-e2e.sh with UI_E2E_DOCS_SHOTS=1), in one 1440×900 browser window, light and dark. They are refreshed in the pull request that changes the interface, so what you see here is the interface that shipped.

The landing

The four counts are the catalogue’s own: case cores, operation bindings, party statements, and validate findings. A findings count above zero means the catalogue itself needs attention before any server is graded.

The catalogue explorer

Chapters first, each with the number of cases it carries. The search and page state live in the URL, so a view is shareable and survives a refresh.

One chapter lists its cases: the identifier, the kind, and the test purpose. One behaviour per case, so a red row names one defect.

One case in full

The case detail carries the description, the specification citations the expectation stands on, the operation bindings that realize it on the wire, and the corpus fixtures it uses. The citation list is the point: an expectation is refuted by a better reading of the cited text, and by nothing else.

The run wizard

Grading starts at Connect: the CDR base URL, the display identity, the authentication choice from the ixit’s own vocabulary, and a probe whose answer renders verbatim before anything continues.

Scope takes the claim the run grades: the vendor’s own statement.json (the ICS, the document that says which profiles and capabilities are claimed) pasted into the box, or a committed example loaded into it. The document is held to the published statement schema before anything is stored, and saving answers with the claim overview — product, claimed tiers, capability count — beside the selection preview, so the screen says what will run before anything starts. An empty box is an honest no-claim run. The verdict later certifies exactly the pasted claim against the recorded evidence.

The live screen renders the engine’s own progress stream: the case counter, the elapsed clock, a moving-median estimate labelled as such, and the tail of the engine’s output. When the run finishes the outcome links straight to the record.

The record: results and verdicts

Results reads the finished run’s own record, red rows first. A row links to its detail: the recorded reason beside the case’s specification citations, because a red row names a defect in exactly one of three suspects — the server, the runner, or the catalogue — and the cited text is the reference.

Verdicts is the same pure function the command line runs, over the same record: the profile matrix with its coverage bounds printed, and the rendered documents byte-for-byte.

Two real servers, side by side

The same catalogue, the same wizard, the same judgement — driven against two live CDRs pulled at their latest published images: FerroEHR’s quickstart and EHRbase’s official pairing. The run behind each column is the EHR-service case family. The point of the pairing is the comparison: one instrument, two records, and every difference below traces to a case id and its citation.

FerroEHR (latest)	EHRbase (latest)

These captures come from the same E2E harness that gates the console (UI_E2E_REAL_SUTS=1 scripts/ui-e2e.sh): the browser drives the real wizard against the real servers, and the book shows what it photographed.

The conformance method

• The oracle is the released specification text
• The catalogue’s shape: one behaviour, one case
• Positive and negative testing carry equal weight
• When a run goes red
• Where the specification does not say
• Verdicts are computed, never asserted
• Coverage is a mandate

A conformance verdict is only worth what its method is worth. This chapter sets out the method: what counts as authority, what happens when a run goes red, why the catalogue tests refusals as hard as it tests successes, and what the instrument does where the specification does not say.

The oracle is the released specification text

The vendored openEHR specification text is the oracle, and it is never a suspect. Every expectation in the catalogue cites the section it comes from, and the only thing that can refute an expectation is a better reading of that text. Not a maintainer’s confidence, and not what every other CDR happens to do.

Two components are both oracles for a functional case. The Service Model anchors the operation and the naming the cases use; the ITS-REST specification binds that operation to the wire. Where the ITS-REST prose is silent, the released OpenAPI bundle shipped with the same release grounds the expectation, cited as the OpenAPI document and never passed off as prose. It loses on any conflict with the prose.

Some sources look like authority and are not. The official openEHR Conformance component never reached a stable release: its Platform Conformance Test Schedule says which behaviours are worth testing, which is genuinely useful, but it does not settle what the correct answer is, and in places it contradicts a released component. Its Robot test suites are in the same position. Where any of those disagrees with a released component, the released component wins.

The catalogue’s shape: one behaviour, one case

A case core describes one narrow conformance requirement, in the Service Model’s vocabulary, with no status code, header or media type in it. A separate operation binding says how that operation reaches the wire on a given interface specification. That split is what lets the same case be graded against a future wire specification without rewriting the behaviour it tests.

Errors in a case are expressed as kinds, never as codes. The catalogue distinguishes a duplicate EHR from a non-existent EHR from a missing template from a validation failure, because those are different behaviours; the mapping from a kind to a status code lives in the binding, since the same kind maps to different codes on different operations.

Cases stay small on purpose. One behaviour per case means a red row names one defect, which is the difference between a report a vendor can act on and a report they have to reverse-engineer.

Positive and negative testing carry equal weight

A server that accepts everything passes every positive test. That is why the catalogue treats a refusal as a first-class behaviour with its own cases.

Every invalid shape in the corpus is kept as its own entry, marked invalid, with the defect it carries and the specification reference that makes it invalid. Each one has a valid twin. The valid twin proves the server accepts what it must accept; the invalid twin pins the refusal, so a lenient server fails it. Deleting an invalid shape would silently narrow the claim, so it never happens.

The negative surface the catalogue covers includes:

• Content negotiation. An Accept header nothing can satisfy must produce 406; an unsupported payload media type must produce 415.
• Identity and state conflicts. A duplicate identifier, a stale preceding version, a missing version precondition, a delete of something already deleted.
• Content validity. Missing mandatory attributes, empty lists where the Reference Model requires at least one member, structurally wrong documents, and content committed against the wrong template.
• Scope negatives. A format applied where the specification does not define it.
• Authentication. A sweep over the route table checking that every platform route refuses an unauthenticated request.

An invalid payload cannot be authored through a typed model, because a typed model will not construct it. Those fixtures are therefore raw bytes, which is also what makes them catch encoder and decoder bugs that a construct-then-serialize fixture cannot reach.

When a run goes red

A red row is a statement that the specification, the catalogue and the server do not all three agree. Which one is wrong is decided before anything is changed.

Two reflexes are both refused, and the second is why this instrument exists separately from any server:

1. “The catalogue must be wrong, the server is right.” This is the reflex a vendor brings to a failing conformance run. The catalogue answers it by construction: an expectation traces to a citation, so it is refuted by a better reading of the released text and by nothing else.
2. “The server must be wrong, the instrument is right.” Veredictum is the thing people are being asked to trust, so it is a suspect on every red row, ahead of the server. The first live triage attributed 7 of 7 diagnosed defects to the instrument and none to the server under test.

Each red row is attributed to exactly one of three suspects:

The derivation per row is the same every time: read what was actually sent and received, read what the case and its binding claim to expect and why, then read the governing released specification text first-hand and derive independently what a conformant server must return for the exchange that actually happened. The three-way comparison names the defect.

Two rules keep that honest. A transport fault or a step that could not be resolved classifies as inconclusive on the instrument’s side, never as a server failure. And an attribution naming the server carries the reproduced exchange plus the citation with the decisive sentence quoted, because a verdict about somebody else’s product is held to a higher evidence bar than a verdict about our own.

Warning

A catalogue expectation is never adjusted to match observed server behaviour. A server response is evidence in the three-way comparison. It is not the reference, and treating it as one is how a conformance suite quietly becomes a description of whatever the last server did.

Where the specification does not say

Sometimes the released text leaves a behaviour undefined, or two released documents disagree. Inventing an expectation there would fail every conformant server and teach nobody anything. Hiding it would be worse.

Instead, each such case is recorded in the ambiguity register with four things: the ambiguity, the first-hand citation that establishes it, the handling a runner must apply, and a typed disposition the pipeline branches on.

The dispositions are a closed set:

Four properties make this a transparency mechanism rather than an excuse list:

• An entry has to be proven. A claimed ambiguity that the specification actually defines is a catalogue defect. The entry is removed and the case becomes gating.
• Nothing is absorbed. A report_only or editorial entry must carry a link to an upstream report, and that requirement is enforced by the schema rather than by review. A carried divergence always has an outbound report attached.
• Every case still runs. The register is not an exclusion list. It governs how a spec-silent expectation is derived and whether a genuinely open question gates a certificate.
• A private resolution is non-conformant. A harness that quietly decides an ambiguity for itself is not implementing this schedule.

report_only is a cited, upstream-linked suspension, and it reverts to gating the moment the upstream question resolves. It is not a way to make red rows disappear.

Verdicts are computed, never asserted

A verdict is a pure function of the party’s statement, the recorded results, the catalogue and the capability matrix. The recording step and the judging step are separate commands, so the record survives the judgement and anyone holding the same files can re-derive the same documents.

Measured performance works the same way. A class verdict is re-derived from the HDR histograms embedded in the record, so the stored summary is tamper-checked rather than trusted, and latency is measured from the planned arrival instant under open-loop offered load, which is what stops coordinated omission from hiding a stall.

Coverage is a mandate

A green run over a thin catalogue proves nothing, so coverage is machine-checked rather than asserted. A gate enumerates the wire surface from the released sources alone, the Service Model’s platform interfaces crossed with their ITS-REST branches, and fails on any operation, status-code branch, header rule, negotiation variant or error family that has neither a covering case nor a cited exception.

A behaviour the specification defines and the catalogue misses is a gap to close or an honest boundary recorded in the register. Cases are added; they are never removed to make a run go green.

Authoring the catalogue

• The artifact families
• A case core
• Operation bindings
• The closed vocabularies
• The corpus, and why invalid fixtures stay
• The rules an author works under
• Checking your work

The catalogue is data, not code. You can read it, diff it, and add to it without touching the runner, and a harness written in another language can execute the same files. This chapter is an overview of its shape and the rules an author works under.

The full grammar is the published JSON Schema set, in schemas/. Those files are emitted by veredictum emit-schemas and drift-tested against it, so the published format and the code that reads it are one thing. Author against them; this page will not repeat every field.

The artifact families

Everything lives under one artifact root, which is artifacts/ in the repository.

Chapters under schedule/ follow the openEHR service components: ehr, composition, contribution, directory, query, the three definition_* chapters, demographic, admin, messaging, security, smart, simplified_formats, system and content. A separate performance family holds the measured-workload journey definitions, which are not case cores and do not carry capabilities.

A case core

One file, one behaviour. The fields that shape every case:

• id is a global identifier and is never reused. A retired case keeps its identifier with status: retired.
• test_purpose is the one narrow conformance requirement the case exists to check, in prose. If you cannot write it as one requirement, the case is really two cases.
• spec_refs are the citations, by component, document and section. They are resolved against the vendored specification tree by validate, so a citation that does not resolve is a finding rather than a comment.
• capabilities are the capability names whose verdict this case bears. Keep the list minimal: a failure marks every capability listed. Capabilities the case merely touches go in exercises, which is informative and bears no verdict.
• requires is typed prerequisite state, not prose. An empty server, a provisioned template, an EHR with no commits, a folder tree. Each provisioned object mints a named handle the flow can reference.
• flow is the ordered steps of a functional case: which operation, with what arguments, what outcome kind is expected, and what to capture from the response for a later step to use.
• parameters is the data-set dimension. A test is one case run against one data set, so a value matrix in a case produces one test per row, with preconditions re-established around each row by default.
• postconditions are typed assertions evaluated after the flow.

Two more fields carry the honesty:

• applies and guards state when a case is applicable at all, by specification version range or by a cited run condition. A failed guard produces not-applicable with its citation, never a silent skip.
• ambiguities lists the register entries the case is subject to.

What a case core deliberately does not contain: any status code, header name, or media type. Those live in the binding.

Operation bindings

A binding is the wire layer for one Service-Model operation on one interface specification. It maps each outcome kind the case can expect to the wire result that realizes it, and it says what to capture from the response.

The mapping is per operation, because the same kind lands differently in different places. A validation failure is one status code on a composition operation and a different one on EHR creation, and the released specification is what decides which. A kind a binding cannot map is a validation finding.

There are 247 bindings today, and validate checks that every operation a case calls has one.

The closed vocabularies

Three of the vocabularies are closed enumerations, and that is what makes the catalogue machine-checkable.

• Outcome kinds. A case says created, not_found, precondition_failed, validation_failed and so on. There are 26 of them, extensible only by a schedule release, and a case that speaks anything else fails validation. Codes never appear in a case.
• Selectors. How an assertion addresses part of a response body or a header, including the ignore-sets that let a case compare a document while ignoring the fields a server legitimately assigns.
• The capability matrix. The capability, family and profile-tier structure the verdict machinery computes over. This is the openEHR Platform Profiles book’s capability tables as data.

The wire surface under vocab/ is the fourth machine-readable list, and it is the input to the coverage gate: the operations, status-code branches, header rules, negotiation variants and error families the released sources define. A behaviour in that list with no covering case and no cited exception fails the gate.

The corpus, and why invalid fixtures stay

A corpus entry carries its own adjudicated verdict: valid, or invalid with the defect it carries and the specification reference that makes it invalid.

Every invalid shape has a valid twin. The valid twin proves acceptance; the invalid twin pins the refusal, so a lenient server fails it. Removing an invalid fixture narrows the claim without changing any visible count, which is why it does not happen.

Invalid fixtures are raw bytes. A missing mandatory attribute, an empty list where the Reference Model requires a member, an undeclared key: none of those can be constructed through a typed model, because a typed model refuses them. Raw bytes are the only way to author what a reader must reject, and they catch codec defects a construct-then-serialize fixture cannot reach.

Vendored breadth packs carry the same discipline. A pack is exercised in full, with any skip adjudicated and recorded, so a pack never sits in the tree implying coverage it does not have.

The rules an author works under

1. Cite the released specification. An expectation without a resolving citation is a finding. The openEHR Conformance component’s test schedule is a guide to which behaviours are worth covering, and it is not authority for what the correct answer is.
2. One behaviour per case. If a failure of your case could mean two different defects, split it.
3. Never encode a status code in a case core. That is the binding’s job, and keeping the split intact is what lets the behaviour outlive the wire specification.
4. Add cases; never remove one to go green. A removal that makes a run pass is a narrowed claim, not a fix.
5. Never adjust an expectation to match an observed response. Change it only with a new citation that says the old reading was wrong.
6. Record a specification silence, do not resolve it. It goes in the ambiguity register with a typed disposition and an upstream report, and a private resolution makes a harness non-conformant.
7. Keep the invalid twin. See above.

Checking your work

veredictum validate --root artifacts --specs specs/openehr

Zero findings is the only passing result. The command checks identifier uniqueness, citation resolution, binding completeness, coverage of the enumerated wire surface, and claim completeness against the committed party statements, and it exits non-zero if any of them has something to say.

Run it before you drive anything against a server. A catalogue defect found by a server run costs a great deal more to diagnose than the same defect found by validate in a second.

Glossary

• The parties and the claim
• The catalogue and the run
• The verdict
• The authorities

The conformance vocabulary comes from two places: the ISO conformance-testing tradition (ICS, IXIT, SDoC) and the openEHR specifications themselves (RM, AQL, the component names). This page defines every term the instrument and the console put in front of you.

The parties and the claim

• Party: the organization whose product is being graded — the vendor making a conformance claim. Each party directory under party/ carries the two documents below.
• ICS (Implementation Conformance Statement): the party’s claim document, statement.json. It declares which profile tiers and which capabilities the product claims. A verdict certifies this claim against the recorded evidence, so a run without a statement produces results but no verdict. The term comes from the ISO/IEC 9646 conformance-testing methodology.
• IXIT (Implementation eXtra Information for Testing): the deployment facts needed to drive the claim, ixit.json: the endpoint URLs, the authentication mode, the names of the credential environment variables. Same ISO/IEC 9646 origin. The console writes one for you from the Connect form.
• SDoC (Supplier’s Declaration of Conformity): the self-declaration half of the statement, in the sense of ISO/IEC 17050: the supplier declares, the instrument checks, and nobody grades their own homework.
• SUT (System Under Test): the running server a campaign drives — here always a CDR reached over its own REST wire.
• CDR (Clinical Data Repository): a server that stores and serves openEHR clinical data, such as EHRbase or FerroEHR.

The catalogue and the run

• Catalogue: the machine-readable test schedule under artifacts/: case cores, operation bindings, vocabulary, corpora, and the ambiguity register. validate holds it to zero findings before any server is composed.
• Case: one spec-cited behaviour, one file, one row in the record. One behaviour per case is a design rule: a red row then names exactly one defect.
• Case id: the stable identifier of a case, interface first, such as I_EHR_SERVICE.create_ehr-main.
• Case-id filter: a plain substring match over case ids — the engine’s --filter flag. Every case whose id contains the typed text is in scope; an empty filter means the whole catalogue. I_EHR_SERVICE. selects the EHR-service family, create_ehr-main selects one case. The filter narrows the run, never the claim: a claimed capability whose cases were filtered out is reported not_evidenced, so a narrow run cannot pose as full coverage.
• Corpus: the committed wire payloads the cases post, valid and invalid. An invalid entry is deliberate: it pins a refusal the spec requires, so a lenient server fails it.
• Excused: a case the run did not hold against the server, with the citation that permits the excuse — an out-of-claim capability, or an instance the party does not declare. Excuses are printed, never silent.
• Ambiguity register: the record of behaviours where the released specifications are genuinely silent or contradictory, each with a typed disposition and, where one exists, its upstream report. Silence goes here; it is never resolved privately.

The verdict

• Verdict: the computed answer to the claim, produced by a pure function over the statement, the record, the catalogue and the capability matrix. It is computed, never asserted, and re-running it over the same record produces the same bytes.
• Profile tier: one of the CNF profile levels a party can claim — CORE, STANDARD, OPTIONS, and the SEC-BASIC security family. A tier passes only when every capability it requires is passed.
• Capability: one named unit of claimable behaviour, such as EhrOperations or QueryProvisioning. The capability matrix maps each tier to the capabilities it requires, and each capability to the cases that evidence it.
• Evidence tokens: the per-capability answer in the matrix. passed (every selected gating case passed), failed (one failed), inconclusive (one errored and none failed — never counted against the server, but blocking green), not_evidenced (claimed, and no case produced a gating pass or fail), not_claimed (absent from the party’s ICS).
• Errored (inconclusive): a row whose exchange could not be judged — a status mapping to no declared outcome, a transport fault. By the attribution law this is never a server failure; it is a defect in the runner or the catalogue until adjudicated.

The authorities

• Oracle: the released openEHR specification text vendored under specs/. It is the only authority an expectation may cite, and it is never a suspect when a run goes red.
• CNF: openEHR’s conformance specification component. Its Platform Conformance Test Schedule names which behaviours to cover; the released components say what the correct answer is.
• Spec components: RM (the Reference Model — the data structures), AQL (the Archetype Query Language), AM (the Archetype Model), BASE (foundation types and identifiers), TERM (terminology), SM (the Service Model — the abstract operations), ITS (the Implementation Technology Specifications — REST, JSON, XML: how the operations land on a wire).
• Attribution law: the discipline applied to every red row before anything changes: the failure belongs to exactly one of three suspects — the server, the runner, or the catalogue — decided by comparing spec-required against catalogue-expected against observed, with the cited text as the reference.