How fast is each openEHR CDR, measured the same way
Every row below is a benchmark record somebody ran with the
veredictum bench command and submitted as a pull request to this
repository. The record carries the pack it drove, the seed it drove at, every
repetition, the machine it ran on, and the reference deployments it was measured
against on that same machine. CI validates all of that before a maintainer looks
at it, and the merge is the acceptance.
What each pack actually creates and measures → That page is generated from the packs the instrument embeds, so it says what a row below was produced by rather than what anyone remembers it being.
A bench number is not a conformance verdict. This board reports comparative speed. It is not a conformance record, not a certificate, and not a performance-class rating; a bench result may motivate a class run, never substitute for one. A fast server that fails the catalogue is a fast server that fails the catalogue.
Two reference baselines on every row
Each submission measures the pinned reference CDRs on its own machine, in the same session, with the same pack at the same seed, so every row carries one index per reference. An index is the median latency of the submission divided by the same statistic from that reference: below 1.0 is faster than it, above 1.0 is slower. Ordering a list needs a single ruler, so the rows are sorted by the FerroEHR index; the EHRbase index sits beside it on every row and is the same measurement against the other reference.
Rows are grouped by the posture profile they declared, and ranked only inside their group. A number measured with an audit trail on and a number measured with it off describe two different systems, so they are never placed in one ranking. Each row states its profile and which of its items the canaries checked against the running deployment, before and after the measured window.
No submission has been merged yet. The first one to arrive will be the first row.
Written for someone who has never seen the instrument
Why a ratio and not milliseconds. A latency in milliseconds describes a system and the machine it ran on at the same time, so two numbers taken on different hardware cannot be compared. Dividing the submission's median by the reference's, both measured in one session on one machine, cancels the machine out.
Why two references and not one. A single reference makes the board a verdict about that one product. Two independent ones, measured under identical container ceilings in the same session, show whether a row is fast in general or only fast against one comparison. A row whose two indices disagree sharply is telling you something the ordering alone does not.
What the absolute figures are for. They say what the ratio felt like on one specific machine, which is why they never appear without the fingerprint of that machine beside them. Read them as the scale of the work, never as a claim about what your deployment would do.
What the references are. Each submission composes the pinned reference CDRs from image digests, under identical container ceilings. They are a ruler of the same length on every machine, and they set no standard of correctness.
Failed arrivals. Every row states how many measured arrivals never produced an answer. A percentile computed over a run that was mostly failing describes the failures, so read the share before the milliseconds. The submission gate refuses a record in which any operation produced no successful arrival at all.
Open-loop, and what that buys. Arrivals fire at their planned instants regardless of whether an earlier request has come back, and every latency is measured from the planned instant. A server that stalls therefore shows the stall in its percentiles instead of quietly issuing fewer requests. Phases that are closed-loop by construction, such as the bulk load, are labelled as such and are reported as throughput, never as a latency claim.
What was switched on. Every row states the posture profile its run declared: which features were on behind the numbers. The instrument checks each item against the running deployment before and after the measured window, by reading back versions the run itself committed, offering a known-invalid composition, and asking for a compressed response without credentials. An item checked that way is marked verified; audit and tenancy are marked declared-only, because released openEHR REST defines no operation that discloses them. A deployment that contradicted its own declaration had the run refused, so no such row exists. Rows in different profile groups are not comparable with each other.
The tier badge. Every row carries the tier its registry entry states. self-reported means the submitter ran the benchmark, signed the record and it passed CI, and nobody here re-ran it: read it as a claim its author put their name to in a public git history. reproduced means the workflow in this repository composed the deployment from a recipe committed here, drove the pack, and attested the record from that workflow identity. No key stands behind that tier, so there is none to steal.
Repetitions. Three is the floor. One repetition measures a moment, so a record with fewer is rejected before it can be ranked, and each figure on the board is the median across repetitions.
Run the pack, open a pull request
The reference pack for this board is community-vitals. Point the
command at your deployment, let it compose both reference baselines on the same
machine, and commit the record it writes.
# The credential is read from the environment; it never rides argv.
$ export VEREDICTUM_BENCH_PASSWORD=…
$ veredictum bench --base-url https://cdr.example/openehr/v1 \
--auth basic --user <user> \
--pack community-vitals --repetitions 3 --with-baselines \
--out ./bench --label "Your CDR 1.2.3"Then copy the record into
benchmarks/submissions/<system>/<date>-<host>.json and open a pull
request. CI checks the schema, the pack pins, the repetition count, the baselines,
the failed-arrival ceiling the pack pins, the environment fingerprint and the file
name, and refuses any edit to a record
already merged. The submission guide, in full →