How fast is each openEHR CDR, measured the same way
Every row below is a benchmark record somebody ran with the
veredictum bench command and submitted as a pull request to this
repository. The record carries the pack it drove, the seed it drove at, every
repetition, the machine it ran on, and the reference deployments it was measured
against on that same machine. CI validates all of that before a maintainer looks
at it, and the merge is the acceptance.
What each pack actually creates and measures → That page is generated from the packs the instrument embeds, so it says what a row below was produced by rather than what anyone remembers it being.
A bench number is not a conformance verdict. This board reports comparative speed. It is not a conformance record, not a certificate, and not a performance-class rating; a bench result may motivate a class run, never substitute for one. A fast server that fails the catalogue is a fast server that fails the catalogue.
Two reference baselines on every row
Each submission measures the pinned reference CDRs on its own machine, in the same session, with the same pack at the same seed, so every row carries one index per reference. An index is the median latency of the submission divided by the same statistic from that reference: below 1.0 is faster than it, above 1.0 is slower. Ordering a list needs a single ruler, so the rows are sorted by the FerroEHR index; the EHRbase index sits beside it on every row and is the same measurement against the other reference.
No submission has been merged yet. The first one to arrive will be the first row.
Written for someone who has never seen the instrument
Why a ratio and not milliseconds. A latency in milliseconds describes a system and the machine it ran on at the same time, so two numbers taken on different hardware cannot be compared. Every submission therefore measures the reference CDRs on its own machine, in the same session, with the same pack at the same seed. Dividing one median by the other cancels the machine out.
Why two references and not one. A single reference makes the board a verdict about that one product. Two independent ones, measured under identical container ceilings in the same session, show whether a row is fast in general or only fast against one comparison. A row whose two indices disagree sharply is telling you something the ordering alone does not.
What the absolute figures are for. They say what the ratio felt like on one specific machine, which is why they never appear without the fingerprint of that machine beside them. Read them as the scale of the work, never as a claim about what your deployment would do.
What the references are. Each submission composes the pinned reference CDRs from image digests, under identical container ceilings. They are not a standard of correctness. They are a ruler that happens to be the same length on every machine.
Failed arrivals. Every row states how many measured arrivals never produced an answer. A percentile computed over a run that was mostly failing describes the failures, so read the share before the milliseconds. The submission gate refuses a record in which any operation produced no successful arrival at all.
Open-loop, and what that buys. Arrivals fire at their planned instants regardless of whether an earlier request has come back, and every latency is measured from the planned instant. A server that stalls therefore shows the stall in its percentiles instead of quietly issuing fewer requests. Phases that are closed-loop by construction, such as the bulk load, are labelled as such and are reported as throughput, never as a latency claim.
Verified and declared-only. Every row today carries the self-reported tier: the submitter ran the benchmark and the record passed CI, and nobody here re-ran it. A record a maintainer reproduces will carry a reproduced tier, on the same submission channel. Read a self-reported row as a claim its author put their name to in a public git history, and nothing more.
Repetitions. Three is the floor. One repetition measures a moment, so a record with fewer is rejected before it can be ranked, and each figure on the board is the median across repetitions.
Run the pack, open a pull request
The reference pack for this board is community-vitals. Point the
command at your deployment, let it compose both reference baselines on the same
machine, and commit the record it writes.
# The credential is read from the environment; it never rides argv.
$ export VEREDICTUM_BENCH_PASSWORD=…
$ veredictum bench --base-url https://cdr.example/openehr/v1 \
--auth basic --user <user> \
--pack community-vitals --repetitions 3 --with-baselines \
--out ./bench --label "Your CDR 1.2.3"Then copy the record into
benchmarks/submissions/<system>/<date>-<host>.json and open a pull
request. CI checks the schema, the pack pins, the repetition count, the baselines,
the environment fingerprint and the file name, and refuses any edit to a record
already merged. The submission guide, in full →