Reproducible Datasets
A reproducible dataset can be regenerated exactly from its recorded inputs, code, and environment, or independently verified against a stated tolerance.
Reproducibility as a property of data
A dataset is reproducible when someone else, given what you provide, can regenerate it or confirm it matches. This is stronger than merely sharing files: it requires that the recipe be complete. Reproducibility is the property that turns a dataset from a claim into evidence.
The ingredients
- Inputs: raw data, pinned by content hash or version.
- Code: the exact transformations, under version control.
- Environment: library versions and system configuration, captured explicitly.
- Parameters and seeds: every setting that affects the output.
- Expected outputs: with checksums or tolerances for verification.
Two tiers of reproduction
Bit-for-bit reproduction means the regenerated bytes match the original exactly, verified by checksum. It is the gold standard but is not always achievable, because parallel arithmetic and hardware differences can produce tiny numeric variations. Tolerance-based reproduction accepts results that agree within a stated numerical bound. A rigorous deposit states which tier applies and provides the means to check it.
Environment capture
The environment is the most commonly forgotten ingredient and the most common cause of failed reproduction. Recording exact library versions, or providing a container image that pins the whole software stack, removes the works on my machine problem. See model versioning for the same discipline applied to fitted artifacts.
In the Kronos record
The deposited Kronos datasets follow a cold-reproduction runbook: fixed inputs, recorded environment, and a two-tier check, byte-level where possible and tolerance-based where numerics require it. A reproduced result of the breeder operating point, for example, must match the deposited values within the stated bound. This is what lets the published physics basis be independently confirmed rather than merely asserted.