The Cold-Reproduction Runbook
The test that matters is a cold start: a reader with no access to us downloads the deposit and reproduces the result across independent environments.
What 'cold' means
A warm reproduction reruns code on the same machine that produced it — a weak test, because the environment is already tuned. A cold reproduction starts from nothing: a reader downloads the deposit onto a fresh environment, follows a written runbook, and recovers the result without any help from Kronos. That is the reproduction that actually demonstrates the work stands on its own.
The runbook's structure
- Fetch: download the deposit by DOI and verify checksums against the published values.
- Build: recreate the pinned environment so the code has the dependencies it expects; see environment pinning.
- Run: execute the pipeline from inputs to outputs with a single documented command.
- Verify: compare outputs to the reference using the tiered check; see the verify function.
Across multiple environments
Kronos's deposits are exercised across more than one independent environment before release — the AI and quantum deposit, for instance, follows a multi-environment cold-reproduction procedure. Running in several environments catches results that depend on a hidden feature of one machine, which is exactly the kind of fragility a design study must find before hardware does.
Why we invest in the runbook
A deposit without a runbook is a pile of files a motivated expert might eventually reassemble. A deposit with a clear cold-reproduction runbook is an invitation any competent reader can accept. Lowering that barrier is how we turn openness from a claim into a practice; see independent reproduction.
If a stranger cannot reproduce it cold, we do not consider it reproduced.