Benchmarking Against Known Solutions
We test our tools on problems whose answers are already known before trusting them on problems whose answers are not.
Earn trust on the known before using it on the unknown
A tool is only as trustworthy as its record on problems we can check. Before a solver is used for a design number, we run it against benchmarks with established answers -- analytic cases, published experiments, and standard test problems -- and confirm it reproduces them within tolerance. A tool that fails a known case has not earned the right to predict an unknown one.
Kinds of benchmark
- Analytic: problems with exact closed-form solutions.
- Cross-code: comparison against an independent, established solver on the same problem.
- Experimental: reproduction of published measurements from operating devices.
- Community: standard benchmark suites others have documented.
Benchmarks bound the extrapolation
Benchmarking also tells us how far we can trust a tool away from where it was checked. For the breeder (Hyperion), much of the physics sits within or near regimes that operating devices and their data can anchor. For the burner (Aegis / MetroVolt), the honest finding is that the plug regime is 166-830x beyond any operated device, so no benchmark reaches it -- which is precisely why we call that prediction un-post-dictable today rather than dressing it as validated.
A benchmark is a promise kept in advance: this tool behaved correctly where we could check, so here is exactly how much confidence its untested predictions deserve. Kept up as a standing suite rather than a one-time exercise, benchmarks also catch the day a trusted tool silently stops reproducing a case it used to pass -- the earliest warning that something downstream may now be wrong.