Data Versioning and Lineage

Every training set is content-addressed and traceable: which raw pulses, which transforms, and which labels produced it, so any model can be reproduced from its data.

Immutable datasets, traceable to raw signal

A model is a function of its data, so an untracked dataset is an unreproducible model. Kronos treats every training set as an immutable, content-addressed object: hash the exact rows, transforms, and label revisions, and that hash becomes the dataset's identity. Retraining pins the hash; two jobs citing the same hash are guaranteed the same data.

Lineage runs deeper than the dataset. Each engineered feature vector traces back through its transform graph to the specific raw diagnostic channels and pulse identifiers on the breeder or burner that produced it. When a diagnostic is later found to have been miscalibrated, lineage answers immediately which datasets and therefore which models are contaminated.

What lineage records

• Source pulse IDs and machine (breeder / burner) and campaign
• Raw channel list with calibration revision at read time
• Transform DAG: filters, resampling, feature definitions, versions
• Label source and revision (see label provenance)
• Splits: train/validation/holdout membership by hash

dataset = {
  'id': 'sha256:9f2c...',            # content hash = identity
  'machine': 'breeder',
  'pulses': ['H-2029-0441', ...],
  'features': 'featdef@v7',
  'labels':  'disruption-labels@v3',
  'splits':  {'train':'sha256:...','holdout':'sha256:...'},
  'derived_from': ['sha256:...prev']  # append-only lineage chain
}

This lineage feeds directly into model lineage and the broader metadata governance layer. It is also the mechanism behind incident forensics: after any anomaly, lineage lets an auditor walk from the deployed controller back to the exact raw signals it was born from, with no gaps.