Metadata and Data Catalogs
Metadata is data about data; a catalog makes datasets discoverable by recording what they are, where they live, and how they relate.
Data you cannot find is data you do not have
Metadata describes a dataset so it can be understood and used without opening it: what it contains, how it was made, what units apply, who produced it, and when. Without metadata, a file of numbers is a mystery. A catalog is the searchable index of all datasets and their metadata across a program.
Kinds of metadata
- Descriptive: title, summary, keywords, that support discovery.
- Structural: schema, dimensions, units, and relationships between files.
- Administrative: owner, access rights, retention, and license.
- Provenance: how, when, and from what inputs the data was produced.
The catalog
A data catalog collects metadata from across storage systems into one searchable place. Users query it to find datasets by topic, owner, or format, inspect a dataset's schema and lineage before using it, and understand quality and freshness. A good catalog turns a sprawling lake into a navigable library. See catalog search and discovery.
Standards
Metadata standards make catalogs interoperable. Schema.org and DCAT describe datasets for the web; discipline-specific schemas add scientific fields. Using a recognized schema means external tools and other institutions can index and understand your data. This is a pillar of the FAIR principle that data be Findable.
Keeping metadata true
Metadata that drifts from reality is worse than none, because it misleads. The durable pattern is to generate metadata automatically from the pipeline that produces the data, so the description is created at the same moment as the data and cannot fall out of sync. For the Kronos published record, each dataset carries descriptive, structural, and provenance metadata generated at deposit time.