# DataFresh

> Is this dataset or API still fresh? Newest record, update cadence check and schema drift. Point DataFresh at a public JSON or CSV endpoint (or paste the content). It finds the records, infers a schema with a stable fingerprint, finds the newest timestamp, compares its age with the update cadence you expect, and reports schema changes since the last check – as one of: healthy, stale, schema_changed, unavailable, no_timestamp.

Part of usefulweb.site (https://usefulweb.site/). Results are deterministic calculations from the inputs and the listed assumptions; the API returns exactly what the web page shows.

## Use it
- Human interface: https://datafresh.usefulweb.site/ (interactive tool at https://datafresh.usefulweb.site/tool)
- JSON API: POST https://datafresh.usefulweb.site/api/v1/calculate with a JSON object of inputs (schema: https://datafresh.usefulweb.site/api/v1/tool)
- OpenAPI: https://datafresh.usefulweb.site/openapi.json
- MCP (Streamable HTTP): https://datafresh.usefulweb.site/mcp - tool `datafresh`

## Inputs
- url (url): Dataset or API URL - Public http(s) JSON or CSV, max 2 MiB.
- content (textarea): Or paste JSON/CSV content
- format (select): Format
- timestampField (text): Timestamp field (optional) - Leave empty to detect (updated_at, date, …). Nested: parent.child
- expectedCadenceHours (number): Expected update interval - 24 = daily, 168 = weekly, 720 ≈ monthly, 8760 = yearly.
- previousSchema (textarea): Previous schema (optional, JSON from an earlier result)
- remember (boolean): Remember this URL's schema for the next check - Stores only a hash of the URL plus the field list.

## Method
- Records: a JSON root array, or the largest array of objects within 3 levels; CSV uses the first row as header (comma or semicolon).
- Schema: every field (one nesting level flattened as parent.child) with its observed value types and presence ratio; the fingerprint is the first 16 hex chars of SHA-256 over sorted "field:types" lines.
- Timestamp: the field you name, otherwise the first field named like updated/modified/timestamp/date/time/year where ≥ 80% of values parse as dates (ISO 8601, epoch s/ms, YYYY, YYYY-Qn, DD.MM.YYYY). Years count as 31 Dec, quarters as their last day.
- Freshness: age = now − newest timestamp. healthy if age ≤ 1.5 × expected interval, otherwise stale. Without a timestamp field the HTTP Last-Modified header is used if present.
- Drift: compared with the previous schema you paste or, with "remember", the schema stored for the same URL (key = SHA-256 of the URL). Any added/removed field or type change → schema_changed (takes precedence over healthy, not over unavailable).