Changelog

A session that got the drafting work for v1 done: settled the charting tool, finished the story #2 draft, applied the structure benchmark's findings, ran a first enrich rep, and wrote first drafts of both the story and enrich skills. Also trimmed a NEXT.md that had grown into a second backlog.

Charting: Observable Plot. The near-term half of #11 is decided. The line-charts bake-off rates Plot the best chart-per-line-of-code for a data story, and both hand-written stories' friction notes pointed the same way — story #2's normalised ranged-bar scoreboard was about the ceiling of what hand-rolled inline SVG is comfortable for. docs/charting.md records the decision and the pattern: Plot authored in a *-make-charts.mjs, rendered to static SVG in Node (jsdom for the DOM), committed and embedded as a Markdown image — no JavaScript on the published page, same deployment as before. Build-only dependencies live in a scoped site/stories/package.json, the same convention as a dataset's build.ts deps. planetary-boundaries-make-charts.mjs is the reference — ~45 lines for the scoreboard against ~120 hand-rolled, with byte-identical output across runs. The side-by-side that settled it is shipped at site/charting-spike.html. The longer half of #11 — a DataHub/Flowershow-native views standard — stays deferred. The Keeling Curve charts (story #1) stay hand-rolled until touched (#12).

Data story #2 drafted. Planetary Boundaries: the outline + chart plan were committed on 2026-09-05; this session added the two charts and the prose, each its own commit, the #10 way. The scoreboard needed a modelling call the outline's chart plan didn't survive: current ÷ boundary is undefined for novel entities (boundary = 0) and wrong-signed for the "less is worse" indicators, so it uses the framework's own normalisation instead — (current − pre-industrial) / (boundary − pre-industrial), boundary = 1×. Six of nine boundaries crossed; the counter-note is stratospheric ozone, dipping toward the boundary through the twentieth century and recovering after the Montreal Protocol. Left: the author's "sounds like me" voice pass. #4.

structure benchmark findings applied. All eight edits from docs/structure-benchmark.md (benchmark v1, 2026-09-05) landed. In scripts/wrangling-idioms.mjs (tests 24 → 28): excelSerialToIsoDate() — an offset-free Excel-serial → ISO date converter, because reading a spreadsheet date through a JS Date shifts it a day in positive-UTC zones and silently turned 1987-05-20 into 1987-05-19 in the oil-prices build; num(raw, sentinels) — a strict numeric parser with a per-column "no data" sentinel list; and toCsv() upgraded to real RFC 4180 quoting. In skills/structure/SKILL.md: exceljs vs SheetJS xlsx for legacy .xls, the timezone-naive-date rule, naming conventions vs a published dataset, LF line endings with no Frictionless dialect block, and a "build needs a dependency ⇒ its own package.json" section. AGENTS.md picked up the exceljs/SheetJS split and the dependency rule.

story skill — first draft. skills/story/SKILL.md, from the two hand-made stories plus the craft docs (story-craft.md, voice-guide.md, charting.md). It is the workflow — three committed, independently reviewable artefacts (outline + chart plan → charts → prose), the outline template lifted from the two stories, a definition of done, and the "sounds like me" voice pass kept as a separate human stage — and it references the craft docs rather than restating them. Deliberately not symlinked into .claude/skills/: it is there for review, not activation, until it has been run on a real story (#10).

enrich skill — first draft, after one hand rep. The rep: enrich.ts

• SUMMARY.md for co2-ppm — a reproducible zero-dependency stats script (per-resource n / missing / min / max / mean / median / std dev, skipping time-encoding identifier columns) that rewrites only a fenced block in SUMMARY.md, plus a hand-written "What stands out" (the Mauna Loa uncertainty columns are constant and carry no information; the growth rate is never negative; the rise is accelerating; two decadal rows are partial) and "See also". status bumped structured → enriched. The skill (skills/enrich/SKILL.md) generalises that: enrich.ts + fenced SUMMARY.md + one to three declarative views for first charts (not Observable Plot SVGs — those stay a story's job) + the status bump. Also DRAFT, also unsymlinked, pending a second rep (oil-prices, to settle how parallel same-schema resources are summarised).

NEXT.md trimmed; roadmap moved to an epic. The "Roadmap to v1" arc and a growing "Already done" log had turned NEXT.md into a second backlog plus a duplicate changelog. The five-step arc is now epic #14; NEXT.md is back to a short pointer. site/README.md's status and roadmap sections, stale since before the benchmark, were refreshed to match.

A session aimed at getting the skills better by benchmarking them and adding samples, not just adding features.

structure benchmark. docs/structure-benchmark.md scores the wrangling skill across three real runs — the NOAA text file (comment lines, sentinels), the Project Drawdown markdown table, and a new one: eight legacy .xls workbooks of EIA oil prices. The oil-prices build is a ground-truth test — its output is content-identical to the long-running community datasets/oil-prices (the only difference is LF vs CRLF line endings). The benchmark turned up a real bug class the skill doesn't warn about — Excel serial dates are timezone-naive, and parsing them via JavaScript Date shifts every date by a day — plus four other prioritised skill edits, all filed as tracked issues. The edits aren't applied yet; the point of this pass was to gather the evidence first.

New datasets. oil-prices (Brent + WTI spot prices, four frequencies, from EIA). And co2-ppm gained the global CO₂ series, NOAA's published growth rates, and a decadal-mean table.

Project Drawdown moved to its own repo — https://github.com/datasets/project-drawdown.

Story craft, written down. docs/story-craft.md (what makes a good data story) and docs/voice-guide.md (the house voice — let the numbers carry it), drafted from the best practitioners and story #1.

Data story #2 begun. Planetary Boundaries — source snapshot archived and the outline + chart plan committed, the #10 way (argument signed off before prose). Prose still to come.

DataPressr now has a published site — https://datapressr-2-rufuspollock.flowershow.me — with a landing page, a running "what to review now" list, and a datasets catalogue, built from site/ with the Flowershow CLI. It also has a written roadmap to v1 (prove structure on a messy source → write 1–2 data stories by hand → pick a charting standard → write the enrich/story skills → ship the site) tracked in issues #6–#13, and it started from a candid status pass that named what wasn't real yet: at that point only the validator was genuinely tested, structure had had one easy trial, and no data story existed. The rest of the same session then closed the first two of those gaps.

The skill playbooks moved out of .claude/ into a top-level skills/ directory in the flat layout npx skills expects, so any agent can install them — npx skills add datasets/datapressr — not just Claude Code, which keeps working through symlinks. The three last-mile commands (init, validate, push) became SKILL.md playbooks alongside capture, archive, and structure, so there is now one consistent format for all six.

Project Drawdown's 2020 Table of Solutions was run through structure: a reproducible build.ts, two tidy typed CSVs (82 solutions × 2 scenarios; the solution-to-sector links), validation clean. drawdown.org's own site terms are all-rights-reserved, but the dataset is just the numbers — measured and modelled emissions-impact estimates, which are facts and not per se copyrightable — restructured into tidy tables, not the prose or figures. We release the compilation as PDDL-1.0 and attribute Project Drawdown as the source. It still needs relocating into the datasets/climate-and-environment catalog repo before it goes live.

The structure skill got its first real run against a messy primary source: NOAA's Mauna Loa CO₂ record, from the raw text file (40 comment lines, negative "no data" sentinels, no date column) to a clean typed dataset with a reproducible build.ts. Diffing it against the long-running community dataset at github.com/datasets/co2-ppm turned up a live bug there — NOAA restructured the CSV, the community version's shell updater didn't follow, and its published monthly file now has its last two columns mislabelled with nothing to catch it. This build asserts the source header and stops if the shape changes; the lesson went back into the skill as a "government/scientific text data" idioms note.

On top of the dataset: the first hand-written data story — the Keeling Curve, what it is and what it says (316 ppm in 1959, 427 in 2025, up every single year). It's built from a committed outline, and it's deliberately a story about the data, not about the wrangling — the chart leads, the "how this was made" part is a short section near the end. Charts are hand-rolled SVG for now: the near-term policy is just to make charts that look good by whatever's easiest, and leave picking a standard mechanism (that DataHub or Flowershow support natively) to a later investigation.

DataPressr moves from three thin publishing commands to the start of an actual, opinionated skill suite for turning raw data into published datasets — and, unusually for an AI-authored playbook, the parts that can be tested now are.

/validate is no longer an LLM re-deriving a checklist from prose each run — it's a small, dependency-free script with its own test suite, checking a dataset's schema, license, and source metadata before anything gets published. A new structure skill walks through turning a messy raw source into a clean, typed dataset, written and verified against two datasets already on DataHub — including a genuine gotcha caught while parsing a 27MB multi-sheet spreadsheet (Excel formula cells silently breaking a naive parser, now documented with the fix). Its cleanup patterns are themselves a tested module, not just prose. The wrangling approach changed after testing against real data, too: an initial DuckDB-first decision got reversed for Node/TypeScript once it was clear no dataset in the project actually used DuckDB.

Every dataset now also needs a recorded license and source before it's more than a stub — see AGENTS.md for the full set of conventions and the structure skill for the playbook.

Separately: BACKLOG.md, INBOX.md, and DASHBOARDS.md are retired in favour of GitHub issues and a short NEXT.md — a queue that can actually be searched, instead of a growing markdown file.