Reality Check 1.1.0 skill release prep: evidence labels, bilingual triggers, regression harness

What they did

The author prepared version 1.1.0 of Reality Check, a skill that audits a landing page and returns one verdict, exactly three damage-ranked fixes, and 1-3 locked elements. The core structure was kept; the work was an audit of the skill itself against seven questions, then a commit-by-commit rewrite. First, the root skill copy was synced byte-for-byte with the package folder. Then the instruction text was rewritten: Hebrew script triggers moved into the description (the only part that drives activation), with one canonical trigger list in tests/triggers.json checked against the description, SKILL_HE.md, and both READMEs, plus new negative triggers for performance audits, interior pages, and 'reality checks' of plans or code. Diagnosis rules were added: stop on a client-rendered JavaScript shell instead of judging it, and under Ship an empty slot reads 'No further fix needed' rather than inventing a problem. Duplicated rules were stated once, cutting the body from 1,646 to 1,511 words while adding rules. Every fix is now tagged [verified] / [observed] / [assumed] (with Hebrew equivalents), a Basis line names the input and what was not checked, and 'dead' or 'static' claims require verification. Docs followed: a full Hebrew SKILL_HE.md translation, a new README.he.md, a changelog, and real reference outputs in both READMEs. A regression harness and CI were added: five cases with reference outputs, alternative valid answers, counterexamples, contract samples, an output checker with unit tests, and a package checker. A handoff doc for ChatGPT Work was written, then Codex review fixes and five checker bug fixes landed, each reproduced before the fix and covered by unit tests. Behavior runs in ChatGPT Work were analyzed: no skill instruction failure was confirmed, but the checker was too strict about em dashes and negation, and case 02 was blocked because no browser could reach the local server. A temporary public page was published on gh-pages for that case.

How it went

The package checker, output checker, and 19 unit tests pass on Python 3.11, 3.12, and 3.13, and skills-ref validates both copies. Nothing was merged, tagged, or released: the author gated release on a fresh blind run of cases 01-05 with zero FAIL and zero BLOCKED, including case 02 against the public page. Case 02 has never run with a real browser interaction, and Grok, Codex CLI, and Codex cloud have not run any case.

Worth knowing

The release is explicitly unapproved and untagged. Case 02's fixture still contains an HTML comment listing the deliberate faults, so a local run can read the answers; fixing it is a one-line follow-up not done in the PR. The gh-pages branch must be deleted and Pages turned off after the run. Earlier failing outputs were not edited or re-scored as passes, so the fixes are unconfirmed until a fresh blind run. The slug reality-check collides with other publishers' skills, and renaming would break installs.

Wiring

Repo layout: SKILL.md at root plus a reality-check/ package folder kept byte-identical and enforced by CI; metadata.json holds version and tags; tests/triggers.json is the canonical trigger list checked against the description, SKILL_HE.md, and both READMEs. Tests run as python3 tests/check_package.py (copies, frontmatter, metadata shape, version vs changelog, triggers and labels across four files, rule anchors, Hebrew parity, body size budget, README examples vs case files), python3 tests/check_output.py --all (reference outputs, alternative answers, counterexamples, blocked sample, contract samples), and python3 -m unittest discover -s tests -p 'test_*.py'. Case fixtures live in tests/cases/ with alt-*.md and counterexample-*.md files; cases marked needs_interaction report BLOCKED (exit 2) when the page was not exercised. skills-ref 0.1.1 validates both copies. Tagging v1.1.0 triggers the existing attestation workflow.

Try it yourself

Audit my agent skill repo for release readiness. Read SKILL.md, the package copy, metadata.json, both READMEs, and any Hebrew translation. Check: (1) triggers live in the description, not the body, and one canonical list is checked against every file; (2) no rule is stated more than once; (3) frontmatter is flat with string values only, and the description is under 1,024 characters and 1,024 UTF-8 bytes; (4) the root copy and package copy are byte-identical; (5) every output claim is tagged verified/observed/assumed with a Basis line naming what was not checked. Then add tests/triggers.json, a package checker, an output checker with unit tests, and a CI workflow that runs all three on Python 3.11-3.13. Report gaps as a table before changing anything.