Deterministic guarantees for a Claude Code plugin: read-only reviewer agents, Stop/PreToolUse hooks, and evals

What they did

The maintainer of an agentic-development-workflows plugin moved its most important guarantees out of prose instructions and into enforcement points that Claude Code actually supports, keeping the skill text as a portable fallback for other harnesses. Concretely: (1) made review read-only by dispatching three plugin agents — quality-reviewer, spec-reviewer, standards-reviewer — each granted only Read, Grep, and Glob and preloading its skill via `skills:`, with a validator that rejects any other tool; (2) added a `Stop` hook (`guard-review.mjs`) that sends Claude back once when a turn ran `reviewing-change` without dispatching all three reviewers, regardless of diff size, keeping one marker file per reviewer so parallel dispatch never loses an update, and failing open; (3) added a `PreToolUse` hook (`guard-push.mjs`) that blocks pushes to the remote default branch, `--all`, or `--mirror`, following `cd` and line continuations, disableable via the `guard_default_branch` plugin option; (4) wrote an unattended-run contract into developing-feature, fixing-bug, committing-change, and shipping-change — the prompt is the only authorization, open decisions go in the report, and merge/deploy/release always wait for a human; (5) scoped lessons to the directory they belong to via nested instruction files or `.claude/rules/`; (6) added `pnpm eval:report` for per-skill score and delta, a CI-fixer template whose agent job holds no write credentials, and metrics for agent-assisted changes. They verified with 155/155 tests, per-commit `pnpm test` and `pnpm check` via `git rebase --exec`, a clean `pnpm build`, `claude plugin validate --strict`, runtime acceptance scenarios 30–32 under `claude --plugin-dir`, and 7 eval cases at 3 runs per arm with a haiku judge.

How it went

Shipped as release 0.6.0. All 155 tests pass, every commit passes test and check independently, the build leaves the tree clean, and plugin validation plus the release check pass. Runtime scenarios confirmed the Stop hook blocked a review that skipped a reviewer, the push hook blocked a default-branch push with the option on and allowed it with the option off, and the unattended contract held. Evals showed large deltas: review-readonly-agents 1.00 vs 0.40, commit/bugfix unattended 1.00 vs 0.33, feature and ship unattended 1.00 vs 0.00, lessons-path-scoped 1.00 vs 0.50. Without the plugin, every unattended case committed to or pushed main. The first eval run also caught a real bug: graders used `(?i)` which JavaScript RegExp rejects, fixed to `flags: i` with a test that compiles every grader regex.

Worth knowing

Hooks and agents are Claude Code only; Codex, Cursor, Gemini CLI, and OpenCode keep the skill text, which still merely instructs read-only subagents and no default-branch pushes. `guard-push` is a guard, not a sandbox — it does not parse quoted separators, `bash -c`, subshells, variables, glob refspecs, or a bare `git push` under `push.default=upstream`. `guard-review` sends a turn back at most once via `stop_hook_active`, so it cannot loop but also cannot re-enforce. The full 47-case suite was not run on this branch, the `lessons-path-scoped-claude-md` case passes without the plugin so it only catches regressions, and `guard-push` has tests but no eval case. Rollback is reverting the merge or pinning 0.5.3; hook state lives in `${CLAUDE_PLUGIN_DATA}` or the temp directory.

Wiring

The plugin lives under `plugins/workflows` and is loaded at runtime with `claude --plugin-dir plugins/workflows` on Claude Code 2.1.300. Reviewer agents are declared as plugin agents with a restricted `tools` list (Read, Grep, Glob) and a `skills:` preload; the review skill dispatches all three by name (`workflows:*-reviewer`). Hooks are registered in the plugin's hook config: a `Stop` hook pointing at `guard-review.mjs` and a `PreToolUse` hook pointing at `guard-push.mjs`. `guard-review` writes one marker file per reviewer under `${CLAUDE_PLUGIN_DATA}` (or the temp directory) and reads `stop_hook_active` to cap itself at one bounce. `guard-push` inspects the Bash command string, resolves `cd` and line continuations, compares the target ref against the remote default branch, and honors the `guard_default_branch` plugin option. Skill text carries the same rules for other harnesses. Verification runs through `pnpm test`, `pnpm check`, `pnpm build`, `pnpm eval:report`, `node scripts/release.mjs check main`, `claude plugin validate plugins/workflows --strict`, and `claude plugin eval` with 3 runs per arm and a haiku judge; runtime acceptance scenarios live in `docs/runtime-acceptance.md`.

Try it yourself

In this repo, harden my Claude Code plugin so its key guarantees are enforced rather than merely described. Do this in order: (1) find the review skill and split it into three read-only reviewer agents (quality, spec, standards) that declare only Read, Grep, and Glob and preload their skill with `skills:`; add a validator that rejects any other tool. (2) Add a `Stop` hook that detects when a turn ran the review skill without dispatching all three reviewers and sends Claude back exactly once, using one marker file per reviewer so parallel dispatch does not lose updates, and failing open on error. (3) Add a `PreToolUse` hook that blocks `git push` to the remote default branch, `--all`, and `--mirror`, following `cd` and line continuations, gated behind a plugin option. (4) Write an unattended-run contract into the feature, bugfix, commit, and ship skills: the prompt is the only authorization, open decisions go in the report, and merge/deploy/release always wait for a human. (5) Scope lessons to nested instruction files or `.claude/rules/`. Then add eval cases for each change, run 3 runs per arm with and without the plugin, and report score and delta per skill. Keep the skill text as the fallback for non-Claude-Code harnesses, and tell me explicitly which guarantees are enforced versus merely instructed.