Self-learning skill layer for Claude Code that distills, merges, and prunes skills from real sessions

What they did

They installed AutoHarness as a Claude Code plugin from the tigerless-labs marketplace, then let it run in the background while doing normal work. Each session that did enough work was distilled into a skill under .claude/skills/, with a reflector comparing new episodes against existing skills so same-scenario ones fold into a single consolidated skill rather than piling up near-duplicates. Every session opened with a grouped index of the skills AutoHarness itself wrote, so recall didn't depend on the host surfacing them. When they wanted a lesson kept immediately, they ran /learn to push the current session through the same proposal-and-validation chain the background pass uses. Skills survived by being adhered to in later turns rather than by a held-out benchmark score, and each create/update logged its scenario and decision to a per-skill ledger. Anything they wrote or installed themselves was left untouched — the plugin only manages skills it generated.

How it went

The layer stayed clean on its own: duplicates folded instead of stacking, unused skills got pruned, and the grouped index made recall reliable without changing the host's native behavior. Setup was zero-config after install and reload, and the ledger left raw material for building a benchmark from real usage later.

Worth knowing

Requires Python 3.11+ as the python3 on PATH; an older interpreter earlier on PATH (macOS ships 3.9.6 at /usr/bin/python3) silently disables every hook for the session, with only a stderr warning. Hooks and the MCP server won't fire without it. Cadence and lifecycle thresholds are tunable but not zero-thought. Skill quality depends on sessions doing enough real work to distill.

Wiring

Install via the Claude Code plugin marketplace (two slash commands) followed by /reload-plugins or a restart. AutoHarness runs entirely as dependency-free Python 3.11+; its hooks resolve bare python3 and its MCP server is registered in .claude-plugin/plugin.json. Hooks watch sessions and trigger a background distillation pass once a session has done enough work; /learn is the manual entry point into the same proposal-and-validation chain. The reflector compares each new episode against existing skills and folds same-scenario ones, recording which skill absorbed which. A session-start hook injects a grouped index of plugin-authored skills on top of the host's native recall. All writes are scoped to skills the plugin generated, with per-skill ledgers logging scenario and decisions.

Try it yourself

Install AutoHarness into this repo as a Claude Code plugin: run `/plugin marketplace add tigerless-labs/autoharness` then `/plugin install autoharness@autoharness`, then `/reload-plugins`. First verify `python3 --version` is 3.11 or newer on PATH and fix it if not, since hooks silently disable otherwise. Then work through a real task in this repo, run `/learn` to distill the lesson, and show me the resulting skill file under `.claude/skills/` plus its ledger entry. Confirm that no pre-existing skills I wrote were modified.