feat(docs): generate plugin docs + badges from the e2e run

SPEC.md

@@ -1128,6 +1128,36 @@ while half the fleet was still broken. Collector → `e2e/harness/_artifacts.sh`
 harnesses; prune → `e2e/ci/prune-artifacts.sh`, invoked by both workflows. A reusable workflow can
 call the script from the consumer's vendored devkit path, so there is still exactly one definition.
 
+**DEC-077 — Plugin documentation is generated from the plugin's own e2e run.**
+Every green e2e already captures one screenshot per BDD scenario, and each scenario title is a
+plain-English statement of a behaviour the plugin guarantees. Those two artefacts together *are* the
+documentation, but they were being thrown away: the screenshots expired as CI artifacts and the
+feature files were read only by developers. The result was a fleet where one plugin had **zero**
+images in its README and the rest had hand-captured ones that quietly go stale.
+`make docs` joins them into `docs/BEHAVIOUR.md` + `docs/screenshots/`. Because the source is a
+*passing* run, a screenshot can never depict a state the plugin no longer produces, and a deleted or
+renamed scenario drops out of the docs automatically. Captions are the scenario titles verbatim — the
+generator invents no prose, and a scenario with no captured screenshot says so rather than being
+silently omitted.
+Implementation note worth keeping: Playwright **truncates** long scenario titles in its result
+directory names and inserts a hash, so matching a screenshot back to its scenario cannot be an exact
+comparison. It matches on the longest shared run of *trailing* tokens (the tail survives truncation)
+and requires at least two, since a single shared token is meaningless in a feature file where every
+scenario mentions the same nouns. The first implementation missed this and produced a document in
+which all twelve scenarios reported "no screenshot" — a plausible-looking empty result rather than an
+obvious failure, which is why `test_docs_from_e2e.py` pins this behaviour with real artifact names.
+
+**DEC-078 — Every plugin README carries the same, verifiable badge block.**
+The e2e suite is the strongest quality claim these plugins have — nested real A0, real browser, and a
+seam-off red-proof that fails the build if the scenarios could pass with the plugin *not* installed.
+None of it was visible to a reader: badge coverage across the fleet was 1, 0, 0, 1 (and 5 on the
+devkit itself). `make badges` splices a standard block under the H1, idempotently, between
+`BADGES:START`/`BADGES:END` markers.
+Constraint on what may appear there: every badge must be independently checkable by clicking it.
+Workflow status comes from the live Actions badge, the release from the tag list, the licence from
+the repo. The one non-GitHub badge ("BDD e2e") is a static label whose link lands on the actual
+feature directory. No invented coverage percentages.
+
 **DEC-079 — devkit-sync is on-demand, and never deletes its own branch.**
 Two corrections to the sync mechanism, one of policy and one of implementation.
 
@@ -1148,7 +1178,7 @@ e2e had passed and the result was discarded and re-queued for nothing. The same
 alone is sufficient and non-destructive: an open PR follows its head ref and picks up the new commit.
 
 > **Numbering note:** DEC-070–072 were introduced as code comments only and were never written up
-> here. This section documents 073–076; backfilling 070–072 is tracked separately.
+> here. This section documents 073–079; backfilling 070–072 is tracked separately.
 
 ---
 

e2e/Makefile.devkit

@@ -30,7 +30,7 @@ PODMAN_E2E_FLAGS := --rm --device /dev/fuse --security-opt label=disable \
 # Riding along here means the standard one-line include gives them for free.
 -include $(DEVKIT_DIR)/e2e/make/bdd.mk
 
-.PHONY: build package up down e2e conformance update-devkit link-workflows link-gemini link-skills link-devkit help-devkit
+.PHONY: build package up down e2e conformance update-devkit link-workflows link-gemini link-skills link-devkit help-devkit docs badges
 
 # Which devkit ref to bump to. Empty (default) = the latest release tag within your
 # MAJOR channel (.devkit.yml `devkit_major`, default 1) — a new MAJOR (e.g. v2, which
@@ -115,9 +115,24 @@ up: ## boot A0 nested for manual/agent exploration (inside the devcontainer)
 	  -v $(CURDIR):/workspace:ro -e A0_IMAGE=$(A0_IMAGE) \
 	  $(DEVKIT_IMAGE) -lc 'bash /workspace/$(DEVKIT_DIR)/e2e/harness/a0-up.sh && echo "A0 on :80 — exec in to explore" && sleep infinity'
 
+docs: ## regenerate docs/BEHAVIOUR.md + docs/screenshots/ from the last e2e run (DEC-077)
+	@# ARTIFACTS defaults to the local run's output; in CI the workflow points it
+	@# at the collected artifact dir. Never invents screenshots — a scenario with
+	@# no captured shot is reported as such in the generated doc.
+	python3 $(DEVKIT_DIR)/e2e/scripts/docs-from-e2e.py \
+	  --artifacts $(or $(ARTIFACTS),artifacts) \
+	  --features tests/e2e/features \
+	  --plugin $(PLUGIN_NAME) \
+	  --out docs
+
 conformance: ## assert this repo exposes the frozen target set + allowed assets
 	@for t in build package up e2e; do \
 	  $(MAKE) -n $$t >/dev/null 2>&1 || { echo "missing target: $$t"; exit 1; }; \
 	done
 	@test -f tests/e2e/hooks/verify-installed || echo "note: no tests/e2e/hooks/verify-installed (optional)"
 	@echo "conformance OK"
+
+badges: ## refresh the standard badge block in README.md (DEC-078)
+	python3 $(DEVKIT_DIR)/e2e/scripts/readme-badges.py \
+	  --repo $(shell git config --get remote.origin.url | sed -E 's#.*github.com[:/]([^/]+/[^/]+?)(\.git)?$$#\1#') \
+	  --plugin $(PLUGIN_NAME) --apply README.md

e2e/scripts/docs-from-e2e.py

@@ -0,0 +1,174 @@
+#!/usr/bin/env python3
+"""
+Build a plugin's visual behaviour documentation from its own e2e run (DEC-077).
+
+WHY THIS EXISTS
+    Every green e2e already captures a screenshot per BDD scenario, and each
+    scenario is a plain-English statement of a behaviour the plugin guarantees.
+    Those two facts together ARE the documentation — but the screenshots were
+    only ever uploaded as a CI artifact that expires, and the feature files were
+    only read by developers. So every repo either had no screenshots at all
+    (chat-comments) or hand-captured ones that silently go stale.
+
+    This joins them: scenario name -> its screenshot -> docs/BEHAVIOUR.md. The
+    docs are therefore regenerated from a passing test run, so a screenshot can
+    never show a state the plugin no longer produces. If a scenario is deleted
+    or renamed, its screenshot disappears from the docs on the next run.
+
+WHAT IT DOES NOT DO
+    It does not invent prose. The feature file's Scenario line is the caption,
+    verbatim. If that reads badly, fix the feature file — the same text is the
+    contract the test asserts.
+
+Usage:
+    docs-from-e2e.py --artifacts <dir> --features <dir> --plugin <name>
+                     --out docs [--title "Display Name"]
+"""
+from __future__ import annotations
+
+import argparse
+import re
+import shutil
+import sys
+from pathlib import Path
+
+# Playwright's per-scenario dir is flattened into the artifact filename by the
+# collector (_artifacts.sh), e.g.
+#   chat_comments-tests-e2e-features-10-comm-6bdad-an-be-commented-BEH-5-BEH-8--test-finished-1.png
+# The middle is a TRUNCATED, slugified scenario title with a hash — Playwright
+# shortens long titles, so an exact match back to the feature file is impossible.
+# Match on the longest shared token-suffix instead (see _match_scenario).
+SHOT_RE = re.compile(r"^(?P<plugin>.+?)-(?P<slug>.+?)--(?P<kind>[a-z-]+-\d+)\.png$")
+
+
+def _slug_tokens(text: str) -> list[str]:
+    """Lowercase alphanumeric tokens — the common ground between a feature-file
+    scenario title and Playwright's slugified, truncated directory name."""
+    return [t for t in re.split(r"[^a-z0-9]+", text.lower()) if t]
+
+
+def parse_features(features_dir: Path) -> list[dict]:
+    """Extract scenarios in file order. Keeps the tag comment (# BEH-5, BEH-8)
+    because it is the traceability link back to the spec."""
+    scenarios: list[dict] = []
+    for feature in sorted(features_dir.glob("*.feature")):
+        feature_title = ""
+        for raw in feature.read_text(encoding="utf-8").splitlines():
+            line = raw.strip()
+            if line.startswith("Feature:"):
+                feature_title = line.split(":", 1)[1].strip()
+            elif line.startswith("Scenario:"):
+                body = line.split(":", 1)[1]
+                title, _, comment = body.partition("#")
+                # Match on title AND the trailing "# BEH-5, BEH-8" comment:
+                # Playwright's directory name is built from the whole Scenario
+                # line, so the refs are part of the slug's surviving tail.
+                scenarios.append(
+                    {
+                        "feature": feature_title,
+                        "title": title.strip(),
+                        "refs": comment.strip(),
+                        "tokens": _slug_tokens(f"{title} {comment}"),
+                        "shot": None,
+                    }
+                )
+    return scenarios
+
+
+def _match_scenario(scenarios: list[dict], slug: str) -> dict | None:
+    """Playwright truncates the middle of long titles, so the artifact slug is a
+    prefix+hash+SUFFIX of the real title. The suffix survives intact, so score on
+    the longest matching run of trailing tokens and require at least two — one
+    shared token ("chat", "comment") is far too weak across a feature file where
+    every scenario mentions the same nouns."""
+    slug_tokens = _slug_tokens(slug)
+    best, best_score = None, 0
+    for sc in scenarios:
+        if sc["shot"] or not sc["tokens"]:
+            continue
+        score = 0
+        for a, b in zip(reversed(sc["tokens"]), reversed(slug_tokens)):
+            if a != b:
+                break
+            score += 1
+        if score > best_score:
+            best, best_score = sc, score
+    return best if best_score >= 2 else None
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--artifacts", required=True, type=Path)
+    ap.add_argument("--features", required=True, type=Path)
+    ap.add_argument("--plugin", required=True)
+    ap.add_argument("--out", required=True, type=Path)
+    ap.add_argument("--title", default=None)
+    args = ap.parse_args()
+
+    title = args.title or args.plugin.replace("_", " ").title()
+
+    if not args.features.is_dir():
+        print(f"::warning::no features dir at {args.features} — nothing to document")
+        return 0
+
+    scenarios = parse_features(args.features)
+    if not scenarios:
+        print(f"::warning::no scenarios found in {args.features}")
+        return 0
+
+    shots_dir = args.out / "screenshots"
+    shots_dir.mkdir(parents=True, exist_ok=True)
+
+    # Only the final screenshot of each scenario: it shows the asserted end state,
+    # which is exactly what the caption claims. Intermediate frames would show
+    # states the scenario does not guarantee.
+    matched = 0
+    for png in sorted(args.artifacts.rglob("*.png")):
+        m = SHOT_RE.match(png.name)
+        if not m or not m.group("kind").startswith("test-finished"):
+            continue
+        sc = _match_scenario(scenarios, m.group("slug"))
+        if sc is None:
+            continue
+        dest_name = re.sub(r"[^a-z0-9]+", "-", sc["title"].lower()).strip("-") + ".png"
+        shutil.copyfile(png, shots_dir / dest_name)
+        sc["shot"] = f"screenshots/{dest_name}"
+        matched += 1
+
+    lines = [
+        f"# {title} — behaviour",
+        "",
+        "<!-- GENERATED by the devkit (`make docs`) from this plugin's own e2e run.",
+        "     Do not edit by hand: every screenshot below is the asserted end state of a",
+        "     passing BDD scenario, so it cannot show behaviour the plugin no longer has.",
+        "     Captions are the scenario titles verbatim — reword them in tests/e2e/features/. -->",
+        "",
+        f"Each section is one scenario from `tests/e2e/features/`, with the screenshot the",
+        f"test captured at its final assertion. {matched} of {len(scenarios)} scenarios have one.",
+        "",
+    ]
+
+    current_feature = None
+    for sc in scenarios:
+        if sc["feature"] != current_feature:
+            current_feature = sc["feature"]
+            lines += [f"## {current_feature}", ""]
+        lines.append(f"### {sc['title']}")
+        if sc["refs"]:
+            lines.append(f"<sub>{sc['refs']}</sub>")
+        lines.append("")
+        if sc["shot"]:
+            lines.append(f"![{sc['title']}]({sc['shot']})")
+        else:
+            # Honest gap rather than a silently missing section.
+            lines.append("_No screenshot captured for this scenario in the last run._")
+        lines.append("")
+
+    out_file = args.out / "BEHAVIOUR.md"
+    out_file.write_text("\n".join(lines), encoding="utf-8")
+    print(f"[docs-from-e2e] {out_file}: {matched}/{len(scenarios)} scenarios illustrated")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

e2e/scripts/readme-badges.py

@@ -0,0 +1,83 @@
+#!/usr/bin/env python3
+"""
+Emit the standard plugin badge block, and splice it into README.md (DEC-078).
+
+WHY
+    The e2e suite is the strongest quality claim these plugins have — a nested
+    real A0, a real browser, a seam-off red-proof that fails the build if the
+    scenarios could pass without the plugin installed. None of that was visible
+    to anyone reading the README: badge coverage across the fleet was 1 badge,
+    0 badges, 0 badges, 1 badge and 5 (the devkit itself).
+
+WHAT IS AND ISN'T CLAIMED
+    Every badge here is backed by something a reader can click and verify:
+    workflow status comes from the live Actions badge, the licence from the repo,
+    the release from the tag list. The one badge that is not GitHub-generated
+    ("BDD e2e") is a static label pointing at the feature directory, so a reader
+    lands on the actual scenarios. No invented percentages, no "100% coverage".
+
+Usage:
+    readme-badges.py --repo owner/name --plugin chat_comments [--apply README.md]
+"""
+from __future__ import annotations
+
+import argparse
+import re
+import sys
+from pathlib import Path
+
+START = "<!-- BADGES:START (generated by the devkit — `make badges`) -->"
+END = "<!-- BADGES:END -->"
+
+
+def build(repo: str, workflow: str, features_path: str) -> str:
+    base = f"https://github.com/{repo}"
+    rows = [
+        f"[![plugin-e2e]({base}/actions/workflows/{workflow}/badge.svg?branch=main)]"
+        f"({base}/actions/workflows/{workflow})",
+        f"[![BDD scenarios](https://img.shields.io/badge/tests-BDD%20e2e-brightgreen)]"
+        f"({base}/tree/main/{features_path})",
+        f"[![Release](https://img.shields.io/github/v/tag/{repo}?label=release&sort=semver)]"
+        f"({base}/tags)",
+        f"[![License: Apache-2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)]"
+        f"({base}/blob/main/LICENSE)",
+        f"[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)]"
+        f"({base}/blob/main/CONTRIBUTING.md)",
+    ]
+    return "\n".join([START, "", " ".join(rows), "", END])
+
+
+def apply(readme: Path, block: str) -> str:
+    text = readme.read_text(encoding="utf-8")
+    if START in text and END in text:
+        return re.sub(
+            re.escape(START) + r".*?" + re.escape(END), block, text, flags=re.S
+        )
+    # First insert: place directly under the H1 so it is the first thing seen.
+    lines = text.splitlines()
+    for i, line in enumerate(lines):
+        if line.startswith("# "):
+            return "\n".join(lines[: i + 1] + ["", block] + lines[i + 1 :])
+    return block + "\n\n" + text
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--repo", required=True, help="owner/name")
+    ap.add_argument("--plugin", required=True)
+    ap.add_argument("--workflow", default="plugin-e2e.yml")
+    ap.add_argument("--features", default="tests/e2e/features")
+    ap.add_argument("--apply", type=Path, help="README.md to splice into")
+    args = ap.parse_args()
+
+    block = build(args.repo, args.workflow, args.features)
+    if args.apply:
+        args.apply.write_text(apply(args.apply, block), encoding="utf-8")
+        print(f"[readme-badges] spliced badge block into {args.apply}")
+    else:
+        print(block)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())