Repository navigation
feat(docs): generate plugin docs + badges from the e2e run - #53
Merged
Merged
Conversation
DEC-077 — docs from e2e. Every green run already captures a screenshot per BDD scenario, and each scenario title is a plain-English behaviour guarantee. Both were being discarded: screenshots expired as CI artifacts, feature files were read only by developers. Fleet state today: chat-comments has ZERO images in its README; the others have hand-captured ones that go stale silently. `make docs` joins them into docs/BEHAVIOUR.md + docs/screenshots/. Sourced from a PASSING run, so a screenshot cannot show a state the plugin no longer produces, and a renamed/deleted scenario drops out automatically. Captions are scenario titles verbatim — no invented prose; a scenario with no shot says so. The matching is the subtle part: Playwright TRUNCATES long titles in its result dir names and inserts a hash, so an exact match is impossible. Matching is on the longest shared run of trailing tokens, minimum two — one shared token is meaningless where every scenario says "comment". My first version missed this and produced a doc where all 12 scenarios reported "no screenshot": a plausible empty result rather than an obvious failure. test_docs_from_e2e.py pins it with real artifact names from run 30866557253. DEC-078 — badges. The e2e suite (nested real A0, real browser, seam-off red-proof that fails the build if scenarios could pass with the plugin absent) was invisible to readers: badge coverage was 1, 0, 0, 1 across the four plugins. `make badges` splices a standard block under the H1, idempotently, between BADGES markers. Every badge is click-verifiable — workflow status, release tag, licence all GitHub-generated; the one static label links to the actual feature directory. No invented coverage numbers. Verified: generator run against the real chat-comments artifact and feature file illustrates 11/12 scenarios (the 12th is the devkit lifecycle scenario, correctly not in the plugin feature file); badge splice is idempotent across two runs.
omar-nahhas
force-pushed
the
feat/docs-from-e2e
branch
from
August 4, 2026 22:30
8a37dff to
3ba95c8
Compare
Merged
omar-nahhas
added a commit
that referenced
this pull request
Aug 4, 2026
v2.1.5 was tagged BEFORE the docs/badges tooling merged (#53), so consumers pinning v2.1.5 lack docs-from-e2e.py / readme-badges.py and `make docs` cannot run for them. This tag delivers DEC-077/078 plus the DEC-079 devkit-sync fix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two new devkit targets, so this is built once in the devkit rather than hand-made four times.
make docs— DEC-077Every green e2e already captures one screenshot per BDD scenario, and each scenario title is a plain-English statement of a behaviour the plugin guarantees. Both were being thrown away: screenshots expired as CI artifacts, feature files were read only by developers.
Fleet state today: chat-comments has zero images in its README; the other three have hand-captured ones that go stale with no signal.
make docsjoins them intodocs/BEHAVIOUR.md+docs/screenshots/. Because the source is a passing run, a screenshot cannot depict a state the plugin no longer produces, and a renamed or deleted scenario drops out of the docs automatically. Captions are scenario titles verbatim — the generator invents no prose, and a scenario with no captured screenshot says so rather than being silently omitted.The subtle part. Playwright truncates long scenario titles in its result directory names and inserts a hash:
so matching back to the feature file cannot be an exact comparison. It matches on the longest shared run of trailing tokens (the tail survives truncation), requiring at least two — one shared token is meaningless in a file where every scenario says "comment". My first version missed this and produced a document where all twelve scenarios reported "no screenshot" — a plausible-looking empty result rather than an obvious failure.
test_docs_from_e2e.pypins this with real artifact names.Verified against the real chat-comments artifact and feature file: 11/12 scenarios illustrated (the 12th is the devkit's own lifecycle scenario, correctly absent from the plugin's feature file).
make badges— DEC-078The e2e suite is the strongest quality claim these plugins have — nested real A0, real browser, and a seam-off red-proof that fails the build if the scenarios could pass with the plugin not installed. None of it was visible to a reader:
make badgessplices a standard block under the H1, idempotently, betweenBADGES:START/BADGES:ENDmarkers (verified: applying twice yields one block).Constraint on the content: every badge must be independently checkable by clicking it. Workflow status, release tag and licence are all GitHub-generated. The single static label ("BDD e2e") links to the actual feature directory. No invented coverage percentages.
Follow-up, not in this PR
omar-nahhas/a0-plugins/plugins/has no entry for them. The schema supports up to 5screenshotsURLs plus a ≤20 KB square thumbnail, so the generated screenshots feed it directly once the entries exist.