docs(archdev-skill): own the sealed risk assessment flow for plan, task, and PR events - #28
Conversation
783a740 to
24e8617
Compare
|
Holding this PR until the integration plan is concrete: ArchAstro/firstlanding#14994 (https://github.com/ArchAstro/firstlanding/issues/14994). Short version of the plan: today the sealed risk.pr assessment only rides the room post, while the PR Overview reads the annotation row alone, so the two never meet. The plan stores sealed results as a per-head custom object written by |
…sk, and PR events The CLI requires a sealed risk assessment, graded under the pinned risk definitions in cli-foundations, on every plan/task/pr activity event. The skill described it in one bullet that left out what an agent trips on: the fact payload is the extractor's value object, the default deterministic runner yields nothing for lifecycle extractors, extract run needs the seal embedded as risk, a bare PR number on CLI 0.45.7 resolves the subject as local, and an unassessed component omits grade. monitor.md gains a 'Risk assessments' section: the two components with the definition's grade meanings and consequence anchors, the five steps (brief, fact payload, judgment with honest evidence kinds, finalize, log post --assessment), the optional extract run check, when to assess, and what is not the agent's to run. SKILL.md beat 3 points at it.
ba99b5e to
f298379
Compare
Review on ArchCode
Rebased onto
mainon 2026-09-25 (#27 is merged and dropped from this branch; the compact five-step Report bullet that landed meanwhile is merged with this section).Problem and author intent
The CLI requires a sealed risk assessment on every structured
plan.*,task.*, andpr.*activity event, graded under the versioned risk definitions insrc/ts/cli-foundations/src/risk/(Rob's lane) and posted witharchdev log post --assessment. The agent is the producer: it collects evidence, grades uncertainty and consequence, and the CLI validates, derives the combined grade, and seals. The activity-risk design (docs/plans/2026-09-22-activity-risk-assessments.mdin firstlanding) lists the coding-agent skill as the piece that teaches this flow, including the honesty fields (evidence[].kind,exposure,missingInputs).The skill carries the flow as one compact bullet under Report. Following it end to end on firstlanding PRs took wrong turns the bullet does not prevent: the fact payload is the extractor's value object, not the envelope
extract runwrites; the default deterministic runner produces no value for plan/task/PR lifecycle extractors; validating throughextract run --runner file:needs the sealed result embedded asrisk; an unassessed component must omitgrade; andinput.subjectgets typed from memory instead of taken from a collected packet. Rob's review of firstlanding#14994 asked for the last one specifically: author seals should rest on the evidenceextract context pr.riskcollects, not on a hand-written packet, and the seal should say which it was.What changed
archdev/references/monitor.md, Report: the compact bullet keeps its steps (now six, adding the fact-payload step) and points at the new section. The "mitigate, then recompute" step from docs(archdev-skill): mitigate risks found while computing risk, then recompute #26 stays in both places.archdev/references/monitor.md, new "Risk assessments" section: what the two components mean (a faithful compression of the shared definition's grade meanings and consequence anchors); the six steps (brief once per definition per session, fact payload fromextract context, judgment,extract finalize, mitigate and recompute,log post --project --assessment); the optionalextract run --runner file:check withriskembedded; when to assess (every event, fresh per PR head; the CLI checks subject identity only, not the head); and what is not the agent's to run (session.riskpackets,extract run pr.riskas an independent second opinion that cannot be attached to a post).extract context pr.risk <owner/repo>#<num> --jsonand buildinputfrom the collected packet (itssubjectwith base, head, reportedBase and diffIdentity; its evidence items with their ids and kinds; itsmissingInputs), trimmed to the 64 KB attachment cap, and record inexposure.ambientContextwhether the input is the trimmed collected packet or hand-built. Fallback when the collector cannot run: base, head and diff digest from the stored annotation row viaextract show pr.review-annotations.archdev/SKILL.md: Monitor beat 3 names the sealed-assessment requirement and points at the section.Not in this PR: the step that publishes
risk.code-regionseals for focus ranges (extract finalize --publish <pull>, firstlanding#15315). It waits for a CLI release that carries the flag, since the skill only documents commands a released CLI has.Testing
Docs only. Every command, flag, file name, and field was run against CLI 0.46.2 on 2026-09-25 while sealing firstlanding#15315 and #15317:
extract brief risk.pr --out(files DEFINITION.md, INPUT_SCHEMA.json, OUTPUT_SCHEMA.json, example.json, brief.json),extract context pr.lifecycle,extract context pr.risk <owner/repo>#<num> --json(169 KB collectedinput, 58 evidence items, subject with base/head/reportedBase/diffIdentity),extract finalize(combined low),log post --project --event pr.created --payload-file <value> --assessment sealed/result.json. The earlier review pass against CLI source (nine findings) still stands for the unchanged text.Risk
Low. Instruction text only.