Skip to content

docs(archdev-skill): own the sealed risk assessment flow for plan, task, and PR events - #28

Merged
rafael-archastro merged 1 commit into
mainfrom
docs/archdev-skill-risk-assessments
Sep 25, 2026
Merged

rafael-archastro merged 1 commit into
mainfrom
docs/archdev-skill-risk-assessments

Conversation

@rafael-archastro

@rafael-archastro rafael-archastro commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Review on ArchCode

Rebased onto main on 2026-09-25 (#27 is merged and dropped from this branch; the compact five-step Report bullet that landed meanwhile is merged with this section).

Problem and author intent

The CLI requires a sealed risk assessment on every structured plan.*, task.*, and pr.* activity event, graded under the versioned risk definitions in src/ts/cli-foundations/src/risk/ (Rob's lane) and posted with archdev log post --assessment. The agent is the producer: it collects evidence, grades uncertainty and consequence, and the CLI validates, derives the combined grade, and seals. The activity-risk design (docs/plans/2026-09-22-activity-risk-assessments.md in firstlanding) lists the coding-agent skill as the piece that teaches this flow, including the honesty fields (evidence[].kind, exposure, missingInputs).

The skill carries the flow as one compact bullet under Report. Following it end to end on firstlanding PRs took wrong turns the bullet does not prevent: the fact payload is the extractor's value object, not the envelope extract run writes; the default deterministic runner produces no value for plan/task/PR lifecycle extractors; validating through extract run --runner file: needs the sealed result embedded as risk; an unassessed component must omit grade; and input.subject gets typed from memory instead of taken from a collected packet. Rob's review of firstlanding#14994 asked for the last one specifically: author seals should rest on the evidence extract context pr.risk collects, not on a hand-written packet, and the seal should say which it was.

What changed

  • archdev/references/monitor.md, Report: the compact bullet keeps its steps (now six, adding the fact-payload step) and points at the new section. The "mitigate, then recompute" step from docs(archdev-skill): mitigate risks found while computing risk, then recompute #26 stays in both places.
  • archdev/references/monitor.md, new "Risk assessments" section: what the two components mean (a faithful compression of the shared definition's grade meanings and consequence anchors); the six steps (brief once per definition per session, fact payload from extract context, judgment, extract finalize, mitigate and recompute, log post --project --assessment); the optional extract run --runner file: check with risk embedded; when to assess (every event, fresh per PR head; the CLI checks subject identity only, not the head); and what is not the agent's to run (session.risk packets, extract run pr.risk as an independent second opinion that cannot be attached to a post).
  • Judgment step for PRs: run extract context pr.risk <owner/repo>#<num> --json and build input from the collected packet (its subject with base, head, reportedBase and diffIdentity; its evidence items with their ids and kinds; its missingInputs), trimmed to the 64 KB attachment cap, and record in exposure.ambientContext whether the input is the trimmed collected packet or hand-built. Fallback when the collector cannot run: base, head and diff digest from the stored annotation row via extract show pr.review-annotations.
  • Dropped the CLI 0.45.7 vs 0.45.8 bare-number notes: the skill requires 0.46.0 now.
  • archdev/SKILL.md: Monitor beat 3 names the sealed-assessment requirement and points at the section.

Not in this PR: the step that publishes risk.code-region seals for focus ranges (extract finalize --publish <pull>, firstlanding#15315). It waits for a CLI release that carries the flag, since the skill only documents commands a released CLI has.

Testing

Docs only. Every command, flag, file name, and field was run against CLI 0.46.2 on 2026-09-25 while sealing firstlanding#15315 and #15317: extract brief risk.pr --out (files DEFINITION.md, INPUT_SCHEMA.json, OUTPUT_SCHEMA.json, example.json, brief.json), extract context pr.lifecycle, extract context pr.risk <owner/repo>#<num> --json (169 KB collected input, 58 evidence items, subject with base/head/reportedBase/diffIdentity), extract finalize (combined low), log post --project --event pr.created --payload-file <value> --assessment sealed/result.json. The earlier review pass against CLI source (nine findings) still stands for the unchanged text.

Risk

Low. Instruction text only.

@rafael-archastro
rafael-archastro force-pushed the docs/archdev-skill-annotate-every-push branch from 783a740 to 24e8617 Compare September 23, 2026 20:58
@rafael-archastro

Copy link
Copy Markdown
Contributor Author

Holding this PR until the integration plan is concrete: ArchAstro/firstlanding#14994 (https://github.com/ArchAstro/firstlanding/issues/14994).

Short version of the plan: today the sealed risk.pr assessment only rides the room post, while the PR Overview reads the annotation row alone, so the two never meet. The plan stores sealed results as a per-head custom object written by archdev log post --event pr.*, has the Overview verdict prefer the sealed combined grade with uncertainty/consequence shown under the AI summary, ties focus items to risk.code-region rows by range overlap, swaps the minimap adapter, has archdev publish and Factory produce the rows, and last aligns the annotation producer's grades with the shared definition (needs Rob). Once tasks 1 and 2 land, this section's steps change (the seal becomes visible on the PR and code-region assessments join the flow), so the wording here should follow the implementation rather than lead it. Not rebased on purpose.

Base automatically changed from docs/archdev-skill-annotate-every-push to main September 23, 2026 21:51
…sk, and PR events

The CLI requires a sealed risk assessment, graded under the pinned risk
definitions in cli-foundations, on every plan/task/pr activity event.
The skill described it in one bullet that left out what an agent trips
on: the fact payload is the extractor's value object, the default
deterministic runner yields nothing for lifecycle extractors, extract
run needs the seal embedded as risk, a bare PR number on CLI 0.45.7
resolves the subject as local, and an unassessed component omits grade.

monitor.md gains a 'Risk assessments' section: the two components with
the definition's grade meanings and consequence anchors, the five steps
(brief, fact payload, judgment with honest evidence kinds, finalize,
log post --assessment), the optional extract run check, when to assess,
and what is not the agent's to run. SKILL.md beat 3 points at it.
@rafael-archastro
rafael-archastro force-pushed the docs/archdev-skill-risk-assessments branch from ba99b5e to f298379 Compare September 25, 2026 15:57
@rafael-archastro
rafael-archastro merged commit 1d19038 into main Sep 25, 2026
5 checks passed
@rafael-archastro
rafael-archastro deleted the docs/archdev-skill-risk-assessments branch September 25, 2026 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant