docs: senior-dev's DeepSWE results, three campaigns as per-task tables

[ZeroPoint95]

Adds docs/benchmarks/deepswe/ — the committed record of how senior-dev, the coding harness vendored into this product, does on the DeepSWE v1.1 corpus. Docs only: 18 files, 308 KB, no code touched.

Three campaigns, one directory each:

Directory	What	Headline
2026-09-12-harness-comparison-v4-flash	Ten harnesses, DeepSeek V4 Flash, 113 tasks each	senior-dev 1st — 62/113
2026-09-15-senior-dev-278a076-v41-flash	senior-dev 278a076, DeepSeek V4.1 Flash	88/113 — 77.9%
2026-09-22-senior-dev-f3b9716-kimi-k3	senior-dev f3b9716, Kimi K3	78/113 — 69.0%

Why docs/benchmarks/

It matches what's already there: docs/benchmarks/ is "classified by what is being measured", performance/ covers the CLI's footprint, and this covers task-solving. The index gets one new entry beside it.

bench/deepswe/ stays what it is — the protocol — and its results/ is gitignored, so it isn't a home for a committed record.

Two things the index makes explicit

1. These did not come from bench/deepswe/. That rig measures this repository's codeaf binary. These campaigns ran on senior-dev's own rig against the official DeepSWE verifiers at 0b9fabb. Each run's README names its manifest, preregistration and reproduction command.

Denominators differ too: bench/deepswe/ works from the 117-task v1.1 checkout, these campaigns from the 113 scored tasks. A 113 here and a 117 there are not the same set.

2. Two codeafs. The artifacts behind these runs came from a binary that used to be called codeaf and is not this repository's codeaf — the other Agent-Field project of that name renamed away, to swe-pro and then to senior-dev. Values are rewritten to the current name throughout.

What a row carries

reward · f2p · p2p · the four test counts · seconds · started · finished · cost_usd · model_calls · exit · model · harness_version · attempt_id, plus each task's band, difficulty_rank, language and dev_set. Rows are ordered by difficulty rank, so the same line is the same task in every file and two harnesses diff directly. The ten comparison tables share a column layout:

awk 'FNR>1 || NR==1' tasks/*.csv > all-arms.csv   # 1,130 rows

Caveats carried with the numbers, not buried

• Five tasks across four arms have an empty reward — no usable verifier outcome. They stay in the 113-task denominator and contribute zero; F2P and P2P macro-averages use that same basis.
• Six of the nine other harnesses recorded no per-attempt cost, so those columns are empty; their spend is arm-level only, in arms.csv.
• The Kimi K3 run resolves no difference against the V4.1 Flash run. The arms differ in model, binary and coder prompt inseparably — the re-anchor that would have isolated the model was never run. Unpaired intervals overlap; paired, exact McNemar two-sided p = 0.0895, which is the preregistered rule. Its README carries that and seven other registered limits.

Not included

The per-attempt artifacts — events.ndjson, run.log, traces, verifier stdout, patches — roughly 23 GB. They stay on the machines that produced them.

🤖 Generated with Claude Code

[ZeroPoint95]

Superseded by #1511, which landed the same DeepSWE results on dev. Closing; nothing here is needed.