Skip to content

chore(bench): commit the canonical bench runs, clean the scratch - #166

Merged
log0u7 merged 1 commit into
mainfrom
feat/bench-clean
Nov 15, 2026
Merged

log0u7 merged 1 commit into
mainfrom
feat/bench-clean

Conversation

@log0u7

@log0u7 log0u7 commented Nov 15, 2026

Copy link
Copy Markdown
Owner

Why

experiments/* was wholesale-ignored in .gitignore (only .gitkeep and ctf_ingest/ un-ignored): no bench run could ever be committed. The MR9/MR10 raw results (run-20261111-mr9-10rep, run-20261113-mr10-*) were destroyed by a worktree remove --force because they lived on disk only. Results are data, not scratch.

What

  • Un-ignore experiments/final-bench-results/.
  • Commit the 12 canonical runs (20261103..20261108, 19 MB): each is cited in docs/explanation/benchmark.md, the CHANGELOG, or the published-article drafts (sleep 30-rep, fix20/fix36, mr8, model-only series).
  • Delete the uncited iterations: the 20261105 debug/rebench series, mr8-model-v4, sft-facts, mr6-v2-3ep, 17ch-e3, per-target scratch dirs, stray root results.json/md (1-rep probe).
  • experiments/final-bench-results/README.md: run table + convention (completed run = committed run, before the next one starts).
  • Bench protocol gains rule 5 (same lesson, in scripts/run_final_bench.sh header).

Verification

  • git check-ignore on run files: UNIGNORED.
  • No secrets in the committed results (grep api-key/Bearer/sk-).
  • Suite green on this branch (CI).

The bench results directory was 100% untracked: 22 run directories
plus per-target scratch lived on disk only. The MR9/MR10 raw results
(run-20261111-mr9-10rep, run-20261113-mr10-*) were destroyed by a
worktree remove --force because nothing was committed. Results are
data, not scratch.

- keep the 12 canonical runs (20261103..20261108): each is cited in
  docs/explanation/benchmark.md, the CHANGELOG, or the published
  article drafts (sleep 30-rep, fix20/fix36, mr8, model-only series);
- delete the uncited iterations: the 20261105 debug/rebench series,
  mr8-model-v4, sft-facts, mr6-v2-3ep, 17ch-e3, the per-target
  scratch dirs and the stray root results.json/md (1-rep probe);
- add experiments/final-bench-results/README.md: the run table plus
  the convention (completed run = committed run, before the next);
- bench protocol gains rule 5: a completed run is committed before
  the next one starts.
@log0u7
log0u7 merged commit 3ba285d into main Nov 15, 2026
8 checks passed
@log0u7
log0u7 deleted the feat/bench-clean branch November 15, 2026 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant