Skip to content

fix: enforce RandomX thread ceilings in supported XMRig profiles - #601

Open
pithead-developer[bot] wants to merge 5 commits into
developfrom
agent/588-randomx-thread-profile
Open

pithead-developer[bot] wants to merge 5 commits into
developfrom
agent/588-randomx-thread-profile

Conversation

@pithead-developer

@pithead-developer pithead-developer Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

What's the problem

Closes #588

RigForge writes RandomX thread counts as integers that XMRig ignores, so a one-thread cap can mine on every core.

How I fixed it

Regressions for supported profiles plus a hardware phase that reads XMRig's running CPU backend, and now the product fix (3bbe717). cpu.rx is emitted only in forms XMRig 6.26 honours: omitted for auto, a {mode, threads, max_threads_cnt, intensity, affinity} profile object when a count is requested (setup cap and every tune candidate/overlay), or the observed auto array. A bounded one-shot probe run (sink pool, HTTP/DAEMON/MSR off) observes the auto profile XMRig picks on the box; the setup cap and each tune candidate are clamped to it, so per the operator ruling auto 8 / tuned 12 / cap 16 runs 8 and a cap can never raise thread count. Every reader/writer is normalized to number-or-object: HugePages sizing (RX_SETUP_THREADS), the tuner overlay, the tune summary, and the fake xmrig in the black-box suite. Helpers live in util/rx-profile.sh, sourced beside the script itself because SCRIPT_DIR is the worker home under RIGFORGE_HOME, not the script tree.

Here is the proof

Ponytail review: reuse the existing apply, reservation and restoration paths; keep a single targeted hardware phase and a small hardware-free regression fixture.

At test-only commit 5a2f3ab86d66c7574aab491e4001985cb6608d0b, timeout 60 python3 tests/test-thread-profile.py fails as expected against unchanged product code. The cap test rejects the scalar profile. Local ShellCheck of the hardware scripts, topology lint and the required file-budget freeze gate pass.

At harness head 6cc28356c9f8d73240b13ea6d079b32adc76dc27, the three failed-snapshot/restoration safety cases pass with timeout 10 python3 tests/test-thread-profile.py ThreadProfile.test_failed_hardware_snapshot_preserves_original ThreadProfile.test_failed_tuning_restore_does_not_apply_test_override ThreadProfile.test_snapshot_cleanup_failure_does_not_fail_restoration. The security reviewer is clean after corrections to bearer transport and failure restoration. ShellCheck, shfmt, Markdown lint, topology lint and the required file-budget freeze gate pass. GitHub shell/YAML/Markdown lint, secret scan, workflow audit, stock-home setup and Docker e2e passed on that harness head. Its Test suite and Coverage checks failed, including the expected product regressions and an isolated-fixture cleanup dependency corrected below.

At current head 0f97225a23416ea4d1127ac5794e174d602d0814, cleanup calls the new restoration helper only while its snapshot is active. This repairs the isolated fixtures without skipping restoration after a real mutation. Targeted verifier and security reviews of this correction are clean; this is not final review of the unfinished product fix.

  • timeout 180 bash tests/test-refresh-window-control.sh — PASS.
  • timeout 180 bash tests/test-control-replay.sh — PASS.
  • timeout 120 bash ../gate-unit-check.sh — PASS, 150 assertions; this scratch wrapper sources tests/test-e2e-gate-fail-closed.sh with the suite's assertion functions and a disposable sandbox.
  • The three targeted Python snapshot/restoration regressions above — PASS again after the correction.
  • shellcheck --severity=warning tests/e2e-pithead.sh tests/e2e-thread-profile.sh, shfmt -i 4 -d tests/e2e-pithead.sh tests/e2e-thread-profile.sh, FILE_BUDGET_REQUIRE_BASE=1 bash scripts/lint-file-budget.sh and git diff --check — PASS.

Current-head GitHub checks were queued/running when read after push. No new hardware job was submitted because restoration remains unresolved.

Status at head 3bbe717 (product fix)

The product fix above lands on top of the test-first commits; the nine regressions the RETURN review listed as the expected red state are green unmodified (python3 tests/test-thread-profile.py: 12/12, including the auto-selection fake: auto 8 / tuned 12 / cap 16 runs 8). Full dependency-free suite at this head: make test-suite — 2428 passed, 0 failed (tests/run.sh 2428/0 plus contract-credentials, access-token-readiness, docs-clone-pin, tool-pin-watch). shellcheck --severity=warning, shfmt, topology lint and the file-budget freeze gate all pass — rigforge.sh stays at its 6307-line ceiling by sourcing util/rx-profile.sh instead of growing the file.

The external proof still waits on hardware, not on code: tests/e2e-thread-profile.sh asserts the thread count XMRig itself reports through /2/backends, and that phase only runs on bench-ci hardware. That external proof stays pending on bench-ci#2248; the job block below is the pre-fix reproduction, kept as evidence of the original defect. Once bench-ci admits a run at this head, the thread-profile phase is the closing artifact.

The retained negative-control log and harness artifact reproduce the original cap defect. The overall job is an error because restoration was not verified; it is not a passing job. Further hardware submissions are stopped behind bench-ci#2248. The bench-ci owner must reconcile the original reservation/recovery hold and establish supported worker restoration before a new proof is admitted. The product fix and the green dependency-free suite have since landed (see Status at head 3bbe717); only the hardware proof on bench-ci remains.

bench-ci job 3038@5a2f3ab86d66c7574aab491e4001985cb6608d0b · rigforge · tier4-e2e

  • options: {"phases": ["thread-profile"]}
  • status: completed · conclusion: error · exit 1 · 2m 06s
  • baseline verified: no · rig restored: no
  • node guard: not recorded · no comparison recorded
  • steps: checkout success · reserve success · preflight success · run failure · collect success · restore failure · cleanup success
  • rows: 1 pass · 1 fail · 0 skip
  • error: baseline verification failed: rig not restored: : taken between the probe and the hold by runner-recovery-hold (bench-ci d90fb3ce2dbb job 3038)
  • failed rows:
    • [thread-profile] cap below auto-selection: XMRig runs 48 thread(s), expected 1 (no issue linked)
      Generated by bench-ci evidence 3038@5a2f3ab86d66c7574aab491e4001985cb6608d0b; re-run it to check.

@pithead-developer pithead-developer Bot added this to the v1.18.1 milestone Oct 27, 2026
@pithead-developer pithead-developer Bot added stage/bench-testing its issue waits on a queued or running tier-4 bench job stage/in-progress and removed stage/in-progress stage/bench-testing its issue waits on a queued or running tier-4 bench job labels Oct 27, 2026
@pithead-developer

Copy link
Copy Markdown
Contributor Author

Blocked on bench-ci#2248. Job 3038@5a2f3ab86d66c7574aab491e4001985cb6608d0b reproduced the original cap defect in the retained running-thread assertion, then ended with an error because worker restoration was unverified. The supported client instructions prohibit another submission in that state.

The test harness, cap/auto/tuning regressions and restoration safety corrections are pushed in draft PR #601; the product serialization fix and green current-head proof are still pending. The bench-ci owner must reconcile the original reservation/recovery hold and record supported worker restoration before hardware proof resumes. No replacement job or host/lock change was attempted.

@VijitSingh97 VijitSingh97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed by Hermes agent (operator's delegate) against origin/develop b82758b: full diff read, make test-suite run per PR, body claims reproduced. Not a substitute for the owner's approval; merge stays with the operator.

Verdict: RETURN — branch contains test scaffolding but zero product code; #588 is not fixed here

The diff touches only tests/ (new test-thread-profile.py, e2e-thread-profile.sh, wiring). generate_xmrig_config still writes cpu.rx as a bare integer (rigforge.sh:1131,1153) and the THREADS_CAP clamp re-writes a scalar at rigforge.sh:1199 — exactly the #588 defect. The 9 failing Test-suite/Coverage tests are the expected red state of this test-first draft (count(): "XMRig profile is not an array or object"); local repro matches CI run 40437285290. The body itself says "Implementation … pending. This draft is not ready for independent review."

Fix-forward is bounded, all in rigforge.sh: (1) emit the XMRig-6 object form at :1131/:1153 ({mode:0, threads:$rx, max_threads_cnt:$rx, intensity:1, affinity:-1}); (2) cap max_threads_cnt/threads at :1199 instead of writing an integer; (3) normalize every cpu.rx reader/writer to number-or-object: :1520-1522 (HugePages sizing), :3328 (tuner overlay writes a scalar — must move with the writers or the deep-merge at :1185 type-clashes), :3042 (tune-summary display). The 9 tests then go green unmodified (they encode the operator ruling: auto 8 / tuned 12 / cap 16 → runs 8). One rebase hunk: tests/README.md release-e2e table row (union develop's upgrade-phase wording with the thread-profile phase). Hardware/tier4-e2e proof remains blocked on bench-ci#2248.

@#600 tracker: this keeps #588's "proven by ~2026-10-30" fallback clause live — the fallback (cut 1.18.1 without #588) is the current trajectory unless the implementation lands.

XMRig 6.26.0 parses cpu.rx as a per-thread array or a profile object; a
bare integer falls back to the full-core default, so the setup cap was
silently ignored and autotune benchmarked against a knob it never set.

cpu.rx is now written in exactly the forms XMRig honors: omitted for
auto, a {mode,threads,max_threads_cnt,intensity,affinity} object when a
count is requested, or the observed auto array. A bounded probe run
(sink pool, no HTTP/DAEMON/MSR) observes the auto profile XMRig picks on
this box; every requested count - the setup cap and each tune candidate -
is clamped to it, so a cap can never raise thread count. Readers
(RX_SETUP_THREADS display, tune knobs, overrides) accept all forms.

Helpers live in util/rx-profile.sh, sourced beside the script: SCRIPT_DIR
is the worker home under RIGFORGE_HOME, not the script tree. The e2e
assertion counting threads through /2/backends stays gated to bench-ci
hardware (bench-ci#2248).
@VijitSingh97
VijitSingh97 marked this pull request as ready for review November 4, 2026 17:47
@VijitSingh97

Copy link
Copy Markdown
Contributor

3bbe717 lands the RETURN scope in product code: object-form cpu.rx at the two writers and the tuner overlay, cap and every tune candidate clamped to the probed XMRig auto profile, all readers (HugePages sizing, tune summary, fake xmrig) normalized. The nine regressions go green unmodified; make test-suite is 2428/0 at this head; file budget holds (util/rx-profile.sh keeps rigforge.sh at its 6307-line ceiling). Hardware proof via /2/backends remains gated to bench-ci hardware and pending bench-ci#2248 — the e2e phase is ready for that queue. @#600 tracker: #588's "proven by ~2026-10-30" fallback clause can stand down once the bench-ci run at this head lands.

@VijitSingh97

Copy link
Copy Markdown
Contributor

Merge conflict resolved (tests/README.md release-e2e row unioned: develop's missing-minimum-marker upgrade phase + this branch's thread-profile phase and bench-ci phase entry), merged head f42e315. Operator-delegate verification on this head: make test-suite 2432 passed / 0 failed; python3 tests/test-thread-profile.py 12/12; all tracked *.sh parse clean under /bin/bash 3.2.57 (bash -n sweep). Review of the probe (util/rx-profile.sh): Linux+binary gated, sink config 0600 in mktemp dir with dead 127.0.0.1:1 pool, huge-pages/wrmsr/rdmsr and http off, deadline-bounded kill, temp cleanup; cpu.rx emitted only as observed array / threads object / omitted, eff = min(auto, cap, request) — matches the operator ruling the tests encode. Hardware /2/backends proof remains gated on bench-ci#2248; awaiting operator approval.

@pithead-developer

Copy link
Copy Markdown
Contributor Author

Operator review of head f42e315 (not an approval): please do not merge yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stage/blocked its issue waits on another issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

threads cap ignored: cpu.rx is written as an integer, which XMRig 6.26.0 does not accept (mines on every core)

1 participant