Skip to content

fix(optimizer): track Adam updates per parameter - #267

Open
superme1one wants to merge 2 commits into
InfiniTensor:masterfrom
superme1one:fix/adam-per-parameter-steps
Open

superme1one wants to merge 2 commits into
InfiniTensor:masterfrom
superme1one:fix/adam-per-parameter-steps

Conversation

@superme1one

@superme1one superme1one commented Nov 16, 2026 •

Copy link
Copy Markdown

Summary

Track Adam update counts per parameter, serialize named counters, and retain loading of legacy global-step checkpoints.

Motivation

Parameters that receive gradients late or intermittently must use their own update count for bias correction. Closes #235.

Type of Change

fix; no public API removal.

Platforms Affected

CPU and CUDA optimizer/checkpoint code.

Smoke Test Result

Delayed, intermittent, absent and zero-gradient cases, named reorder/resume, legacy loading, and unequal-count checkpoint round trips passed. Three 10,000-step runs (SGD, default Adam and nondefault Adam) passed 675 parameter/state/counter comparisons against CPU PyTorch. Counters matched exactly; the largest Adam parameter difference was 7.16e-7.

The combined candidate stack passed 228 CPU tests (3 skips) and 229 Iluvatar GPU tests (2 CPU-only death checks skipped), with no failures. Coverage includes optimizer state, shared parameters, autograd and checkpoint round trips. The GPU build uses CoreX 4.3.0.20250624 with a separately recorded compatibility layer. NVIDIA execution and hosted CI are pending.

Test Results on Supported Platforms

Validated on the platforms listed above. Local formatting and patch-application checks passed. Hosted CI is tracked separately from the recorded target-platform tests. Affected backends not available in this session require their normal CI/owner checks.

Benchmark / Performance Impact

Correctness fix; no performance gain claimed.

Notes for Reviewers

Base: InfiniTensor/InfiniTrain@e19a56dcb56bec11f1af63c089a1e8ae03ff24c7. Verification date: 2026-11-16. Detailed test inputs, source hashes and logs are retained in the separate validation package; performance reports are excluded from the source patch.

Closes #235.

CI approval status at submission

The upstream pull-request workflows are action_required and have not scheduled jobs. A repository maintainer needs to approve the runs. Target-platform tests are recorded above. This repository did not automatically run checks for the feature-branch push. Local formatting and target-platform results are recorded above. Backend tests requiring upstream infrastructure remain pending.

Validation after include cleanup

Added direct standard-library and project includes to Adam and its optimizer/checkpoint tests; removed an unused include. Calculation behavior is unchanged.

  • Commit: afd10d6c09be5d236fc5237f80603986abb10fb3.
  • Clean CPU Release build on the Iluvatar server: 74 passed (optimizer and checkpoint suites).
  • Full, unchanged fork workflows passed: Format Check, IWYU, clang-tidy.
  • The new upstream workflows are awaiting maintainer approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adam 对延后获得梯度的参数使用了错误的步数

1 participant