Repository navigation
fix(autograd): schedule backward over the root-reachable graph - #276
Open
superme1one wants to merge 2 commits into
Open
superme1one wants to merge 2 commits into
superme1one wants to merge 2 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backward currently waits for forward consumers outside the selected loss and for every differentiable output of a multi-output function. Creating an unused branch, or selecting only one
Splitoutput, can therefore leave leaf gradients null.Fixes #273.
Ten device-parameterized regression tests cover live/destroyed unused branches, shared intermediates and repeated edges, uneven
Splitoutputs, all-used output accumulation, null-gradient propagation and lifetime cleanup, non-differentiableTopKindices, both checkpoint modes, and pending shared-parameter gradients during reentrant backward. The original branch/Split cases fail on unmodifiedmaster.Validation on Linux x86_64, CPU Release, Clang 18.1.8/GCC 13.4.0 libstdc++, CMake 3.31.6, with CUDA/NCCL/OpenMP disabled:
test_tensor_cpu,test_autograd_cpu,test_hook_cpu,test_module_cpu,test_optimizer_cpu, andtest_checkpoint_cpu: successful, with six existing CPU skips for accelerator/autocast cases.master; neither depends on the other.Runtime validation in this submission was CPU-only. Upstream CI requires maintainer approval for this contributor; the fork results use the same workflow definitions.