Fix zero-centered gamma parity for ET RMSNorm (#23193) - #23193
chinnadhurai wants to merge 1 commit into
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23193
Note: Links to docs will display an error until the docs builds have been completed. ⏳ No Failures, 1 PendingAs of commit 45a262e with merge base 36302a1 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
|
|
@chinnadhurai has exported this pull request. If you are a Meta employee, you can view the originating Diff in D121895059. |
This PR needs a
|
Summary: Preserve zero-centered gamma semantics for transformer block and output RMSNorms in both ExecuTorch and rlformers ET paths while leaving Q/K norms unchanged. Preserve output_norm_gain_center_type so center-zero output weights are not shifted. Authored with assistance from Codex. Differential Revision: D121895059
2142b7e to
45a262e
Compare
Summary:
Preserve zero-centered gamma semantics for transformer block and output RMSNorms in both ExecuTorch and rlformers ET paths while leaving Q/K norms unchanged. Preserve output_norm_gain_center_type so center-zero output weights are not shifted.
Authored with assistance from Codex.
Differential Revision: D121895059