Skip to content

Conversation

@vsandwar-sumer
Copy link

@vsandwar-sumer vsandwar-sumer commented Dec 20, 2025

What does this PR do?

This PR resolves a bug introduced in #15555 that caused errors to occur when a user trains a model with bf16-mixed which doesn't have a scaler, gradient clipping and fused adamW optimizer. The protection was there as AMPs with scaler would clash with the fused optimizer as the scaling was contained in the process. However, it only checked if the optimizer required internal scaling and not whether the AMP datatype did. Adding an additional condition there should resolve this issue.

Fixes #21435

Before submitting
  • Was this discussed/agreed via a GitHub issue? (not for typos and docs)
  • [ x ] Did you read the contributor guideline, Pull Request section?
  • [ x ] Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes? (if necessary)
  • [ x ] Did you write any new necessary tests? (not for typos and docs)
  • [ x ] Did you verify new and existing tests pass locally with your changes?
  • Did you list all the breaking changes introduced by this pull request?
  • [ x ] Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist
  • [ x ] Is this pull request ready for review? (if not, please submit in draft mode)
  • [ x ] Check that all items from Before submitting are resolved
  • [ x ] Make sure the title is self-explanatory and the description concisely explains the PR
  • [ x ] Add labels and milestones (and optionally projects) to the PR so it can be classified

📚 Documentation preview 📚: https://pytorch-lightning--21436.org.readthedocs.build/en/21436/

@github-actions github-actions bot added the pl Generic label for PyTorch Lightning package label Dec 20, 2025
@vsandwar-sumer vsandwar-sumer changed the title update: tests fix: remove catch protections for unscaled AMP types Dec 20, 2025
@vsandwar-sumer vsandwar-sumer changed the title fix: remove catch protections for unscaled AMP types fix: remove catch protections for unscaled AMP types for gradient scaler Dec 20, 2025
@codecov
Copy link

codecov bot commented Dec 22, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79%. Comparing base (2e25642) to head (7a52a99).
⚠️ Report is 1 commits behind head on master.
✅ All tests successful. No failed tests found.

❗ There is a different number of reports uploaded between BASE (2e25642) and HEAD (7a52a99). Click for more details.

HEAD has 385 uploads less than BASE
Flag BASE (2e25642) HEAD (7a52a99)
cpu 119 30
lightning_fabric 30 0
pytest 60 0
python3.12 36 9
lightning 59 15
python3.11 23 6
python3.12.7 36 9
python3.10 12 3
python 12 3
pytorch2.9 6 3
pytorch_lightning 30 15
pytest-full 59 30
pytorch2.1 12 6
pytorch2.7 6 3
pytorch2.8 6 3
pytorch2.2.2 6 3
pytorch2.4.1 6 3
pytorch2.5.1 6 3
pytorch2.3 5 3
pytorch2.6 6 3
Additional details and impacted files
@@            Coverage Diff            @@
##           master   #21436     +/-   ##
=========================================
- Coverage      87%      79%     -8%     
=========================================
  Files         270      267      -3     
  Lines       24059    24004     -55     
=========================================
- Hits        20859    18952   -1907     
- Misses       3200     5052   +1852     

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pl Generic label for PyTorch Lightning package

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bf16-mixed causing issues with fused AdamW

1 participant