Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
lightning/pytorch_lightning/plugins/precision
History
Carlos Mocholí 03f01fb5ec
Fix gradient norm tracking and gradient clipping (#9287) 
* WIP

* Progress

* Undo test change

* Fix plugin closure execution order

* Update CHANGELOG

* Fix manual optimization on AMP and skipping backward

* Fix for deepspeed

* Typo

* Hook test for manual closure

* Add skipping test with AMP

* You are hideous, apex

* Add deepspeed test

* Update CHANGELOG

* Fix for broken master

* Add RunIf

* FIXMEs

* Rename

* Fix grad norm

* add a simple test

* update test

* update  test

* update test

* fix merge conflicts

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Sea of changes

* Undo change

* Introduce TPUPrecisionPlugin

* Undo changes

* Undo changes

* Resolve FIXME

* Undo change

* Undo change

* Undo change

* Fix FIXMEs

* Fix FIXME

* Correct value

* Bad merge

* Fix circular imports

* WIP

* Fixing clipping

* Fixes

* Bad merge

* Move optimizer step and clipping into the `PrecisionPlugin`

* Fix AMP

* Update CHANGELOG

* Fix tests

* Underscore

* Progress

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove pre_optimizer_step

* Missed one

* Progress

* Progress

* Fix test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Update FIXMEs

* Fix test

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix test

* DeepSpeed warning. mypy

* Rename

* Finish tests

* Update CHANGELOG

* Dumb fixes

* accelerator=auto

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update on comments

* Use ClassifModule

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
 		2021-10-28 15:23:27 +00:00
..
__init__.py Rename `TPUHalfPrecisionPlugin` to `TPUBf16PrecisionPlugin` (#10026) 2021-10-19 21:09:37 +00:00
apex_amp.py Move optimizer step and clipping into the `PrecisionPlugin` (#10143) 2021-10-26 17:26:26 +02:00
deepspeed_precision.py Fix gradient norm tracking and gradient clipping (#9287) 2021-10-28 15:23:27 +00:00
double.py Introduce `PrecisionPlugin.forward_context()` (#9988) 2021-10-18 12:58:19 +00:00
fully_sharded_native_amp.py Pass the scaler as an input to `NativeMixedPrecisionPlugin` (#10055) 2021-10-28 14:13:53 +00:00
ipu_precision.py Move optimizer step and clipping into the `PrecisionPlugin` (#10143) 2021-10-26 17:26:26 +02:00
mixed.py CI: precommit - docformatter (#8584) 2021-09-06 12:49:09 +00:00
native_amp.py Pass the scaler as an input to `NativeMixedPrecisionPlugin` (#10055) 2021-10-28 14:13:53 +00:00
precision_plugin.py Fix gradient norm tracking and gradient clipping (#9287) 2021-10-28 15:23:27 +00:00
sharded_native_amp.py Pass the scaler as an input to `NativeMixedPrecisionPlugin` (#10055) 2021-10-28 14:13:53 +00:00
tpu.py Move optimizer step and clipping into the `PrecisionPlugin` (#10143) 2021-10-26 17:26:26 +02:00
tpu_bf16.py Rename `TPUHalfPrecisionPlugin` to `TPUBf16PrecisionPlugin` (#10026) 2021-10-19 21:09:37 +00:00