Justus Schock 0ec4107697 Optimizer closure (#4190) ... * closure for all optimizers * rename hook and take care of alternating backwards * add comment * training_loop_fix * closure whenever possible * training_loop * simple tests that count backward calls * fix test to work with closure * remove debugging statement * better place * check grads after backward * start fixing manual optimization * skip step when result returned by closure was None * fix gradient clipping test to work with closure * attribute dict result only for automatic optimization * adjust backward calls in accelerator * adjust where to call gradient clipping * adjust backward calls in tests * Apply suggestions from code review Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com> * pass kwargs to xla optimizer Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>		2020-10-21 19:34:29 +01:00
..
__init__.py	ref: decouple apex second attemp part 6/n (#4060)	2020-10-10 15:28:25 -04:00
test_backward_calls.py	Optimizer closure (#4190)	2020-10-21 19:34:29 +01:00
test_manual_optimization.py	remove duplicate metric vs step log for train loop (#4173)	2020-10-15 10:47:00 -04:00