lightning

Commit Graph

Author	SHA1	Message	Date
Sean Naren	432e5637d6	Expose DeepSpeed FP16 parameters due to loss instability (#6115 ) * Expose deepspeed config parameters to init function due to instability in parameters * See if tests can run on normal CI, without special tests * Add changelog * Update pytorch_lightning/plugins/training_type/deepspeed.py Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com> Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>	2021-02-21 21:43:11 +01:00
Sean Naren	3b0e4e0b2b	Enable ZeRO tests for CI, fix to/half function calls (#6070 ) * Enable ZeRO optimization, and make sure that the lightning module hook is called when we move to half precision * Added test, update to function	2021-02-21 00:24:44 +00:00
Adrian Wälchli	6cc1a06078	rename accelerator_backend -> accelerator (#6034 ) * rename accelerator backend * rename new additions from master * add proper deprecation * pep8 * warning match * add missing warning type	2021-02-18 15:54:12 +00:00
Sean Naren	7189d673f6	DeepSpeed Integration (#5954 ) * Add initial deepspeed changes * Address code review * Move static method outside of function * Fixes * Add missing annotation * Remove seed setting * Doc changes * Doc changes, add address reviews * Fix docs * Try fixing issue by moving to torch adam * Clean up check * Changes, better APIs! * Add wrapper, swap to git install revision * Add special test * Add warning * Address review * Add better disclaimer * Turn off ZeRO for testing due to compilation * Add description on modifying parameters via the plugin * Doc strings clear * Small doc fixes * Fix hash, reduce test * Added CI change * Move to azure pipeline * Fix test name * Add missing flag * Remove sudo... * Try conda instead * Swap to conda base * Try suggested install * Apply suggestions from code review * Apply suggestions from code review * Revert "Apply suggestions from code review" This reverts commit `41cca05a` * Revert "Apply suggestions from code review" This reverts commit `e06ec29e` * Remove setter * Address most review * Move out function, remove DeepSpeed from requirements * Install deepspeed/mpi4py within container * Use special tests, move to master commit for deepspeed * Export path * Force compile to happen first * Remove! * Debugging ninja * Fix error in optimizer step logic * Attempt to fix symbolic link * Reverse to aid debugging * Export path again * Clean up mess * var * Revert "var" This reverts commit `3450eaca` * Address review, add todo * Add note about unsupported functionality Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com> Co-authored-by: tchaton <thomas@grid.ai> Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>	2021-02-17 15:23:42 -05:00

4 Commits