94f6c3e160
Advanced GPU Documentation ( #7259 )
...
* Added advanced gpu section
* Small changes
* Better documentation
* Address code review
* Add warning about using trainer.model, clean up some of the examples
* Add section for ddp, remove references and old sequential documentation
* Remove Fully Sharded documentation for now
* Apply suggestions from code review
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
* Address code review
* Address code review
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
2021-05-06 13:53:20 +01:00
49c8740338
update docs about transfer_batch_to_device hook while using DP ( #7343 )
...
* update
* clarify
* update
2021-05-05 13:51:16 +00:00
a6aa1a0f82
make gpus=str in Trainer consistent with command line parsing of string ( #6388 )
...
* string gpu input
* update docs
* deprecation warning
* Revert "update docs"
This reverts commit c5f3893413
2021-05-04 09:56:27 +00:00
bfac0e899f
warn about dp + manual optimization in docs ( #7230 )
...
* add warning
* typo
* add link
2021-04-28 09:01:06 +01:00
5cf9afa176
Add fairscale install msg for Sharded Plugins ( #7213 )
2021-04-27 08:22:44 +00:00
e9fca760ac
Set `DistributedSampler` seed if `seed_everything` was called ( #7024 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-04-19 14:50:31 +01:00
7b0b6c3112
Update DeepSpeed docs after single saving PR ( #7036 )
2021-04-15 08:56:04 -07:00
f2021ed225
docs: lightning-bolts ( #6967 )
2021-04-13 18:49:48 +00:00
19e67d18c4
Docs fixes ( #6870 )
2021-04-07 16:57:22 +01:00
f9bb7c641a
DeepSpeed ZeRO Docs update ( #6752 )
...
* Added base docs
* Add more information
* Apply suggestions from code review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-03-30 21:52:02 +00:00
42a7b70585
Add DDP Spawn being default for Multi GPUs ( #6292 )
2021-03-21 22:10:54 +01:00
383565d225
Update DeepSpeed docs ( #6528 )
...
* Clean up docs and add some explicitness around stages
* Apply suggestions from code review
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2021-03-15 18:00:21 +00:00
75c6486ac7
update ( #6403 )
2021-03-09 09:47:51 +00:00
863a70c294
Add specifics around DeepSpeed docs ( #6142 )
...
* Be more specific with DeepSpeed compatibility
* Better wording
2021-02-23 00:08:39 +01:00
7189d673f6
DeepSpeed Integration ( #5954 )
...
* Add initial deepspeed changes
* Address code review
* Move static method outside of function
* Fixes
* Add missing annotation
* Remove seed setting
* Doc changes
* Doc changes, add address reviews
* Fix docs
* Try fixing issue by moving to torch adam
* Clean up check
* Changes, better APIs!
* Add wrapper, swap to git install revision
* Add special test
* Add warning
* Address review
* Add better disclaimer
* Turn off ZeRO for testing due to compilation
* Add description on modifying parameters via the plugin
* Doc strings clear
* Small doc fixes
* Fix hash, reduce test
* Added CI change
* Move to azure pipeline
* Fix test name
* Add missing flag
* Remove sudo...
* Try conda instead
* Swap to conda base
* Try suggested install
* Apply suggestions from code review
* Apply suggestions from code review
* Revert "Apply suggestions from code review"
This reverts commit 41cca05ae06ec29e3450eaca
2021-02-17 15:23:42 -05:00
6a409c7f84
remove outdated info ( #6032 )
2021-02-17 20:57:57 +01:00
5700fd091f
[Feat] Add TORCH_DISTRIBUTED_BACKEND env variable ( #5981 )
...
* add backend support
* resolve flake8
* update changelog
* update
* Apply suggestions from code review
* Update docs/source/advanced/multi_gpu.rst
* add patch as context manager
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-02-17 16:37:39 +00:00
960a60743f
fix fairscale compatible with PT 1.8 ( #5996 )
...
* try to extend fairscale available
* 1.2
2021-02-16 19:43:02 +00:00
1c87f1f6cd
remove legacy plugins ( #5950 )
...
* remove legacy plugins
* imports
* formatting
* fix docs references
* fix cluster environment inheritance
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-02-16 19:20:58 +00:00
0ff4c56d9b
fix typo in multi-gpu docs ( #5402 )
...
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2021-02-03 19:39:28 +01:00
7e2e874d95
Refactor: legacy accelerators and plugins ( #5645 )
...
* tests: legacy
* legacy: accel
* legacy: plug
* fix imports
* mypy
* flake8
2021-01-26 20:04:36 -05:00
dee5553b2b
move to Pages dir ( #4869 )
...
* folders
* common / advanced / extensions
* paths
* flake8
* isort
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-01-26 15:07:07 -05:00
Sean Naren