c00d5709c4
ci: update recurent events ( #5480 )
...
* ci: update recurent events
* split events
* .
* .
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2021-01-12 09:53:41 -05:00
9611a7f897
update nightly & upgrade Twine ( #5458 )
...
* update used Twine
* .
* .
* install
* install
* .
* .
* .
* .
* .
* .
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2021-01-12 07:56:20 -05:00
d30e316a35
[docs] Add ananthsub to core ( #5476 )
...
* Update test_manual_optimization.py
* Update governance.rst
* Update test_manual_optimization.py
* Update test_manual_optimization.py
2021-01-12 09:42:23 +01:00
635df27880
[BUG] Check environ before selecting a seed to prevent warning message ( #4743 )
...
* Check environment var independently to selecting a seed to prevent unnecessary warning message
* Add if statement to check if PL_GLOBAL_SEED has been set
* Added seed test to ensure that the seed stays the same, in case
* if
* Delete global seed after test has finished
* Fix code, add tests
* Ensure seed does not exist before tests start
* Refactor test based on review, add log call
* Ensure we clear the os environ in patched dict
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: chaton <thomas@grid.ai>
2021-01-11 23:30:27 -05:00
f065ea65bf
populate some more legacy checkpoints ( #5457 )
...
* populate some more legacy checkpoints
* .
* pt freeze
* .
* skip
Co-authored-by: chaton <thomas@grid.ai>
2021-01-11 20:36:12 -05:00
87482935a3
Add automatic optimization property setter to lightning module ( #5169 )
...
* add automatic optimization property setter to lightning module
* Update test_manual_optimization.py
Co-authored-by: chaton <thomas@grid.ai>
2021-01-11 11:21:10 -05:00
92bbf2fdd6
GH action - auto-update PRs ( #5451 )
...
* GH action - auto-update PRs
* .
2021-01-11 07:32:28 -05:00
499d5031e8
fix typos in validation_step and test_step docs ( #5438 )
...
* fixed docs in lightning.py
* few more
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2021-01-11 11:02:30 +00:00
f1e28d1e43
GH action - label conflicts ( #5450 )
...
* GH action - label conflicts
* .
* trigger
* trigger
* .
2021-01-10 15:33:46 -05:00
bb5031b3bf
bugfix: Resolve interpolation bug with Hydra ( #5406 )
...
* resolve bug
* Apply suggestions from code review
* resolve package import
* resolve import
* update on comments
* update on comments
* hacky fix
* update
* exit
* update
* to_container
* typo
* resolve import
* update
* resolve pep8
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
2021-01-09 08:55:55 -05:00
a053d758d0
[bugfix] Logging only on `not should_accumulate()` during training ( #5417 )
...
* resolve bug
* resolve tests
* update
* Update tests/loggers/test_tensorboard.py
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-01-08 19:35:47 -05:00
f2e99d617f
deprecate enable_pl_optimizer as it is not restored properly ( #5244 )
...
* update
* clean test
* still in progress
* udpdate test
* update
* update
* resolve flake
* add test for zero_grad
* update
* works without accumulated_grad
* update
* update
* resolve amp
* revert back to True
* update
* clean tests
* cleaned out
* typo
* update test
* git repare bug
* remove print
* udpate
* Fix formatting/optimizer imports
* Refactor the test for cleanliness
* Add vanilla model to the test, better var names
* Fixed var names, let's clean up these mock tests
* repare test
* update test
* resolve flake8
* add manual_optimization
* update tests
* resolve flake8
* add random accumulate_grad_batches
* improve test
* Update tests/trainer/optimization/test_parity_automatic_optimization.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/trainer/optimization/test_parity_automatic_optimization.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* update
* clean tests
* correct bug
* Apply suggestions from code review
* format
* adress comments
* update on comments
* wip
* typo
* depreceate enable_pl_optimizer
* resolve latest bugs
* update
* resolve merge
* add comment
* Update pytorch_lightning/core/lightning.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/deprecated_api/test_remove_1-3.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/trainer/connectors/optimizer_connector.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/trainer/trainer.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/trainer/trainer.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update tests/trainer/optimization/test_parity_automatic_optimization.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* update on comments
* update restore
* add a property
* remove setstate as not needed anymore
* update test
* provide optimizer to on_before_zero_grad
* update on comments
* update on comments
* Update pytorch_lightning/trainer/trainer.py
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Update tests/trainer/optimization/test_parity_automatic_optimization.py
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Update tests/trainer/optimization/test_parity_automatic_optimization.py
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Update tests/trainer/optimization/test_parity_automatic_optimization.py
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* mofidy import
* update changelog
* resolve flake8
* update
* update
* clean doc
Co-authored-by: SeanNaren <sean@grid.ai>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-62-109.ec2.internal>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
2021-01-08 16:13:12 -05:00
d510707bc9
[bug-fix] Call transfer_batch_to_device in DDPlugin ( #5195 )
...
* hacking out
* update
* remove useless on_before_forward
* update
* remove overriden
* iremove os
* use on_before_forward
* resolve flake8
* add test
* update
* add single_process_per_device
* resolve flake8
* update
* resolve
* update
* update
* update
* add comment
* resolve bug with sharded
* update
* remove property
* update
* resolve test
* resolve bug
* update on comments
* update doc
* Update pytorch_lightning/core/hooks.py
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* update on comments
* Update pytorch_lightning/plugins/ddp_plugin.py
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* Update pytorch_lightning/plugins/ddp_plugin.py
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* resolve pep8
* add device_ids to pipe
* update on comments
* update
* resolve
* update
* update
* update
Co-authored-by: Ubuntu <ubuntu@ip-172-31-62-109.ec2.internal>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
2021-01-08 12:05:22 -05:00
72525f0a83
tests for legacy checkpoints ( #5223 )
...
* wip
* generate
* clean
* tests
* copy
* download
* download
* download
* download
* download
* download
* download
* download
* download
* download
* download
* flake8
* extend
* aws
* extension
* pull
* pull
* pull
* pull
* pull
* pull
* pull
* try
* try
* try
* got it
* Apply suggestions from code review
2021-01-08 10:36:49 -05:00
4c6f36e6e1
Fix pre-commit trailing-whitespace and end-of-file-fixer hooks. ( #5387 )
2021-01-07 10:54:47 +05:30
cc624358c8
docker: run ci only docker related files are changed ( #5203 )
...
* only run ci on docker related files
* docker related files changed!
* install pytorch along with cudatoolkit
* build docker only on SUN
* conda exit status has been fixed
* reverts back to old conda version
* add more docker related files
* conda env update --name
* create env and install pytorch again
* create env and install pytorch again
* ${PYTORCH_CHANNEL}
* dont update pytorch with conda env update
* Apply suggestions from code review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update dockers/base-conda/Dockerfile
* Apply suggestions from code review
* remove checks in cron job
* Apply suggestions from code review
* readd #
* readd #
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
2021-01-06 11:59:42 -05:00
ee8373110a
Update sharded install to latest fairscale release, add reasoning why fork required for sequential parallelism ( #5380 )
2021-01-06 10:02:13 -05:00
019e4ff8cd
Add 1.1.4 section to CHANGELOG ( #5378 )
2021-01-06 07:27:59 -05:00
4d9db866a1
Prepare 1.1.3 release ( #5365 )
...
* Prepare 1.1.3 release
* Fix flake8 error
* suppress
* Remove 1.1.4 section
* Add missing commits to CHANGELOG
* Update PR template
* Add missing commit
* fix
* Update CHANGELOG.md
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-01-05 18:43:46 -05:00
6536ea42fa
FIX-5311: Cast to string `_flatten_dict` ( #5354 )
...
* fix
* params
* add test
* add another types
* chlog
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-01-05 17:04:53 -05:00
ec0fb7a3ec
refactor imports of logger dependencies ( #4860 )
...
* refactor imports of logger dependencies
* fix
* fix
* fix
* name
* fix
* mocks
* fix tests
* fix mlflow
* fix test tube
* fix wandb import check
* whitespace
* name
* name
* hack
* hack
* rev
* fix
* update mlflow import check
* try without installing conda dep
* .
* .
* .
* .
* .
* .
* .
* .
* .
Co-authored-by: Adrian Wälchli <adrian.waelchli@inf.unibe.ch>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-01-05 14:34:47 -05:00
410d67fbe8
Existence check for hparams now uses underlying filesystem ( #5250 )
...
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2021-01-05 10:36:06 -05:00
d568533b6b
Updated metrics/classification/precision_recall.py ( #5348 )
...
There was a typo in Documentation of Code of the ```compute()``` function of ```Recall``` metric at line 210. It said "Computes accuracy over state." which should have been "Computes recall over state."
2021-01-05 08:15:06 -05:00
a40e3a325e
Change the classifier input from 2048 to 1000. ( #5232 )
...
* Change the classifier input from 2048 to 1000.
* Update docs for Imagenet example
Thanks @rohitgr7
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-01-05 07:09:52 -05:00
d5b367871f
[bug-fix] Trainer.test points to latest best_model_path ( #5161 )
...
* resolve bug
* update code
* add set -e
* Update pytorch_lightning/callbacks/model_checkpoint.py
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* update test
* Update tests/checkpointing/test_trainer_checkpoint.py
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
* Update tests/checkpointing/test_trainer_checkpoint.py
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* update on comments
* resolve test
* convert to set
* update
* add error triggering
* update
* update on comments
* update
* resolve import
* update
* update
* Update pytorch_lightning/plugins/rpc_plugin.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* update
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Ubuntu <ubuntu@ip-172-31-62-109.ec2.internal>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-01-05 05:01:59 -05:00
062800aa99
Fix invalid value for weights_summary ( #5296 )
...
* Fix weights_summary
* use mode
* fix
* optional
* what was I thinking
2021-01-05 02:43:18 -05:00
371daea594
Allow log_momentum for adaptive optimizers ( #5333 )
...
* fix
* fix
* chlog
* no momentum warning
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* ref
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-01-05 01:51:22 -05:00
c7d0f4c3a2
Add a check for optimizer attatched to lr_scheduler ( #5338 )
...
* add a check for scheduler and optimizer
* pep
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-01-05 01:07:28 -05:00
f740245521
Disable checkpointing, earlystopping and logging with fast_dev_run ( #5277 )
...
* Disable checkpointing, earlystopping and logger with fast_dev_run
* docs
* chlog
* disable callbacks and enable DummyLogger
* add log
* use dummy logger method
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-01-04 21:54:49 -05:00
b0051e8c03
Add non-existing resume_from_checkpoint acceptance for auto-resubmit ( #4402 )
...
* Add empty resume_from_checkpoint acceptance #4366
* Fix general error catch with focused file check
* Add fsspec HTTP extras
Add fsspec's HTTPFileSystem support through http extras.
pl has supported remote http file (e.g. #2925 ),
so this commit do not add new functionality.
* Fix potential too much logging in DDP
* Add PR changelog
* Add well-written argument explanation
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Fix DDP-compatible restore logging
Notify from where the states are restored.
This feature temporally deleted as a result of PR review.
With succeeding review, added with DDP compatibility.
* Fix utility import pathes
* Refactor load step commentaries
* Refactor hpc ckpt suffix acquisition
* Refactor restore/hpc_load match
* Refactor hpc load trial
* Refactor checkpoint dir check
* Refactor unneeded function nest
* Refactor nested If
* Refactor duplicated cache clear
* Refactor attempt flow with if/elif
* Fix pip8
* Refactor hook commentary
Co-authored-by: chaton <thomas@grid.ai>
* Fix pep8
* Refactor hpc load checkpoint path acquisition
* Fix pip8
* Fix typo
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Fix typo
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Fix doc
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Refactor None Union type with Optional
* Fix build-doc CI failure debuged in #5329
* Fix fsspec import during build-doc #5329
* Fix test epoch
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Fix test with latest test models
* .
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
2021-01-04 19:52:35 -05:00
dd442b6d33
[Docs] update docs for resume_from_checkpoint ( #5164 )
...
* update docs and add pathlib support
* fix
2021-01-04 15:03:14 -05:00
15a400b95f
docs: logits -> probs in Accuracy metric documentation ( #5340 )
...
* fix: logits -> probs in accuracy metrics documentation
* Update metrics.rst
* Update metrics.rst
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-01-04 10:40:01 -05:00
0e593fb6a8
Reordered sections for intuitive browsing. (e.g. limit_train_batches was at the end of the page, far from limit_test/val_batches) ( #5283 )
...
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-01-04 09:05:24 -05:00
17a0784c5e
black formatting and migrated to self.log logging in finetuning example ( #5229 )
...
* black formatting and migrated to self.log logging
* Apply suggestions from code review
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* migrated to accuracy in the metrics package
migrated to accuracy in the metrics package
* removed trailing whitespace
* Apply suggestions from code review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2021-01-04 14:20:04 +01:00
51af3957fc
uniques docs artefact name ( #5336 )
2021-01-04 08:38:46 +01:00
724f1051f0
update isort config ( #5335 )
...
* update isort config
* apply
2021-01-03 23:13:22 +01:00
d20fd8e5ab
supports --num-nodes on DDPSequentialPlugin() ( #5327 )
2021-01-02 14:29:02 +01:00
ab7512d7ba
refactor python in GH actions ( #5281 )
...
* refactor python in GH actions
* .
* .
2020-12-31 17:24:33 +01:00
64163c2662
[Docs] Mention that datamodules can also be used with `.test()` method ( #5286 )
...
* docs
* Apply suggestions from code review
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
* ref
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2020-12-31 03:25:30 -05:00
dd98a60e90
Fixed typo in docs for optimizer_idx ( #5310 )
...
There were four instances where `optimizer_idx` was the argument for **`optimizer_step()`** under [Step optimizers at arbitrary intervals][#link-to-docs]. However, instead of using it what was being used inside (erroneously) was `optimizer_i`.
[#link-to-docs]: https://pytorch-lightning.readthedocs.io/en/latest/optimizers.html#step-optimizers-at-arbitrary-intervals
2020-12-31 11:33:22 +08:00
4913cbb987
Fix metric state reset ( #5273 )
...
* Fix metric state reset
* Fix test
* Improve formatting
Co-authored-by: Ananya Harsh Jha <ananya@pytorchlightning.ai>
2020-12-29 16:09:10 -05:00
dabfeca92e
[Metrics] [Docs] Add section about device placement ( #5280 )
...
* update docs
* Update docs/source/metrics.rst
Co-authored-by: Shreeyak <shreeyak.sajjan@gmail.com>
* Update docs/source/metrics.rst
Co-authored-by: Shreeyak <shreeyak.sajjan@gmail.com>
* Update docs/source/metrics.rst
Co-authored-by: Shreeyak <shreeyak.sajjan@gmail.com>
* Update docs/source/metrics.rst
* Update docs/source/metrics.rst
* Update docs/source/metrics.rst
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Update docs/source/metrics.rst
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update docs/source/metrics.rst
* Update docs/source/metrics.rst
* try fix failing doc test
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
Co-authored-by: Shreeyak <shreeyak.sajjan@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-12-29 04:06:28 -05:00
0c7c9e8540
Apply isort to `pl_examples/` ( #5291 )
...
* Remove examples from isort ignore list
* Apply isort
2020-12-29 13:49:02 +05:30
eb1d61caaf
remove docs ( #5287 )
...
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2020-12-28 14:32:53 -05:00
9ebbfece5e
Trainer.test should return only test metrics ( #5214 )
...
* resolve bug
* merge tests
2020-12-28 15:34:18 +01:00
d1e97a4f11
Fix typo in doc ( #5270 )
2020-12-27 00:22:53 +01:00
8d8098c04e
Minor doc fixes ( #5139 )
...
* minor doc fix
* minor doc fix
* Apply suggestions from code review
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* suggestions
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2020-12-24 18:37:30 +00:00
90c1c0f68b
Update README.md ( #5018 )
...
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2020-12-24 06:15:46 -05:00
b930b5f222
Add TPU example ( #5109 )
...
* Add TPU example
* add badge
* add badge
* add badge
* bullets
* name
* trigger
* add dataset name
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
2020-12-24 03:15:24 -05:00
1d533074b3
Fix typo in Trainer.test() ( #5226 )
...
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-12-24 02:41:24 -05:00
Jirka Borovec