15be986558
add logger to __all__ ( #6854 )
2021-06-09 13:07:02 +00:00
7f4ef6d135
Fix logs overwriting issue for remote fs ( #7889 )
...
* Fix logs overwriting issue for remote fs
* Add test
2021-06-09 11:05:01 +02:00
9097347ea8
feat(wandb): log models as artifacts ( #6231 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-05-27 20:15:02 +02:00
e2ead9abd7
Refactor some loops code and hook tests ( #7682 )
2021-05-25 13:27:54 +02:00
7eafd8eac6
Add run_name argument to the MLFlowLogger constructor ( #7622 )
...
* Add run_name argument to the MLFlowLogger
* Update CHANGELOG
* Fix unnecessary line
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix style by using yapf
* Fix import error when mlflow is not installed
* Update CHANGELOG.md
* Update tests/loggers/test_mlflow.py
Co-authored-by: akiyuki ishikawa <aki.y.ishikwa@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-05-21 09:17:32 +01:00
608de6abf4
TensorBoardLogger sub_dir parameter for grouping logs ( #6195 )
...
* fixed a small typo
* cleaning up
* added sub_dir argument to tensorboard and wrote test
* sub dir arg exclusively for tensorboard, linted
* resolving merge conflict
* resolved merge conflict
* resolved merge conflict
* resolved merge conflict
* resolve merge conflict before revert
* resolving merge conflict
* reverted to pre-lint
* added tensorboard sub_dir test
* pep8 formatting
* removed sub_dir arg from test_all function:
* updated feature description
* typo in doc description
* updated CHANGELOG
* Update pytorch_lightning/loggers/tensorboard.py
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
* swapped argument position
* added expandvars tests
* added expandvars
* removed model init
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix tests
* fix failed test
* Revert "fix failed test"
This reverts commit 50b34c66da
2021-05-19 19:50:58 +00:00
8c0ea92af2
`TrainerState` refactor [5/5] ( #7173 )
...
* `TrainerState` refactor
* flake8
* Update finished check
* Test cleanup
* Fix tests
* Fixes
* Reorder
* flake8
* Update CHANGELOG
* Better docs
* Better docs
* Remove default
* Update tests
* Bad merge
2021-05-04 12:50:56 +02:00
2a20102321
fix(wandb): allow custom init args ( #6989 )
...
* feat(wandb): allow custom init args
* style: pep8
* fix: get dict args
* refactor: simplify init args
* test: test init args
* style: pep8
* docs: update CHANGELOG
* test: check default resume value
* fix: default value of anonymous
* fix: respect order of parameters
* feat: use look-up table for anonymous
* yapf formatting
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-05-04 09:45:36 +00:00
e76ebd640e
[feat] Add BasePredictionWriter 3/3 ( #7127 )
...
* wip
* update
* update
* update
* update
* update
* typo
* update on comments
* update
* update
* update
* update
* update changelog
* update
* Fix merge
* Fix merge
* move code
* resolve test
* add extra test
* add an extra test
* update on comments
* add typing
* resolve flake8
* Refactor and Docs
* Fix tests
* Fix tests
* Fix tests
* Duplicate
* Fix tests
* resolve bug
* update
* update on comments
* Update pytorch_lightning/utilities/imports.py
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* Update pytorch_lightning/utilities/device_parser.py
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* update
* update
* update
* update on comments
* resolve flkae8
* update test
* Apply suggestions from code review
* update on comments
* Update pytorch_lightning/callbacks/prediction_writer.py
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
* Update pytorch_lightning/callbacks/prediction_writer.py
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
* Update pytorch_lightning/callbacks/prediction_writer.py
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
* update on comments
* update
* update on comment
* Apply suggestions from code review
* update
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-04-27 20:23:55 +00:00
ac7d6a35c3
Fix `NeptuneLogger.log_text(step=None)` ( #7194 )
2021-04-26 15:28:55 +02:00
c502e47abf
Fixed setting of _save_dir when run initiated externally ( #7106 )
...
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-04-23 01:14:46 +00:00
ef5feac7ba
fix version + yapf ( #6999 )
2021-04-22 18:25:51 +00:00
9c9e2a0325
fix gpus default for Trainer.add_argparse_args ( #6898 )
2021-04-09 11:20:43 +02:00
89b5326ca5
Fix support for symlink save_dir in TensorBoardLogger ( #6730 )
...
* Add test for symlink support and initial fix
* Respond to comment and add docstring
* Update CHANGELOG.md
* Simplify
* Update pytorch_lightning/utilities/cloud_io.py
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* Make `LightningLocalFileSystem` protected
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-04-06 11:36:25 +02:00
6b990f3fa5
Add artifcact_location arg to MLFlow logger ( #6677 )
...
* Add artifcact_location arg to MLFlow logger
* Add CHANGELOG URL
* Update test
2021-03-26 00:12:03 +01:00
615b2f7363
Improve DummyLogger ( #6398 )
...
* fix dummy logger
* docs
* update docs
* add changelog
* add none return annotation
* return empty string for name, version
2021-03-09 23:18:38 +00:00
ac583781db
Refactor: Runif for TPU and Horovod 5/n ( #6301 )
...
* TPU
* horovod
* extra
* fix
* Apply suggestions from code review
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* doc
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2021-03-02 16:21:20 +00:00
0f9134e043
Refactor: skipif for Windows 2/n ( #6268 )
...
* win
* isort
* flake8
2021-03-02 09:36:01 +00:00
40d5a9d6df
fix(wandb): prevent WandbLogger from dropping values ( #5931 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-02-27 01:52:23 +00:00
a0494aba72
MlflowLogger limit parameter value length to 250 char ( #5893 )
2021-02-16 21:22:06 +00:00
b5d29df646
Fix: hparams.yaml saved twice when using TensorBoardLogger ( #5953 )
2021-02-15 22:31:31 +05:30
a0f7831278
fix miss-leading imports in tests ( #5873 )
...
* fix imorts
* .
2021-02-09 05:10:52 -05:00
bd920b4102
Refactor simplify tests ( #5861 )
...
* add new
* restructure
* yapf
* move
* fix
2021-02-08 11:52:02 +01:00
4faaef7758
formatting tests: 4/n ( #5846 )
...
* models
* ckpt
* core
* log
2021-02-06 12:07:26 +01:00
77be6f6e24
resolve conflits
...
resolve doc
boring commit
docs
torchvision
tpu
Update dockers/tpu-tests/tpu_test_cases.jsonnet
Update dockers/tpu-tests/tpu_test_cases.jsonnet
2021-02-05 21:43:10 +01:00
5dfd62c09e
Disable training with zero num_training_batches when insufficient limit_train_batches ( #5703 )
...
* disable training when zero num_train_batches with limit_train_batches
* refactor train skip condition
* fix formatting issues
* fix formatting issues
* ref: test error msg
* fix tests for data loader calls
* fix train dataloader condition
* update limit_train_batches upper range in test comment
* remove model state check test
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-02-05 21:40:42 +01:00
2abf4693bc
Fix log_dir property ( #5537 )
...
* fix and update tests
* update with ModelCheckpoint
* chlog
* wip wandb fix
* all fixed
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-02-05 21:40:42 +01:00
4bf6dd122a
Close SummaryWriter in TensorBoardLogger on finalize ( #5696 )
...
Not entirely sure this is the "right" solution to this problem, but currently when model fitting is finished the `TensorBoardLogger` attribute `_experiment` (a `SummaryWriter`) is left with an open file handle. This causes issues in particular on Windows systems (and probably others), and also makes the files un-syncable on cloud-synced devices like OneDrive. This PR adds a `close()` to `finalize` to make sure this handle is closed upon fit completion.
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-02-05 21:40:40 +01:00
3ca1fbbf49
Ignore `step` param in Neptune logger's log_metric method ( #5510 )
...
* Ignore `step` param in Neptune logger's log_metric method
The `step` parameter is ignored because Neptune requires strictly increasing step values, a condition which is sometimes violated in Lighting e.g. when `fit()` and `test()` are called one after another on some models. `step` could be enabled again once Lightning guarantees that step values are always strictly increasing.
Also a minor bugfix: the `log_text()` method should use Neptune's `log_text()` method.
* Update neptune.py
* Update test_neptune.py
* Update test_all.py
* fix neptune tests
* add chlog
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
(cherry picked from commit 5d76b31881
2021-02-04 20:55:41 +01:00
59361d595a
fix Neptune logger creating multiple experiments when gpus > 1 ( #3256 )
...
* DP device fix
* potential fix
* fix merge
* update tests
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-02-04 20:55:40 +01:00
f422d93071
[bugfix] Logging only on `not should_accumulate()` during training ( #5417 )
...
* resolve bug
* resolve tests
* update
* Update tests/loggers/test_tensorboard.py
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
(cherry picked from commit a053d758d0
2021-01-26 14:28:47 +01:00
f0fafa2be0
feat(wandb): add sync_step ( #5351 )
...
* docs(wandb): add details to args
* feat(wandb): no sync between trainer and W&B steps
* style: pep8
* tests(wandb): test sync_step
* docs(wandb): add references
* docs(wandb): fix typo
* feat(wandb): more explicit warning
* feat(wandb): order of args
* style: Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* style: long line
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
2021-01-24 17:44:09 -05:00
9515750548
Fix pre-commit isort failure on tests/loggers/*.py ( #5425 )
...
* Remove tests.loggers from skipped module in pyproject.toml
* Fix pre-commit isort failure on tests/loggers/*.py
2021-01-14 14:36:28 -05:00
059f4630c8
prune check on Trainer fit result ( #5453 )
...
* prune check on Trainer fit result
* flake8
* Apply suggestions from code review
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* .
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-01-11 19:36:48 -05:00
d583d56169
[tests/loggers] refactor with BoringModel ( #5440 )
...
* use BoringModel
* use BoringModel
* use BoringModel
* trigger
* limit_batches
* Apply suggestions from code review
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2021-01-10 07:30:06 -05:00
beb8cacf1c
fix formatting - flake8 + isort
2021-01-06 21:31:48 +01:00
21fd56e280
FIX-5311: Cast to string `_flatten_dict` ( #5354 )
...
* fix
* params
* add test
* add another types
* chlog
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
(cherry picked from commit 6536ea42fa
2021-01-06 15:16:46 +01:00
9610ea817b
refactor imports of logger dependencies ( #4860 )
...
* refactor imports of logger dependencies
* fix
* fix
* fix
* name
* fix
* mocks
* fix tests
* fix mlflow
* fix test tube
* fix wandb import check
* whitespace
* name
* name
* hack
* hack
* rev
* fix
* update mlflow import check
* try without installing conda dep
* .
* .
* .
* .
* .
* .
* .
* .
* .
Co-authored-by: Adrian Wälchli <adrian.waelchli@inf.unibe.ch>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
(cherry picked from commit ec0fb7a3ec
2021-01-06 15:16:06 +01:00
9cfbf8d609
Disable checkpointing, earlystopping and logging with fast_dev_run ( #5277 )
...
* Disable checkpointing, earlystopping and logger with fast_dev_run
* docs
* chlog
* disable callbacks and enable DummyLogger
* add log
* use dummy logger method
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
(cherry picked from commit f740245521
2021-01-06 12:57:24 +01:00
74d0652164
flake8 ++
2021-01-05 09:58:37 +01:00
dcd29aef06
feat(wandb): offset logging step when resuming ( #5050 )
...
* feat(wandb): offset logging step when resuming
* feat(wandb): output warnings
* fix(wandb): allow step to be None
* test(wandb): update tests
* feat(wandb): display warning only once
* style: fix PEP issues
* tests(wandb): fix tests
* tests(wandb): improve test
* style: fix whitespace
* feat: improve warning
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* feat(wandb): use variable from class instance
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* tests(wandb): check warnings
* feat(wandb): use WarningCache
* tests(wandb): fix tests
* style: fix formatting
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-01-05 09:58:37 +01:00
0f36525e8f
fix/enable - check F401 ( #5201 )
...
* refactor - check F401
* missed
* fix
2020-12-21 10:15:04 +01:00
6d2c564bc6
refactor - check F841 ( #5202 )
2020-12-21 11:10:55 +05:30
53d7c9555c
drop usage of deprecated distributed_backend ( #5009 )
...
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
2020-12-09 09:18:23 +01:00
849737e7ca
[Bug Fix] Allow logger to support indexing ( #4595 )
...
* [Bug Fix] Allow logger to support indexing
This should fix #4540
* Adding test for indexes for DummyLogger
* Apply suggestions from code review
Co-authored-by: chaton <thomas@grid.ai>
* pep8
* added test for dummyexperiment
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
2020-12-05 21:00:31 +00:00
204a0a2d03
[bugfix] Accumulated_gradient and TensoBoard ( #4738 )
...
* resolve bug
* update
* update
* modify one test
* remove paramters
* update on comments
* update changelog
* update docstring
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2020-11-25 19:44:05 +00:00
70361ebb6d
Fixed a crash bug in MLFlow logger ( #4716 )
...
* warnings.warn doesn't accept tuples, which causes "TypeError: expected string or bytes-like object" when the execution flow gets to this warning. Fixed that.
* Try adding a mock test
* Try adding a mock test
Co-authored-by: rohitgr7 <rohitgr1998@gmail.com>
Co-authored-by: chaton <thomas@grid.ai>
2020-11-24 00:50:34 -05:00
c586e5db77
feat(wandb): let wandb cli handle runs ( #4648 )
...
* feat(wandb): reinit handled by CLI
* fix: typo
* docs(wandb): improve formatting
* test(wandb): set wandb.run to None
* test(wandb): fix tests
* style: fix formatting
* docs(wandb): fix documentation
* Update code markup
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* docs(wandb): update CHANGELOG
* test(wandb): init called only when needed
* Update CHANGELOG.md
* try fix the test
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: edenlightning <66261195+edenlightning@users.noreply.github.com>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
Co-authored-by: rohitgr7 <rohitgr1998@gmail.com>
2020-11-24 01:31:28 +05:30
2d9d7e4daa
Add prefix argument in loggers ( #4557 )
...
* Add prefix parameter in loggers
* chlog
* pep
* patch test
* remove args, access via self
* try fix the test
* try fix the test
* try fix the test
* prefix test
* fix assert has calls
fix assert call
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2020-11-22 06:38:58 +01:00
cc8359be26
Proper casting for np scalars in hparams logging ( #4647 )
...
* first implementation
* add test and changelog
* Update tests/loggers/test_base.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* pep8
* rounding
* increase casting specificity to bool + number
* bugfix
* changelog formatting
* single loop
* Update CHANGELOG.md
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
2020-11-19 15:52:48 +08:00
Jirka Borovec