* ref: added data connector
* ref: added data connector
* ref: added data connector
* ref: added data connector
* ref: added data connector
* ref: added data connector
* ref: added data connector
* ref: added data connector
* ref: added data connector
* ref: added data connector
* tests to ensure correct dataloading interval and sequence
* tests to ensure correct dataloading interval and sequence
* tests to ensure correct dataloading interval and sequence
* tests to ensure correct dataloading interval and sequence
* tests to ensure correct dataloading interval and sequence
* add ddp script variations
* add ddp test
* rename
* shell
* test
* test
* try call
* try without subprocess
* test
* display the error
* list all variations
* try string
* try copy env
* debug
* pythonpath
* path
* update test
* change
* simple ddp test
* replace
* remove random port
* random port
* str
* clean up
* check run spawn
* clean up
* docs
* docs
* update test
* docs
* changelog
* changelog
* fix missing return statement. Do not normalize remote paths
* Update pytorch_lightning/utilities/cloud_io.py
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Add some documentation that we now support s3 and hdfs paths
* suggestion from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
commit 29fb0506cd38a15c359e369cc8bc4435916b0c78
Author: Brendan Fahy <bmfahy@gmail.com>
Date: Sat Aug 8 19:35:30 2020 +0000
fix checking for version for docs to build
commit 467fd640db02275972c7111af031c86bb59333e9
Author: Brendan Fahy <bmfahy@gmail.com>
Date: Sat Aug 8 18:56:05 2020 +0000
remove no local test
commit a7cc9f88de00feec1a5406874d05313c42bd004c
Author: Brendan Fahy <bmfahy@gmail.com>
Date: Sat Aug 8 18:46:44 2020 +0000
fix
commit 3fdbb729da79ae9348c83410a138666bad467951
Author: Brendan Fahy <bmfahy@gmail.com>
Date: Sat Aug 8 18:23:30 2020 +0000
revert requirements
commit 9b8686bd83e2bc243cf329e26f1c667c6949cf67
Author: Brendan Fahy <bmfahy@gmail.com>
Date: Sat Aug 8 18:16:42 2020 +0000
make it a fixture
commit eec74953d24c8b25268d3b6dde3cc4affdd5cb8f
Author: Brendan Fahy <bmfahy@gmail.com>
Date: Sat Aug 8 18:01:32 2020 +0000
fix up the testing
commit 896d94a0e60083d52c81db2a036b7f1e015cad11
Author: Brendan Fahy <bmfahy@gmail.com>
Date: Sat Aug 8 17:47:28 2020 +0000
fix some tests
commit 6d22bde19767bf2b71dfd44839b01efdf6888f83
Merge: 6175d4e2 6ebe0d72
Author: Brendan Fahy <bmfahy@gmail.com>
Date: Sat Aug 8 10:20:47 2020 +0000
Merge remote-tracking branch 'origin/master' into tb_use_gfile
commit 6175d4e26b15a43c412c26d501762cd0b570616a
Author: Brendan Fahy <bmfahy@gmail.com>
Date: Fri Aug 7 10:16:36 2020 +0000
Use tensorboard.compat.gfile to support remote writing
* Test using torchtext.data.Field with include_lengths=True/False
* Fix issue that Tensors in a Batch generated by torchtext with torchtext.data.Field configured as include_lengths=True
* Add description for fix of issue #2688
* changes to accomodate CodeFactor issues
* Another attemt to make last CodeFactor issue pass (it's a false alarm)
* temporarly disable test of test_grad_tracking to check if testing will pass
* reenable test in test_grad_norm
* Update CHANGELOG.md
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Renamed get_torchtext_data_iterator to _get_torchtext_data_iterator as suggested by @borda
* Update pytorch_lightning/utilities/apply_func.py
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* adding tests more specific to batch_move_data_to_device with tochtext Batch
* added check that Tensors were moved to target device
* removed tests using RNN models to be moved into a separate PR
* fixing FLAKE8 errors that showed up after merge from master branch
modified: tests/base/datamodules.py
modified: tests/callbacks/test_model_checkpoint.py
* parameterized test to reduce code duplication
* Added check only if length tensor exist. Removed left over comments.
* rearranged device parameterization and added pytest.param
* Try to figure out why only one device is tested on Linux machines
* Testing on CPU and GPU devices (GPU test is skip if no cuda device is available.
* added test for TPU device (experimental)
* Adding test parameterization for TPU test (experimental)
* change import statement to limit what is imported for a TPU environment
* made test work with TPU
* Change to trigger CI
* Change to trigger CI
* uncommented TPU test to check CI
* reenabling TPU test
* small change to trigger CI build
* small change to trigger CI build
* small change to trigger CI build
* adding tests/utilities/test_apply_func_torchtext.py to CI TPU test
* try to make test not skipped on CI with TPU
* remove testing on TPU
* undo an accidental change to test_tpu.py (file should not have been touched)
* small change to trigger CI build
* small change to trigger CI build
* Update tests/utilities/test_apply_func_torchtext.py
* Revert to previous version
* Apply suggestions from code review
* Change to trigger CI
Co-authored-by: Thomas Schaaf <tschaaf@mmm.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Thomas Schaaf <tschaaf@cs.cmu.edu>
* Fix fast_dev_run to run for all val_dataloaders
* fast_dev_run check
* changelog
* explicit
* limit_batches with fast_dev_run in init
* add test
* whitespace and comment fix
* comment and assertion
* added tests
* Fix fast_dev_run to run for all val_dataloaders
* fast_dev_run check
* changelog
* explicit
* limit_batches with fast_dev_run in init
* add test
* whitespace and comment fix
* comment and assertion
* added tests
* added tests
* added tests
* added tests
* update rtol
* Revert "update rtol"
This reverts commit 4320329540.
* added tests
Co-authored-by: William Falcon <waf2107@columbia.edu>
* recursive dtype device apply
* simplify
* simple test
* submodule test
* rename
* explicit
* type hints
* test for dp backend
* fix test skip
* rename
* add ddp_spawn test
* fix None index in test
* try fix ddp_spawn test
* changelog
* move _dtype and _device to mixin
* additional doctest
* r
* r
* r
* patched optimizer closure with sr
* patched optimizer closure with sr
* patched optimizer closure with sr
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added train step structured result
* added autoreduce for train step
* added auto reduce on train
* added auto reduce on train
* added auto reduce on train
* added auto reduce on train
* added auto reduce on train
* added auto reduce on train
* added hooks
* added hooks
* added hooks
* added hooks
* added hooks
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* cache
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* Update pytorch_lightning/callbacks/early_stopping.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/callbacks/early_stopping.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/callbacks/early_stopping.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Update pytorch_lightning/callbacks/model_checkpoint.py
* Update pytorch_lightning/core/step_result.py
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* Apply suggestions from code review
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
* simple
* finished tests for structured results on train epoch
* simple
* simple
* revert
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* Update tests/base/deterministic_model.py
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* finished tests for structured results on train epoch
* docstring typos
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* finished tests for structured results on train epoch
* Update pytorch_lightning/core/step_result.py
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
* Update pytorch_lightning/overrides/data_parallel.py
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
* Fix local rank zero casting
The environment variable 'LOCAL_RANK' can be a string, causing the `if rank_zero_only.rank == 0` check to fail
* Update distributed.py
address comment
* no cov
* no cov
* ReduceOp
* group
* reduce_op.sum
* Update sklearns.py
* formatting
* horovod
* Apply suggestions from code review
* horovod
* horovod
* horovod
* horovod
* ci
* print
* ci
* timeout
* timeout
* time
* fix
* distributed cpu
* pipes
* time
* cpu
* spawn
* spawn
* spawn
* tp
* separate
* os
* os
* npm
* Fix load_from_checkpoint() not working with URL on Windows
* Update CHANGELOG
* Update CHANGELOG.md
Co-authored-by: Peter Yu <2057325+yukw777@users.noreply.github.com>
* Apply suggestions from code review
* fix
* fix meta tags creating empty lines
* pyright
* node
* fix httpserver address
* drop tutils.default_trainer_options
* imports
* Better fix for load_from_checkpoint() not working with absolute path on Windows (#2294)
* Fix load_from_checkpoint() not working with URL on Windows
* Update CHANGELOG
* Update CHANGELOG.md
Co-authored-by: Peter Yu <2057325+yukw777@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Peter Yu <2057325+yukw777@users.noreply.github.com>
* drop duplicate
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: airium <airium@outlook.com>
Co-authored-by: Peter Yu <2057325+yukw777@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: AIRIUM <38249940+airium@users.noreply.github.com>
* Fixed the load_from_checkpoint path detected as URL bug
* Fixed the load_from_checkpoint path detected as URL bug
* fixed Caps lock typo
* Added .absolute() to checkpoint path to force hard drive prefix in string
* drop train_percent_check
* drop train_percent_check
* drop train_percent_check
* drop train_percent_check
* drop train_percent_check
* drop train_percent_check
* drop train_percent_check
* drop train_percent_check
* drop train_percent_check
* drop train_percent_check
* drop train_percent_check
* drop train_percent_check
* drop train_percent_check
* drop train_percent_check
* drop train_percent_check
* drop train_percent_check
* drop train_percent_check
* chlog
* deprecated
* deprecated
* deprecated
* tests
* tests
* Apply suggestions from code review
* tests
* hydra support
* tests
* hydra support
* hydra support
* hydra support
* tests
* typo
* typo
* Update test_dataloaders.py
* docs
* docs
* docs
* docs
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* allow loading checkpoints from urls
* tmpdir_server fixture
* test cases for loading checkpoints from url
* dir => root_dir
* default map_location to None
* test case for resume_from_checkpoint
* changelog
* doc update
* monkeypatch TORCH_HOME to avoid caching
* Use a threading server with random ports so that it is easier to clean up
* test fixes
* pep8 fix
* ThreadingHTTPServer support in 3.6
* pep8 fix
* fix changelog
* separate tests for urls
* typo
Co-authored-by: Peter Yu <2057325+yukw777@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* refactor and added hook
variant a
variant b
add test
revert rename
add changelog
docs
* resolve merge duplication
* overridden typo
* fix test
* tpu id
* raise if TPU not available
* re-use apply_to_collection function for parsing collections
* comment
* make utility function available to user
* documentation
* move changelog entry to top
* fix tpu transfer call
* fix call
* remove hardcoded string
* improve test
* call model hook by default
* Apply suggestions from code review
* rename utility function
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
William Falcon