Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,976 Commits 59 Branches 217 Tags 359 MiB
Commit Graph

2976 Commits

Author SHA1 Message Date
William Falcon f770a91864
Update README.md 2020-07-28 11:37:48 -04:00
William Falcon bfa31e7583
Add files via upload 2020-07-28 11:36:43 -04:00
William Falcon 37d1bc6a42
Update README.md 2020-07-28 10:30:32 -04:00
William Falcon fa16624cc2
Update README.md 2020-07-28 10:28:59 -04:00
Jirka Borovec 590e7fb1fd
tests: add default_root_dir=tmpdir (#2392) 
* tests: add default_root_dir=tmpdir

* remove duplicate tmpdir args

* add missing fixture

* test requires multi gpu

* typo

* resize

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
 2020-07-28 09:47:53 -04:00
Jirka Borovec a3aebc1350
skip CircleCI config on master (#2732) 
* circleci config

* circleci config

* circleci config

* circleci config
 2020-07-28 06:34:01 -04:00
William Falcon 2e6e254c76
quick start docs (#2731) 
* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests

* added tests
 2020-07-27 23:53:19 -04:00
Jirka Borovec 0fe933e23d
fixing TPU tests (#2632) 
* init

* rename

* tpu_core_idx

* idx 8

* idxs

* @pl_multi_process_test

* assert

* assert

* deamon

* no close

* imort

* msg

* use_single_gpu

* dataset

* idx

* fix idx

* dataset

* format

* add pickable

* typo

* apex

* typo

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* docs

* typo

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* docs

* docs

* Apply suggestions from code review

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Apply suggestions from code review

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>

* docs

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
 2020-07-27 19:07:09 -04:00
Rohit Gupta 84c507c4df
Fix max_batches with fast_dev_run. (#2581) 
* Fix fast_dev_run to run for all val_dataloaders

* fast_dev_run check

* changelog

* explicit

* limit_batches with fast_dev_run in init

* add test

* whitespace and comment fix

* comment and assertion

* added tests

* Fix fast_dev_run to run for all val_dataloaders

* fast_dev_run check

* changelog

* explicit

* limit_batches with fast_dev_run in init

* add test

* whitespace and comment fix

* comment and assertion

* added tests

* added tests

* added tests

* added tests

* update rtol

* Revert "update rtol"

This reverts commit 4320329540.

* added tests

Co-authored-by: William Falcon <waf2107@columbia.edu>
 2020-07-27 17:56:55 -04:00
Adrian Wälchli 26afcaa30e
update conda packages (#2593) 2020-07-27 12:54:48 -04:00
Adrian Wälchli d03953260d
Fix weights_save_path when logger is used + simplify path handling + better docs (#2681) 
* fix weights_save path and drop ckpt_path

* add tests

* unused import

* update docs

* changelog

* pep8

* fix horovod test

* make backward compatible

* perform same test for all loggers

* fix for when logger=False and weights_save_path is set

* update changelog

* update docs

* update tests

* do not set save dir dynamically

* remove duplicate test

* remove duplicated tests

* update tests

* update tests

* remove remaining ckpt_path references

* move defaults to init as suggested by @Borda

* test deprecation
 2020-07-27 12:53:11 -04:00
zcain117 3f2c1022ab
Add a GKE cleanup workflow to run once per hour. (#2682) 
* Add a GKE cleanup workflow to run once per hour.

* Add fixes. Temp use workflow as triggered by commit so we can see that command works.

* Add back in schedule.
 2020-07-26 09:44:57 -04:00
William Falcon 54601bdf8e
DM docs (#2713) 
* updated docs
 2020-07-26 01:38:55 -04:00
William Falcon afd7f67047
cpu backend (#2712) 
* cpu backend

* cpu backend

* cpu backend
 2020-07-25 22:55:09 -04:00
William Falcon 0d96b2698a
refactor 4 (#2711) 
* refactor ddp_spawn
 2020-07-25 22:06:18 -04:00
William Falcon 44d85c1219
copyright (#2710) 2020-07-25 20:58:09 -04:00
William Falcon 4dbd761a1c
refactor 3/n (#2709) 
* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator
 2020-07-25 20:56:50 -04:00
William Falcon b34217e410
Refactor 2/n (#2708) 
* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator
 2020-07-25 17:31:34 -04:00
Rohit Gupta e9ed9b72dc
Add Rohit (rohitgr7) to core (#2706) 2020-07-25 23:10:09 +02:00
William Falcon 071e09fe38
refactor 1/n for v1.0.0 (#2704) 
* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator

* reactor into gpu accelerator
 2020-07-25 14:38:51 -04:00
William Falcon 5dc8c1d355
Update __init__.py 2020-07-25 13:04:06 -04:00
Nathan Raw 9076551aec
Enable val/test loop disabling + datamodule tests (#2692) 
* 🎨 warn instead of error out on loaders

* 🐛 test misconfiguration should still fail

* 🚧 .

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

* updated docs with new result obj

Co-authored-by: William Falcon <waf2107@columbia.edu>
 2020-07-25 12:57:40 -04:00
Peter Yu 4bf1918b3a
Add Peter Yu (yukw777) to core maintainers (#2690) 2020-07-25 17:22:08 +02:00
Rohit Gupta cb0c6ad51a
fix setup call while testing (#2624) 
* fix setup call while testing

* changelog

* drop if condition

* add test to check setup call

* flake8

* update test to check model stage

Co-authored-by: William Falcon <waf2107@columbia.edu>
 2020-07-24 23:57:31 -04:00
Rohit Gupta 8599b670c5
Fix logging interval (#2694) 2020-07-24 20:59:00 -04:00
William Falcon 4794175545
Update __init__.py 2020-07-24 15:44:06 -04:00
Jirka Borovec 0105ca33ce
fix nb tests for auto-merge (#2686) 2020-07-24 15:41:48 -04:00
Nathan Raw 1caf8beb2c
Datamodule (#2668) 
* ✨ Add copy of pl_bolts datamodule to lightning

* ✨ add datamodule to necessary init files

* 🚧 add datamodule property to LightningModule

* 🚧 .

* 🎨 Let DataModule do its own thing

* 🚧 add back setup and run both hooks implicitly

* 🚧 .

* 🐛 fix add_argparse_args

* 💄 apply black formatting and isort

* 📝 docstrings

* 📝 .

* 📝 .

* 🐛 overwrite cls prepare_data instead of instance

* 📝 .

* ✅ add some tests

* Update datamodule.py

* Update datamodule.py

* Update datamodule.py

Co-authored-by: William Falcon <waf2107@columbia.edu>
 2020-07-24 11:42:15 -04:00
Adrian Wälchli 938ec5a6c1
remove duplicate tests (#2685) 
* remove duplicate test

* remove duplicated tests
 2020-07-24 08:15:40 -04:00
Ananya Harsh Jha 6780214b27
quick-fix for --gpus flag bug (#2674) 
* quick-fix for --gpus flag bug

* warning added

* warning added

* set on_gpu using data_parallel_device_ids

* self.on_gpu repositioned

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
 2020-07-24 08:26:05 +00:00
Travis Addair 1369012bc7
Horovod: adjust base LR used by schedulers to scale with the number of workers (#2626) 
* Horovod: Adjust base LR used by schedulers to match that of the optimizer after scaling by number of workers

* Added unit test

* Removed debug statements

* Updated changelog

* Apply suggestions from code review

Co-authored-by: William Falcon <waf2107@columbia.edu>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
 2020-07-23 12:14:57 -04:00
Jeff Yang bda7cf1653
metrics: add SSIM (#2671) 
* metrics: add SSIM

* Update CHANGELOG.md

fix codefactor issue

fix doctest

fix doctest

fix test

* added test for raise Error
 2020-07-23 12:13:52 -04:00
zcain117 d0b8e850a4
integrate with CircleCI (#2486) 
* add circleCI

* wip

* CircleCI setup that worked on my private repo. Use a working pytorch-lightning commit

* Fix the orb imports

* Update circleci header comment

* Try to pull the GITHUB_REF from the CI_PULL_REQUEST

* Use null instead of space for 'sed'

* Add TODO for codecov

* Remove echo of GKE_CLUSTER since it will be redacted by CircleCI.

* Try running codecov upload.

* Try using codecov orb

* Use pip install codecov

* Use codecov orb again since it should be approved

* dockers/tpu-tests/Dockerfile

* action

* suggestions

* drop suggestion

* suggestion

Co-authored-by: Jirka <jirka@pytorchlightning.ai>
 2020-07-23 12:13:10 -04:00
Adrian Wälchli 1e68968ed7
support num_sanity_val_steps=-1 (#2246) 
* support sanity_val_step=-1

* fix list size

* simplification

* simplify

* add test for num_sanity_val_steps=-1

* update test

* update docs

* extend tests to multiple dataloaders

* changelog

* Update tests/trainer/test_trainer.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* improve test

* refactor the sanity check decision

* fix merge

* Update trainer.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
 2020-07-23 07:07:03 -04:00
William Falcon 62ce00f96c
EvalResult support for val loop (PR 3/5) (#2651) 
* add EvalResult to support to val/test loops
 2020-07-22 13:53:10 -04:00
Adrian Wälchli a3934ad04b
unambiguous (#2664) 2020-07-22 10:35:33 -04:00
Jeff Yang 0a65826462
metrics: add BLEU (#2535) 
* metrics: added bleu score and test bleu

* metrics: fixed type hints in bleu

* bleu score moved to metrics/functional/nlp.py

* refactor with torch.Tensor

* Update test_sequence.py

* refactor as Borda requests and nltk==3.2

* locked nltk==3.3

* nltk>=3.3, parametrized smooth argument for test

* fix bleu_score example

* added class BLEUScore metrics and test

* added class BLEUScore metrics and test

* update CHANGELOG

* refactor with torchtext

* torchtext changed to optional import

* fix E501 line too long

* add else: in optional import

* remove pragma: no-cover

* constants changed to CAPITALS

* remove class in tests

* List -> Sequence, conda -> pip, cast with tensor

* add torchtext in test.txt

* remove torchtext from test.txt

* bump torchtext to 0.5.0

* bump torchtext to 0.5.0

* Apply suggestions from code review

* ignore bleu score in doctest, renamed to nlp.py

* back to implementation with torch

* remove --ignore in CI test, proper reference format

* apply justus comment

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
 2020-07-22 09:58:24 -04:00
Rohit Gupta 5025be7860
Fix missing docs (#2659) 
* dataloader_idx typo

* typo

* update test_step docs

* missing optimizer_idx
 2020-07-22 09:54:14 -04:00
Rosario Scalise ee1488ea62
Update lightning_template model to save hparams. (#2665) 2020-07-22 09:52:24 -04:00
Adrian Wälchli a5538af355
fix dtype/device property not getting updated in submodules (#2657) 
* recursive dtype device apply

* simplify

* simple test

* submodule test

* rename

* explicit

* type hints

* test for dp backend

* fix test skip

* rename

* add ddp_spawn test

* fix None index in test

* try fix ddp_spawn test

* changelog

* move _dtype and _device to mixin

* additional doctest
 2020-07-21 15:18:57 -04:00
William Falcon 6d10ac2ac8
Structured results (train loop only. val loop separate PR) (PR 2/5) (#2615) 
* r

* r

* r

* patched optimizer closure with sr

* patched optimizer closure with sr

* patched optimizer closure with sr

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added autoreduce for train step

* added auto reduce on train

* added auto reduce on train

* added auto reduce on train

* added auto reduce on train

* added auto reduce on train

* added auto reduce on train

* added hooks

* added hooks

* added hooks

* added hooks

* added hooks

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* cache

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* Update pytorch_lightning/callbacks/early_stopping.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/callbacks/early_stopping.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/callbacks/early_stopping.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/callbacks/model_checkpoint.py

* Update pytorch_lightning/core/step_result.py

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* Apply suggestions from code review

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* simple

* finished tests for structured results on train epoch

* simple

* simple

* revert

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* Update tests/base/deterministic_model.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* finished tests for structured results on train epoch

* docstring typos

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* Update pytorch_lightning/core/step_result.py

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Update pytorch_lightning/overrides/data_parallel.py

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
 2020-07-20 19:00:20 -04:00
krfricke 816d8cff06
[docs] add ray tune tutorial link to sidebar and readme (#2648) 
Co-authored-by: Kai Fricke <kai@anyscale.com>
 2020-07-20 16:24:17 -04:00
Jaesun Park d14d3b2a34
Fix typo (#2646) 2020-07-20 14:13:56 -04:00
krfricke 5c346e2e86
[docs] Add link to Ray Tune in hyperparameters.rst (#2647) 
Co-authored-by: Kai Fricke <kai@anyscale.com>
 2020-07-20 13:27:16 -04:00
ananthsub ed581eb64f
Fix local rank zero casting (#2640) 
* Fix local rank zero casting

The environment variable 'LOCAL_RANK' can be a string, causing the `if rank_zero_only.rank == 0` check to fail

* Update distributed.py

address comment
 2020-07-18 20:12:06 -04:00
Jonas Haag 9759491940
Clarify limit_{train,val,test}_batches behaviour (#2630) 2020-07-17 05:54:24 -04:00
Chapman Siu 4bf1c3c0cb
Fix typo of handling (#2625) 2020-07-17 08:25:14 +02:00
Tushar Jain 7b4db3045d
Add Transformers Question Answering to community examples (#2603) 
* Update index.rst

Add QA Finetuning in community examples.

* Update README.md
 2020-07-15 11:26:30 +02:00
William Falcon aaa1553e35
tests for val loop flow (#2605) 
* add tests for single scalar return from training

* add tests for single scalar return from training

* add tests for single scalar return from training

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only

* fixing val step only
 2020-07-14 14:20:45 -04:00
Jirka Borovec 548dbd171b
add simple CircleCI (#2609) 
* add simple CircleCI

* ignore some

* note
 2020-07-14 14:04:04 -04:00