e23756b15d
CI: debug TPU failing tests ( #13679 )
...
* list pytest
* docs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* list
* test
* fix GK
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-07-15 17:40:04 -04:00
954fd7e5a3
bump base NGC image ( #13346 )
2022-07-15 21:36:19 +00:00
aa62fe36df
add testing PT 1.12 ( #13386 )
...
* add testing PT 1.12
* Fix quantization tests
* Fix another set of tests
* Fix check since https://github.com/pytorch/pytorch/pull/80139 is only going to be available for 1.13
* Skip this test for now for 1.12
Co-authored-by: SeanNaren <sean@grid.ai>
2022-07-15 19:41:23 +02:00
bb5e8be2e8
Simplify TPUSpawn rank management ( #11163 )
...
Co-authored-by: Kaushik B <kaushikbokka@gmail.com>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
2022-07-14 15:43:41 +00:00
56ff89743b
Fix TPU circleci tests ( #13432 )
...
* Fix TPU circleci tests
* Fix TPU circleci tests
* Fix TPU circleci tests
* Fix TPU circleci tests
* Fix TPU circleci tests
* Fix rank issue
* Fix rank issue
* debug alternative fix
* Revert properties
Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
2022-07-11 13:25:32 -04:00
30dce29005
fix PL release docker ( #13439 )
2022-06-29 19:36:36 +02:00
b137ef7134
CI: fix requirements freeze ( #13441 )
...
* allow freeze
* ci
* typo
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* ipu
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-06-29 09:35:57 -04:00
511f1a6515
Reroute profiler to profilers ( #12308 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
2022-06-22 20:55:39 -04:00
b08259d536
Add `XLAEnvironment` plugin ( #11330 )
...
* add xla environment class
* add api reference
* integrate
* use xenv
* remove properties
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Kaushik B <kaushikbokka@gmail.com>
Co-authored-by: Kaushik B <45285388+kaushikb11@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2022-06-22 10:57:50 +02:00
ad87d2cad0
Future 5/n: Move requirements ( #13306 )
...
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
2022-06-21 17:11:33 +02:00
388ea92386
Update HPU Dockerfile to latest version ( #13344 )
2022-06-21 17:08:44 +02:00
8ceab223c0
Fix repository links ( #13304 )
...
* GH org rename Lightning-AI
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* repo name
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-06-15 19:33:43 -04:00
ab59f308b1
Future 4/n: test & legacy in test/ folder ( #13295 )
...
* move: legacy >> test/
* move: tests >> test/
* rename unittests
* update CI
* tests4pl
* tests_pytorch
* proxi
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* ci
* link
* cli
* standalone
* fixing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* .
* Apply suggestions from code review
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* alone
* test -> tests
* Standalone fixes
* ci
* Update
* More fixes
* Fix coverage
* Fix mypy
* mypy
* Empty-Commit
* Fix
* mypy just for pl
* Fix standalone
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2022-06-15 18:10:49 -04:00
9cc714cdd1
Future 2/n: stand-alone examples ( #13294 )
...
* move: pl_examples >> src/
* convert pl_examples package to plain examples
* update CI for examples
* ci
* missing
* install
2022-06-15 08:53:51 -04:00
759e89df21
Future 1/n: package in src/ folder ( #13293 )
...
* move: pytorch_lightning >> src/
* update setup & install
* update CI
* ci
* update CI for examples
* Self review
* mypy
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* ci
* make
* docs
* typo
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* ci: gpu
* .
* hpu
* typing
* docs
* tpu
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-06-14 20:54:55 -04:00
0cf9d73d28
Drop PyTorch 1.8 support ( #13155 )
...
* Drop PyTorch 1.8 support
* Missed update
* Skip profiler test until supported
* Upgrade ipu dockerfile pytorch version
* Update XLA version
2022-06-14 20:46:44 -04:00
78ff201c7e
Update CI setup ( #13291 )
...
* drop mamba
* use legacy GPU machines
2022-06-14 17:11:54 +00:00
bfa8b7be2d
Create hpu-ci-runner Dockerfile ( #13239 )
...
* Create hpu-ci-runner Dockerfile
* Add ENTRYPOINT script 'start.sh' to hpu-ci-runner
* rename dirs
* ci
* add docker
* Fix build failure
* Fix build failure
* Fix title of nightly ci runner build
* Fix comments
* Fix comments
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
2022-06-08 16:02:16 -04:00
3c5a8a833e
Decouple pulling legacy checkpoints from existing GHA workflows and docker files ( #13185 )
...
* Add pull-legacy-checkpoints action
* Replace pulls with the new action and script
* Simplify
2022-06-02 15:39:14 +02:00
de4ab1c027
update NGC docker ( #13136 )
...
* update docker
* Apply suggestions from code review
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2022-06-02 12:54:13 +00:00
fab2ff35ad
CI: Azure - multiple configs ( #12984 )
...
* CI: Azure - multiple configs
* names
* benchmark
* Apply suggestions from code review
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2022-05-14 01:59:03 +00:00
fec9a09672
add freeze for development and full range for install ( #12994 )
...
* freeze versions
* unfreeze
* dependabot
* Apply suggestions from code review
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* fix all req
* ...
* use base
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix refs
* Apply suggestions from code review
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
* Apply suggestions from code review
* dockers
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
2022-05-12 09:14:18 -04:00
3f78c4ca7a
Track CPU stats with DeviceStatsMonitor ( #11795 )
...
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Kaushik B <kaushikbokka@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2022-05-10 10:57:38 +00:00
783ec43a85
parse strategies as own extras ( #12975 )
...
* parse strategies as own extras
* prune devel
* Update Makefile
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
* revert parse_requirements
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2022-05-09 09:25:53 -04:00
7ce948edb6
Unpin CUDA docker image for GPU CI ( #12373 )
...
* unpin CUDA docker image for GPU CI
* Apply suggestions from code review
Co-authored-by: Aki Nitta <nitta@akihironitta.com>
Co-authored-by: Akihiro Nitta <akihiro@pytorchlightning.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-05-06 02:56:57 +00:00
bb51e2a55b
Merge pull request #12723 from PyTorchLightning/req/strategies
...
Separate strategies' requirements
2022-05-04 10:06:02 -04:00
ecd135e939
Update nvidia gpg key to fix nightly docker builds ( #12930 )
...
* Update gpg key
* Use curl instead of wget
* Install key manually
2022-05-02 09:00:44 +02:00
98b206e836
Use cmake installed with apt ( #12907 )
2022-04-28 07:44:52 +00:00
ace6a5827b
Update building docker images ( #12837 )
...
Co-authored-by: Akihiro Nitta <akihiro@pytorchlightning.ai>
2022-04-21 22:10:42 +00:00
16b9580958
build more dockers & slack fails ( #12675 )
...
* build dockers
* add slack
* Apply suggestions from code review
Co-authored-by: Akihiro Nitta <nitta@akihironitta.com>
2022-04-13 17:24:08 +02:00
f9b69ce5b0
CI: check docker requires ( #12677 )
...
* check docker requires
* ci update
* bagua
* conda
* cuda
2022-04-12 00:29:54 +09:00
bd035af78a
Fix TPU CI ( #12419 )
2022-03-23 11:35:38 +05:30
fe940e195d
CI: update prune_pkgs ( #12382 )
2022-03-21 12:50:50 +00:00
1eff3b53c1
Update fairscale version ( #11567 )
...
Co-authored-by: Aki Nitta <nitta@akihironitta.com>
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-03-21 11:38:55 +00:00
efa870eebc
Docker: fix NCCL building Horovod ( #12318 )
...
* Horovod w. MPI
* nccl_built
* fix
2022-03-18 14:23:19 +00:00
7ee690758c
CI: fix running PT 1.11 ( #12304 )
...
* fix fire
* horovod
* assistant
* cmake
* u20
* cuda
* -j2
* fix mypy
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2022-03-12 09:00:20 +00:00
bc8172856f
aggregate multiple helper scripts to single CLI ( #11147 )
...
* nightly release
* min version
* fire
2022-03-11 11:13:43 +00:00
1144673cd9
CI: sanity check for req. pkgs ( #11819 )
...
* CI: sanity check for req. pkgs
* scripts
* rename
* gcsfs ?
* rich !
* install extra
* move
* set -e
Co-authored-by: Aki Nitta <nitta@akihironitta.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2022-03-11 09:20:47 +00:00
3b4061f39a
CI: enable testing for PT 1.11 ( #11792 )
...
* enable PT 1.11
* horovod
* Apply suggestions from code review
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Aki Nitta <nitta@akihironitta.com>
2022-03-10 18:38:47 +00:00
8577ef7bba
Skip horovod 0.24.0 only ( #12248 )
...
* try skip horovod 0.24.0 only
* HOROVOD_BUILD_CUDA_CC_LIST
* fix test
Co-authored-by: Aki Nitta <nitta@akihironitta.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2022-03-10 16:01:08 +00:00
a0655611de
Add bagua installation in dockerfile ( #11283 )
...
Co-authored-by: Aki Nitta <nitta@akihironitta.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
2022-02-24 15:17:31 +01:00
7bc87015ea
Unblock GPU CI ( #11934 )
...
Co-authored-by: Carlos Mocholi <carlossmocholi@gmail.com>
2022-02-16 21:15:44 +01:00
0a1b8b880d
Fix horovod installation `base-cuda` Dockerfile ( #11811 )
...
* pip install --user
* add checks
* rm unrelated comment
* consistent format
* Fail if horovod not found
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2022-02-10 16:48:33 +09:00
86b177ebe5
Fix `apex` installation path in Dockerfile ( #11596 )
...
* empty commit
* Specify apex installation target directory
* pip install --user
2022-01-27 20:14:16 -05:00
650c710efa
Rename training plugin test files & names to strategy ( #11303 )
2022-01-04 14:32:45 +01:00
3692eba807
Drop Python 3.6 support ( #11117 )
2021-12-21 17:06:15 +00:00
2a5d05b562
Fix tpu spawn plugin test ( #11131 )
2021-12-18 02:53:37 +00:00
c66cd12445
Remove partitioning of model in ZeRO 3 ( #10655 )
2021-12-17 12:36:53 +00:00
e8659bd40e
update NGC ( #10770 )
2021-11-29 14:14:37 +00:00
d2aaf6b4cc
Upgrade CI after the 1.10 release ( #10075 )
2021-11-10 17:59:10 +01:00
Jirka Borovec