Commit Graph

373 Commits

Author SHA1 Message Date
Jirka Borovec 937f11c05b
try fix: Docker with Conda & PT 1.8 (#5842)
* ci

* ver

* list

* pt

* nk

* ch

* 4.9
2021-02-09 08:22:35 +00:00
Jirka Borovec 4d2b13f653
fix s3 download for PT 1.8 (#5840)
* fix s3 download for PT 1.8
2021-02-06 09:15:30 +01:00
Jirka Borovec 834f4bbf8e replace upload (#5765) 2021-02-05 21:43:10 +01:00
Jirka Borovec a39b382fe1 hotfix for GHA tpu (#5762)
* -y

* t

* .

* t
2021-02-05 21:43:10 +01:00
Jirka Borovec 0038095427 add make for docs (#5685)
* add make docs

* docs
2021-02-05 21:40:40 +01:00
Sumanth Ratna 1c44f35cf3 Fix mypy 0.800 plus when prepending $PYTHONPATH to sys.path (#5698)
* Fix mypy when prepending $PYTHONPATH to sys.path

* attempt mypy fix

* Revert "attempt mypy fix"

This reverts commit fb7ed827d9.

* fix mypy

Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
2021-02-05 21:40:40 +01:00
Jirka Borovec e45fde51b6 add contrib questions (#5691)
* add contrib questions

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Apply suggestions from code review

Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: ananthsub <ananth.subramaniam@gmail.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
2021-02-05 21:40:40 +01:00
Jirka Borovec bde2e133bf CI: skip horovod in testing docs (#5702) 2021-02-05 21:40:40 +01:00
Carlos Mocholí a7d288cb31 Update CODEOWNERS (#5561)
* Update CODEOWNERS

* Update CODEOWNERS

* Apply suggestions from code review

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: edenlightning <66261195+edenlightning@users.noreply.github.com>
2021-02-05 21:40:40 +01:00
Jirka Borovec 230f2b723c update mergify - master only (#5682) 2021-02-05 21:40:40 +01:00
Jirka Borovec 6d67da2142 mergify: less updates (#5639)
* mergify: less updates

* Drop label
2021-02-04 20:55:41 +01:00
Jirka Borovec 62be060cb8 name typo (#5612) 2021-02-04 20:55:40 +01:00
Jirka Borovec 70252f1284 update mergify (#5608)
* try fix update and review

* less spam
2021-02-04 20:55:40 +01:00
Jirka Borovec 74d175bfac fix PyPI releasing (#5605) 2021-02-04 20:55:40 +01:00
chaton e8206a9295 Mnodes (#5020)
* add a multi-nodesworkflow

Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
2021-02-04 20:55:40 +01:00
Carlos Mocholí fd6b3ec349 Drop greetings comment (#5563)
Co-authored-by: chaton <thomas@grid.ai>
2021-02-03 19:41:46 +01:00
Jirka Borovec c27d4d2d9c pipeline release CI (#5494)
* pipeline release CI

* trigger

* trigger

* .

* t1

* t2

* t1

* t2
2021-02-03 19:39:28 +01:00
Jirka Borovec 03c861bdb4 fix generate checkpoint (#5489) 2021-02-03 19:39:27 +01:00
Carlos Mocholí aa03b73e60
Remove psf/black references (#5739)
* Update pyproject.toml

* Update setup.cfg

* Update test.txt

* Update CONTRIBUTING.md

* Update requirements/test.txt
2021-02-03 08:37:06 +00:00
Jirka Borovec 07f24d2438
add nvidia docker image (#5668)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2021-01-29 11:01:03 -05:00
Jirka Borovec 817a41d635
drop bots (#5679) 2021-01-28 03:04:49 -05:00
Jirka Borovec 99ea2a3b35
define Yapf config (#5591)
* define YAPF

* add check

* add check

* add temp ignore

* apply yapf

* ex
2021-01-27 21:58:33 -05:00
Jirka Borovec dee5553b2b
move to Pages dir (#4869)
* folders

* common / advanced / extensions

* paths

* flake8

* isort

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-01-26 15:07:07 -05:00
Jirka Borovec bee7d31d44 ci: update recurent events (#5480)
* ci: update recurent events

* split events

* .

* .

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
(cherry picked from commit c00d5709c4)
2021-01-26 14:29:47 +01:00
Jirka Borovec 8db2b2defd update nightly & upgrade Twine (#5458)
* update used Twine

* .

* .

* install

* install

* .

* .

* .

* .

* .

* .

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

(cherry picked from commit 9611a7f897)
2021-01-26 14:29:47 +01:00
Jirka Borovec fd49795090 GH action - auto-update PRs (#5451)
* GH action - auto-update PRs

* .

(cherry picked from commit 92bbf2fdd6)
2021-01-26 14:28:47 +01:00
Jirka Borovec 64b701f72e GH action - label conflicts (#5450)
* GH action - label conflicts

* .

* trigger

* trigger

* .

(cherry picked from commit f1e28d1e43)
2021-01-26 14:28:47 +01:00
Jirka Borovec 9dd04028d5 tests for legacy checkpoints (#5223)
* wip

* generate

* clean

* tests

* copy

* download

* download

* download

* download

* download

* download

* download

* download

* download

* download

* download

* flake8

* extend

* aws

* extension

* pull

* pull

* pull

* pull

* pull

* pull

* pull

* try

* try

* try

* got it

* Apply suggestions from code review

(cherry picked from commit 72525f0a83)
2021-01-26 14:27:56 +01:00
Arnaud Gelas fb6c33a52d Fix pre-commit trailing-whitespace and end-of-file-fixer hooks. (#5387)
(cherry picked from commit 4c6f36e6e1)
2021-01-26 14:27:56 +01:00
Jeff Yang e1a4c2e448 docker: run ci only docker related files are changed (#5203)
* only run ci on docker related files

* docker related files changed!

* install pytorch along with cudatoolkit

* build docker only on SUN

* conda exit status has been fixed

* reverts back to old conda version

* add more docker related files

* conda env update --name

* create env and install pytorch again

* create env and install pytorch again

* ${PYTORCH_CHANNEL}

* dont update pytorch with conda env update

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update dockers/base-conda/Dockerfile

* Apply suggestions from code review

* remove checks in cron job

* Apply suggestions from code review

* readd #

* readd #

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
(cherry picked from commit cc624358c8)
2021-01-26 14:27:56 +01:00
Jirka Borovec e6db5c1886
generalise setup tools (#5617)
* generalize setup tools

* drop unused

* imports

* ci
2021-01-25 23:08:13 -05:00
Jirka Borovec cb58fdeb3d
fix: freeze mypy (#5634)
* update mypy for tests

* freeze
2021-01-24 20:09:08 -05:00
Jirka Borovec 7e4d6cbe48
set minimal req. PT 1.4 (#5418)
* set minimal req. PT 1.4

* chlog
2021-01-12 19:15:35 -05:00
Carlos Mocholí 3ee3c42035 Prepare 1.1.3 release (#5365)
* Prepare 1.1.3 release

* Fix flake8 error

* suppress

* Remove 1.1.4 section

* Add missing commits to CHANGELOG

* Update PR template

* Add missing commit

* fix

* Update CHANGELOG.md

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

(cherry picked from commit 4d9db866a1)
2021-01-06 15:17:27 +01:00
Jirka Borovec 9610ea817b refactor imports of logger dependencies (#4860)
* refactor imports of logger dependencies

* fix

* fix

* fix

* name

* fix

* mocks

* fix tests

* fix mlflow

* fix test tube

* fix wandb import check

* whitespace

* name

* name

* hack

* hack

* rev

* fix

* update mlflow import check

* try without installing conda dep

* .

* .

* .

* .

* .

* .

* .

* .

* .

Co-authored-by: Adrian Wälchli <adrian.waelchli@inf.unibe.ch>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

(cherry picked from commit ec0fb7a3ec)
2021-01-06 15:16:06 +01:00
Jirka Borovec 129c1363f9 uniques docs artefact name (#5336)
(cherry picked from commit 51af3957fc)
2021-01-06 12:49:27 +01:00
Jirka Borovec 336570b2ea refactor python in GH actions (#5281)
* refactor python in GH actions

* .
* .

(cherry picked from commit ab7512d7ba)
2021-01-06 12:48:13 +01:00
Jirka Borovec f77c3d8d43 update PR template (#5206)
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
(cherry picked from commit b22b1c2df2)
2021-01-06 11:40:11 +01:00
Jirka Borovec 5e71c88096 releasing feature as nightly (#5233)
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
(cherry picked from commit c479351a93)
2021-01-06 11:40:06 +01:00
Marijan Smetko d72ba90e1e Github Actions deprecation (#5183)
* Fix deprecation call

* fix

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2021-01-05 09:58:37 +01:00
Jirka Borovec 2718d7d214
CI: fix nightly release version (#5260)
* CI fix nigtly releases

* format

* fix unrelated flake8
2020-12-24 17:18:15 +01:00
Jirka Borovec 63fb7f9510
CI: upload report only on failer (#5086)
* CI: upload report only on failer

* Apply suggestions from code review

Co-authored-by: chaton <thomas@grid.ai>

* Apply suggestions from code review

Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2020-12-12 00:17:19 +01:00
Jirka Borovec 6a99d95d60
fix ci: release (#5037) 2020-12-09 11:53:22 +01:00
Jirka Borovec e2c404bad2
CI: update badges for release (#5002)
* fix images

* not sleep

* a0

* path

* assets

* assets

* bitecode

* rls

* rls

* badges

* fix

* org

* drop

* clean

* codecov

* fix

* clean
2020-12-09 10:59:44 +01:00
Jirka Borovec 2c11d96012
replace pyright by mypy (#5021)
* drop pyright & add mypy

* detail

* name

* fix

* flake8

* ver

Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
2020-12-09 10:57:11 +08:00
William Falcon 02ac3aba05
Update CODEOWNERS 2020-12-03 17:59:14 -05:00
Jirka Borovec 90b87dd9c8
update codeowners (#4881)
* update codeowners

* install

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: chaton <thomas@grid.ai>

* .

* Update .github/CODEOWNERS

Co-authored-by: chaton <thomas@grid.ai>
2020-12-03 17:48:40 +00:00
Jirka Borovec 7b7d4bb86d
ci typo in cofig (#4954)
* ci typo

* v++

Co-authored-by: edenlightning <66261195+edenlightning@users.noreply.github.com>
2020-12-02 22:49:47 +01:00
Jirka Borovec eeae426b33
CI: skip hanging (#4943)
* CI: try increase time limit

* try min 3.8

* no ex

* CI

* dep

* test

* deps

* deps

* drop

* drop

Co-authored-by: chaton <thomas@grid.ai>
2020-12-02 16:18:14 +00:00
Jeff Yang 563f9214fa
upgrade min deps (#4934)
* upgrade min deps

* unused

* replace torchvision and torchtext

* loggers

* freeze pip

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
2020-12-01 17:19:44 +00:00
Jirka Borovec 2fe1eff85d
drop fairscale for PT <= 1.4 (#4910)
* drop fairscale for PT <= 1.4

* fix

* Add extra check to remove fairscale from minimal testing if using minimal torch version 1.3

* Update ci_test-full.yml

* Update gym to .3 to see if this fixes examples CI

* Update omegaconf to minimum for hydra v1.0

* Revert "Update gym to .3 to see if this fixes examples CI"

This reverts commit 4221d4b9

* Revert "Update omegaconf to minimum for hydra v1.0"

This reverts commit 4f579217

Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
Co-authored-by: SeanNaren <sean@grid.ai>
2020-11-30 23:19:30 +00:00
Jirka Borovec 405645daf8
formatting (#4898)
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
2020-11-30 00:57:28 -05:00
Jirka Borovec 597dfa174c
build dockers XLA 1.7 (#4891)
* build XLA 1.7

* night XLA 1.7

* rename

* use 1.7

* tpu ver
2020-11-29 15:14:19 -04:00
Jirka Borovec 500e2853f3
increase Parity threshold (#4795)
* increase Parity threshold

* typos

* increase

* increase
2020-11-20 19:58:45 +00:00
Jirka Borovec e752348e94
update stale bot (#4769)
Co-authored-by: chaton <thomas@grid.ai>
2020-11-19 21:45:44 +01:00
Jirka Borovec 5fd1afb38a
Delay PyPI releasing (#4730)
* Delay PyPI releasing

* Delay PyPI releasing

Co-authored-by: chaton <thomas@grid.ai>
2020-11-18 15:15:41 +01:00
Jirka Borovec bddc6cd77a
pytest default color (#4703)
* pytest default color

* time

Co-authored-by: chaton <thomas@grid.ai>
2020-11-18 10:53:44 +00:00
William Falcon 886702a1af
Update CODEOWNERS 2020-11-15 10:55:50 -05:00
edenlightning 1656061051
Add williamfalcon as owner for API changes (#4610)
* Add   williamfalcon as owner for API changes

* Update .github/CODEOWNERS

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* Update CODEOWNERS

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-11-15 10:53:49 -05:00
Nathan Painchaud 2d78d9b84a
CI: Added isort import check for the code on pull-request (#4242)
* added isort CI job and updated isort config

* changed CI check output from files to full diff

* added isort pre-commit hook

* Added missing first party and restricted files affected by isort

* Applied isort to root-level, docs and benchmarks

* Apply suggestions from code review

Co-authored-by: Nathan Painchaud <nathanpainchaud@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
2020-11-13 22:57:46 +01:00
Jirka Borovec bd6c413829
Conda: PT 1.8 (#3833)
* PT 1.8

* unfreeze PT

* drop nightly from full

* add PT 1.8 to workflow

* readme table

* cuda

* skip cuda

* test 1.8

* unfreeze torch vision

Co-authored-by: ydcjeff <ydcjeff@outlook.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2020-11-12 15:03:43 +01:00
chaton 35f00df176
[FEAT] Add pytest section to Contribution how to ? (#4633)
* update contributing

* formatting
2020-11-12 11:48:54 +00:00
Jeff Yang 79fc92647c
[make] Create Makefile (#4620)
* [make] Create Makefile

* exclude makefile

* contributing info

* rm .run_local_test.sh
2020-11-12 09:25:31 +00:00
Marc Ferradou bff99ee159
Small typo correction on CONTRIBUTING.md (#4625)
* Update CONTRIBUTING.md

Small typo correction.

* Update .github/CONTRIBUTING.md

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
2020-11-12 08:59:33 +01:00
Jirka Borovec abf1d4b992
fix mock pkgs in docs (#4591)
* fix mock pkgs in docs

* sphinx

* CI

Co-authored-by: chaton <thomas@grid.ai>
2020-11-10 14:57:21 +01:00
Jeff Yang f3dfb98444
[ci] tag v1.4.1 for pypa/gh-action-pypi-publish (#4548) 2020-11-06 10:48:27 +00:00
Jirka Borovec 62ea4614f3
update PR template (#4523)
* update PR template

* Update .github/PULL_REQUEST_TEMPLATE.md

Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>

* Apply suggestions from code review

Co-authored-by: edenlightning <66261195+edenlightning@users.noreply.github.com>

Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
Co-authored-by: edenlightning <66261195+edenlightning@users.noreply.github.com>
2020-11-05 22:05:27 +01:00
Jeff Yang e81707ba02
[dockers] use inline cache (#4511)
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
2020-11-04 23:08:17 +01:00
Jirka Borovec 0d1365c442
release v1 (#4516) 2020-11-04 17:06:31 +00:00
Jirka Borovec f79fb3716c
update contrib notes (#4514)
Co-authored-by: chaton <thomas@grid.ai>
2020-11-04 16:55:30 +01:00
Jirka Borovec fc78ffa622
extend release testing (#4506)
* extend release testing

* Drone

* also PR to release

* actions versions
2020-11-04 09:08:37 +00:00
Jeff Yang 1d594c5d0c
[docker] Lock cuda version (#4453)
* lock cuda version

* back to normal
2020-10-31 20:17:07 +06:30
Jeff Yang 0f584faa6b
PyTorch 1.7 Stable support (#3821)
* prepare for 1.7 support [ci skip]

* tpu [ci skip]

* test run 1.7

* all 1.7, needs to fix tests

* couple with torchvision

* windows try

* remove windows

* 1.7 is here

* on purpose fail [ci skip]

* return [ci skip]

* 1.7 docker

* back to normal [ci skip]

* change to some_val [ci skip]

* add seed [ci skip]

* 4 places [ci skip]

* fail on purpose [ci skip]

* verbose=True [ci skip]

* use filename to track

* use filename to track

* monitor epoch + changelog

* Update tests/checkpointing/test_model_checkpoint.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2020-10-30 15:42:14 +00:00
Teddy Koker 41de4538aa
Add individuals to Metrics in CODEOWNERS (#4413)
* ananyahjha93 and teddykoker to codeowners for metrics

* add Justus

Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
2020-10-28 13:25:36 -04:00
Jirka Borovec ce8abd6255
Drone: use nightly build cuda docker images (#3658)
* upgrade PT version

* update docker

* docker

* try 1.5

* badge

* fix typo: dor -> for (#3918)

* prune

* prune

* env

* echo

* try

* notes

* env

* env

* env

* notes

* docker

* prune

* maintainer

* CI

* update

* just 1.5

* CI

* CI

* CI

* CI

* CI

* CI

* CI

* CI

* CI

* CI

* CI

* docker

* CI

* CI

* CI

* CI

* CI

* CI

* CI

* CI

* CI

* push

* try

* prune

* CI

* CI

* CI

* CI

Co-authored-by: Klyukin Valeriy <mr.clyukin@gmail.com>
Co-authored-by: Jeff Yang <ydcjeff@outlook.com>
2020-10-26 10:47:09 +00:00
Jeff Yang d83c4e4d69
Cache docker builds (#3659)
* parent faa357648f
author ydcjeff <ydcjeff@outlook.com> 1601049378 +0630
committer ydcjeff <ydcjeff@outlook.com> 1601469495 +0630

cache docker builds

lock horovod at 0.19.5

done [ci skip] [CI SKIP]

use --cache-from [ci skip]

typo and horovod [ci skip]

exclude pt 1.3 py3.8 [ci skip]

conda no cache [ci skip]

fix

* revert

* align with master [ci skip]

* retry

* remove empty continuation lines

* add comment

* fix build-args
2020-10-25 18:46:10 +06:30
chaton cd0fc2eb71
add bug_report_model to bug_report (#4307)
* add bug_report_model to bug_report

* add notebook should be made public

* update
2020-10-22 16:57:52 -04:00
William Falcon b7ec056f12
Create CODEOWNERS 2020-10-21 18:46:20 -04:00
Roger Shieh a4fa7f890a
encourage draft PR submission (#4274) 2020-10-21 09:32:40 +01:00
Jirka Borovec e320adaafb
update stale bot (#4205)
* update stale bot

* Apply suggestions from code review

Co-authored-by: chaton <thomas@grid.ai>

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Apply suggestions from code review

Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2020-10-21 10:03:54 +02:00
Jirka Borovec e0e402dbe6
Docs/changelog for 1.0.3 (#4267)
* formatting

* miss

* missing & ver++

* path
2020-10-21 00:53:10 +02:00
chaton 829d90b257
activated color in all pytest runs (#4254)
* activated color in all pytest runs

* Update .drone.yml

Co-authored-by: Jeff Yang <ydcjeff@outlook.com>

Co-authored-by: Jeff Yang <ydcjeff@outlook.com>
2020-10-20 16:38:17 +02:00
Jirka Borovec 9edef4023c
prune ignore (#4240)
* prune ignore

* try drop loggers
2020-10-20 13:32:39 +01:00
Jirka Borovec f37444fa3e
CI: add flake8 (#4239) 2020-10-19 21:20:17 +01:00
Jirka Borovec 7c4f80a1af
allow codecov upload to fail (#4221) 2020-10-19 09:28:17 +02:00
Jirka Borovec d3567c33a6
move base req. to root (#4219)
* move base req. to root

* check-manifest

* check-manifest

* manifest

* req
2020-10-18 20:40:18 +02:00
Jirka Borovec 8838586aef
update PR template (#4210) 2020-10-17 22:01:49 +02:00
William Falcon 37a96e590b
Update bug_report.md 2020-10-12 07:43:20 -04:00
William Falcon e4c92838fa
Update bug_report.md 2020-10-12 07:42:21 -04:00
William Falcon 1013dcd81d
Update bug_report.md 2020-10-12 07:41:59 -04:00
William Falcon 0d6a282e93
Update bug_report.md 2020-10-12 06:42:17 -04:00
William Falcon b5eab55243
Update bug_report.md 2020-10-12 06:35:54 -04:00
William Falcon 70334f3cc8
Update bug_report.md 2020-10-12 06:35:05 -04:00
William Falcon cba553c8a1
Update bug_report.md 2020-10-12 06:32:22 -04:00
Jirka Borovec 05cb6fcc58
Update ci_dockers.yml (#3935) 2020-10-07 08:26:07 -04:00
Jirka Borovec 7f4a9b75f3
skip some docker builds (temporally pass) (#3913)
* skip some docker builds

* todos

* skip
2020-10-06 17:29:43 -04:00
Jirka Borovec 064ae53d63
nb steps in early stop (#3909)
* nb steps

* if

* skip

* rev

* seed

* seed
2020-10-06 15:20:08 -04:00
Jirka Borovec f55a9cf63a
fic CI parsing Horovod version (#3804) 2020-10-06 17:18:16 +02:00
edenlightning e6e5406511
update bug template (#3902) 2020-10-06 09:26:37 -04:00
Jeff Yang b76fc5bae5
use docker for conda CI (#3841)
* use docker in conda CI

* update env if needed

* update with pip

* remove setting pytorch
2020-10-04 13:18:20 -04:00
zcain117 0c12065efd
[TPU CI] Use timestamp+pythonVersion to form the docker image tag. (#3779)
* Use timestamp+pythonVersion to form the docker image tag.

* Remove temporary step to check new env var.
2020-10-02 16:22:47 +02:00
edenlightning ab7d9bd1a5
Add link to PL forum in GH questions template (#3708)
* Update how-to-question.md

* Update how-to-question.md

* Apply suggestions from code review

* typo

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2020-10-02 12:05:46 +02:00
Jirka Borovec 1160270882
fix path in CI for release & python version in all dockers & duplicated badges (#3765)
* typo

* path

* check

* trigger

* fix conda

* pip ver

* fix cuda

* fix XLA

* fix xla

* ci

* docker

* BIULD

* unBIULD

* update

* py 3.8

* apex

* apex
2020-10-02 05:26:21 -04:00
Jirka Borovec a5f28ced13
nightly release to tests (#3718) 2020-09-30 08:37:52 -04:00
Jirka Borovec ab508dae0c
run TPU tests with multiple versions (#3024)
* rename

* multi build

* multi build

* copy

* copy

* copy

* copy

* copy

* copy

* clean

* note

* docker

* formatting

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-09-30 08:36:02 -04:00
Jirka Borovec a94728c99b
spec Horovod version (#3661)
* spec Horovod version

* MAKEFLAGS="-j2"

* tests

* CI

* docker

* CI

* docker
2020-09-26 19:30:25 +02:00
Jeff Yang 05e5f03fd7
Enable PyTorch 1.7 in conda CI (#3541)
* enable pt 1.7

* readme

* nightly diff version testing, will delete later

* nightly diff version testing, will delete later

* back to normal [ci skip]

* use __ignored_properties__

* define __ignored_properties__ in respective modules

* change log

* formatting

Co-authored-by: Jirka Borovec <jirka@pytorchlightning.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-09-25 16:20:15 +02:00
Jirka Borovec 7fd8ac6671
update stale config (#3509)
* update stale conf

* labels
2020-09-25 16:00:51 +02:00
Jirka Borovec 0784cf3ab4
dockers nightly (#3615)
* dockers nightly

* typo

* Apply suggestions from code review

Co-authored-by: Jeff Yang <ydcjeff@outlook.com>

Co-authored-by: Jeff Yang <ydcjeff@outlook.com>
2020-09-25 15:58:01 +02:00
Jirka Borovec a25cb300d8
fix building nightly (#3642) 2020-09-25 08:15:06 -04:00
Jeff Yang a2120130ed
Lightning docker image based on base-cuda (#3637)
* use lightning CI docker

* exclude py3.8 and torch1.3

* torch 1.7

* mergify

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <jirka@pytorchlightning.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-09-24 23:14:15 +02:00
Jirka Borovec aa52c930f4
test examples (#3643)
* test examples

* testing

* testing

* typo

* req

* exception

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2020-09-24 17:33:11 +02:00
Jirka Borovec 37a59be21b
build more docker configs (#3533)
* update build cases

* list

* matrix

* matrix

* builds

* docker

* -j1

* -q

* -q

* sep

* docker

* docker

* mergify

* -j1

* -j1

* horovod

* copy
2020-09-23 01:41:35 +02:00
Jirka Borovec 0284f7ab5a
nightly releases (#3552)
* nightly

* nightly

* ls
2020-09-19 18:28:34 -04:00
Jeff Yang 8be79a9a96
stable, dev PyTorch in Dockerfile and conda gh actions (#3074)
* dockerfile and actions file

* dockerfile and actions file

* added pytorch conda cpu nightly

* added pytorch conda cpu nightly

* recopy base reqs

* gh action `include` torch nightly

* add pytorch nightly & conda gh badge

* rebase

* fix horovod

* proposal refactor

* Update .github/workflows/ci_pt-conda.yml

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update .github/workflows/ci_pt-conda.yml

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* update

* update

* fix cmd

* filled &&

* fix

* add -y

* torchvision >0.7 allowed

* explicitly install torchvision

* use HOROVOD_GPU_OPERATIONS env variable

* CI

* skip 1.7

* table

Co-authored-by: Jirka Borovec <jirka@pytorchlightning.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-09-17 20:30:39 +02:00
Jirka Borovec 7b64472ced
fix lib paths after Wandb 0.10 (#3520)
* try

* try

* drop 0.20

* drop 0.19.5

* -U

* Fixed Horovod in CI due to wandb==0.10.0 sys.path modifications (#3525)

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* format

* wb freeze

* types

Co-authored-by: Travis Addair <taddair@uber.com>
2020-09-17 08:37:49 -04:00
Jirka Borovec c64520e658
fix tensorboard version (#3132)
* tensorboard version

* WIP test tb hparams logs (#3040)

* optional

* req

* tensorboard>=2.2.0

* data

* data

* TB

Co-authored-by: Rosario Scalise <rosario@cs.washington.edu>
2020-09-15 23:48:48 +02:00
Jirka Borovec 61b31d94b4
build docs on master (#3492)
* build docs on master

* fomatting
2020-09-15 05:55:03 -04:00
Jeff Yang 951048a81e
docs: use ref for anchor links, fix a few typo (#3486) 2020-09-13 21:04:21 -04:00
Jirka Borovec cbc4f6f8a4
add CI for building dockers (#3383)
* rename

* fix badges

* add docker build

* mergify

* update

* env

* ci

* times

* CI

* name

* comment
2020-09-10 18:38:29 -04:00
Jirka Borovec cd40cb2fad
ignore types in files (#3409)
* ignore types in files

* CI timeout
2020-09-09 07:11:53 -04:00
Jirka Borovec 9f2b29a7cd
build XLA with py3.6 (#2863)
* build py3.6

* info

* conda

* update

* version

* version

* builds

* builds

* builds

* builds

* builds
2020-08-15 15:39:44 -04:00
Jirka Borovec d4491bb14a
update PR template (#2965)
* template

* typo

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2020-08-14 00:13:23 +02:00
Jirka Borovec 4354690e55
add apex test (#2921)
* add apex test

* rename

* level

* events

* wrap

* evt

* miss

* apex

* apex

* apex

* apex

* apex

* apex

* Update tests/models/test_amp.py

Co-authored-by: William Falcon <waf2107@columbia.edu>

* notes

* notes

Co-authored-by: William Falcon <waf2107@columbia.edu>
2020-08-13 10:03:13 -04:00
Nathan Raw 118bd14d16
Update CONTRIBUTING.md (#2927)
* Update CONTRIBUTING.md

* Update CONTRIBUTING.md

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-08-12 12:59:36 +02:00
zcain117 580a5bd1df
Use kubectl to get logs from TPU CI instead of gcloud logging. (#2918)
* Use kubectl to get logs from TPU CI instead of gcloud logging.

* Update Github Action to read logs from kubectl rather than gcloud logging.
2020-08-11 19:30:56 -04:00
Jirka Borovec aa6fe25436
Fix install setup - push pypi (#2872)
* fix setup install

* fix setup install

* 📝 edit docs install command

Co-authored-by: nateraw <nxr9266@g.rit.edu>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2020-08-07 19:02:38 -04:00
Jirka Borovec 91b0d46cd5
do not fails all dockers (#2861) 2020-08-07 09:10:35 -04:00
Jirka Borovec ad956b5ed9
do not fails all dockers (#2860) 2020-08-07 14:14:22 +02:00
Rosario Scalise b37c35a8ff
Update CONTRIBUTING.md (#2855)
* Update CONTRIBUTING.md

Added docker option to testing section.

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-08-07 11:53:29 +02:00
Jeff Yang 4307dd9dc6
fix PR link (#2858) 2020-08-07 09:43:38 +02:00
Jirka Borovec ea658e300c
Tests/install pkg (#2835)
* add install matrix

* nb tests

* win

* cfg

* torch

* link

* Update .github/workflows/install-pkg.yml

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* try

* try

* try

* try

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
2020-08-07 03:08:23 -04:00
edenlightning 1dc411fc53
Update contributing guide (#2830)
* update contributing.md

* Update CONTRIBUTING.md

* Update CONTRIBUTING.md

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update .github/CONTRIBUTING.md

* suggestion from code review

* minor changes

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2020-08-06 22:46:45 +02:00
Nathan Raw 9ab071588b
Setup extras (#2831)
* 🎨 use package extras

* 🎨 get extras from reqs

* 🎨 .

* 📝 docs

* 🎨 .
2020-08-06 07:12:47 -04:00
Jirka Borovec 3772601cd6
update CI testing with pip upgrade (#2380)
* try pt1.5

* cpu

* upgrade

* tpu

* user

* [blocked by #2380] freeze GPU PT 1.4 (#2780)

* freeze

* user
2020-07-31 14:50:06 -04:00
Jirka Borovec bc7a08fbe0
test dockers & add AMP in pt-1.6 (#1584)
* exist images

* names

* images

* args

* pt 1.6 dev

* circleci

* update

* refactor

* build

* fix

* MKL
2020-07-31 08:23:13 -04:00
Jirka Borovec b88fc43871
re-enable skipped tests (#2762)
* re-enable skipped

* timeout
2020-07-31 07:52:17 -04:00
Jirka Borovec fcfdb4df13
conda speedup (#2546)
* conda speedup

* cache

* add pip cache

* suggestion

* cache

* cache

* req
2020-07-31 06:31:23 -04:00
Jirka Borovec 06e8910f06
pytorch 1.6 (#2745)
* pt 1.6

* don't use the new zipfile serialization for now

* quick flake8 fixes

* remove unnecessary f

* coalesce strings

* remove comma

* remove extra commas

* Apply suggestions from code review

Co-authored-by: Peter Yu <2057325+yukw777@users.noreply.github.com>

* set _use_new_zipfile_serialization to False only for pytorch 1.6.0

* remove unnecessary comments

* flake8 fixes

* use pkg_resources instead of packaging

* readme

* format

* version

* chlog

Co-authored-by: Peter Yu <peter@asapp.com>
Co-authored-by: Peter Yu <2057325+yukw777@users.noreply.github.com>
2020-07-31 11:18:32 +02:00
Jirka Borovec bc833fbf52
Horovod & py3.8 (#2764) 2020-07-30 23:39:07 +02:00
Jirka Borovec 40337cce58
freeze PT 1.5 for Horovod issue (#2744)
* freeze pt 1.5

* torchtext

* Apply suggestions from code review

Co-authored-by: Peter Yu <2057325+yukw777@users.noreply.github.com>

* timeout

Co-authored-by: Peter Yu <2057325+yukw777@users.noreply.github.com>
2020-07-28 15:52:23 -04:00
Jirka Borovec 0fe933e23d
fixing TPU tests (#2632)
* init

* rename

* tpu_core_idx

* idx 8

* idxs

* @pl_multi_process_test

* assert

* assert

* deamon

* no close

* imort

* msg

* use_single_gpu

* dataset

* idx

* fix idx

* dataset

* format

* add pickable

* typo

* apex

* typo

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* wip

* docs

* typo

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* tests

* docs

* docs

* Apply suggestions from code review

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Apply suggestions from code review

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>

* docs

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

Co-authored-by: Ethan Harris <ewah1g13@soton.ac.uk>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
2020-07-27 19:07:09 -04:00
zcain117 d0b8e850a4
integrate with CircleCI (#2486)
* add circleCI

* wip

* CircleCI setup that worked on my private repo. Use a working pytorch-lightning commit

* Fix the orb imports

* Update circleci header comment

* Try to pull the GITHUB_REF from the CI_PULL_REQUEST

* Use null instead of space for 'sed'

* Add TODO for codecov

* Remove echo of GKE_CLUSTER since it will be redacted by CircleCI.

* Try running codecov upload.

* Try using codecov orb

* Use pip install codecov

* Use codecov orb again since it should be approved

* dockers/tpu-tests/Dockerfile

* action

* suggestions

* drop suggestion

* suggestion

Co-authored-by: Jirka <jirka@pytorchlightning.ai>
2020-07-23 12:13:10 -04:00
Rohit Gupta 5025be7860
Fix missing docs (#2659)
* dataloader_idx typo

* typo

* update test_step docs

* missing optimizer_idx
2020-07-22 09:54:14 -04:00
William Falcon 6d10ac2ac8
Structured results (train loop only. val loop separate PR) (PR 2/5) (#2615)
* r

* r

* r

* patched optimizer closure with sr

* patched optimizer closure with sr

* patched optimizer closure with sr

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added train step structured result

* added autoreduce for train step

* added auto reduce on train

* added auto reduce on train

* added auto reduce on train

* added auto reduce on train

* added auto reduce on train

* added auto reduce on train

* added hooks

* added hooks

* added hooks

* added hooks

* added hooks

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* cache

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* Update pytorch_lightning/callbacks/early_stopping.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/callbacks/early_stopping.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/callbacks/early_stopping.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update pytorch_lightning/callbacks/model_checkpoint.py

* Update pytorch_lightning/core/step_result.py

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* Apply suggestions from code review

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>

* simple

* finished tests for structured results on train epoch

* simple

* simple

* revert

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* Update tests/base/deterministic_model.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* finished tests for structured results on train epoch

* docstring typos

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* finished tests for structured results on train epoch

* Update pytorch_lightning/core/step_result.py

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

* Update pytorch_lightning/overrides/data_parallel.py

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

Co-authored-by: Jirka <jirka@pytorchlightning.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
2020-07-20 19:00:20 -04:00
Adrian Wälchli ee6f618aa4
Fix CI crash on coverage upload timeout (#2548)
* fix ci crash on codecov timeout

* Update .github/workflows/tpu-testing.yml

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
2020-07-09 07:08:23 -04:00
William Falcon 69cbb62774
Finish #2549 (#2557)
* removed spawns for test_converters and verified tests

Co-authored-by: Ananya Harsh Jha <ahj265@nyu.edu>
Co-authored-by: zcain <zcain@google.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka@pytorchlightning.ai>
2020-07-08 20:33:48 -04:00
Jirka Borovec fb85d493d0
use XLA base image for TPU testing (#2536)
* drop py3.6

* use base image

* typo

* skip extra

* drop cache
2020-07-07 07:05:17 -04:00
Jirka Borovec 977df6ed31
Docker: building XLA base image (#2494)
* refactor

* add TPU base

* wip

* builds

* typo

* extras

* simple

* unzip

* rename
2020-07-06 14:21:36 -04:00
Jirka Borovec 39a6435726
Revert "Revert "join coverage (#2460)" (#2499)" (#2500)
This reverts commit 355918af8d.
2020-07-04 11:31:12 -04:00