2019-08-06 20:37:58 +00:00
< div align = "center" >
2020-03-24 18:50:35 +00:00
2019-03-31 19:32:35 +00:00
2019-08-05 20:02:48 +00:00
# PyTorch Lightning
2019-09-14 06:18:33 +00:00
**The lightweight PyTorch wrapper for ML researchers. Scale your models. Write less boilerplate.**
2019-08-05 20:02:48 +00:00
[](https://badge.fury.io/py/pytorch-lightning)
2019-08-18 23:17:25 +00:00
[](https://pepy.tech/project/pytorch-lightning)
2020-03-14 17:01:57 +00:00
[](https://codecov.io/gh/PyTorchLightning/pytorch-lightning)
2020-04-27 21:41:46 +00:00
2020-05-15 12:36:40 +00:00
[](https://pytorch-lightning.readthedocs.io/en/stable/)
2020-06-17 19:56:19 +00:00
[](https://join.slack.com/t/pytorch-lightning/shared_invite/zt-f6bl2l0l-JYMK3tbAgAmGRrlNr00f1A)
2020-01-14 12:05:26 +00:00
[](https://github.com/PytorchLightning/pytorch-lightning/blob/master/LICENSE)
2020-05-17 16:02:09 +00:00
[](https://shields.io/)
2019-10-06 16:20:13 +00:00
2020-03-27 12:36:50 +00:00
<!--
2020-06-03 16:23:14 +00:00
[](https://www.codefactor.io/repository/github/pytorchlightning/pytorch-lightning)
2019-08-15 13:54:29 +00:00
-->
2019-08-06 20:37:58 +00:00
< / div >
2019-08-05 20:02:48 +00:00
2020-07-25 15:22:08 +00:00
---
2020-06-18 21:55:35 +00:00
## Trending contributors
2020-06-29 10:34:19 +00:00
[](https://sourcerer.io/fame/williamFalcon/pytorchlightning/pytorch-lightning/links/0)
[](https://sourcerer.io/fame/williamFalcon/pytorchlightning/pytorch-lightning/links/1)
[](https://sourcerer.io/fame/williamFalcon/pytorchlightning/pytorch-lightning/links/2)
[](https://sourcerer.io/fame/williamFalcon/pytorchlightning/pytorch-lightning/links/3)
[](https://sourcerer.io/fame/williamFalcon/pytorchlightning/pytorch-lightning/links/4)
[](https://sourcerer.io/fame/williamFalcon/pytorchlightning/pytorch-lightning/links/5)
[](https://sourcerer.io/fame/williamFalcon/pytorchlightning/pytorch-lightning/links/6)
[](https://sourcerer.io/fame/williamFalcon/pytorchlightning/pytorch-lightning/links/7)
2020-06-18 21:55:35 +00:00
2020-02-14 11:49:32 +00:00
---
2020-06-18 21:54:29 +00:00
2020-02-14 11:49:32 +00:00
## Continuous Integration
< center >
2020-06-15 23:41:51 +00:00
| System / PyTorch ver. | 1.3 (min. req.) | 1.4 | 1.5 (latest) |
2020-05-22 11:14:08 +00:00
| :---: | :---: | :---: | :---: |
2020-07-01 12:00:36 +00:00
| Conda py3.7 [linux] |  |  |  |
2020-05-22 11:14:08 +00:00
| Linux py3.7 [GPU] | - | - | [](http://35.192.60.23/PyTorchLightning/pytorch-lightning) |
2020-07-04 15:31:12 +00:00
| Linux py3.7 [TPU] | - | - |  |
2020-05-22 11:14:08 +00:00
| Linux py3.6 / py3.7 / py3.8 | [](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) | - | [](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) |
2020-06-04 15:25:07 +00:00
| OSX py3.6 / py3.7 | - | [](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) | [](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) |
2020-05-22 11:14:08 +00:00
| Windows py3.6 / py3.7 / py3.8 | [](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) |[](https://github.com/PyTorchLightning/pytorch-lightning/actions?query=workflow%3A%22CI+testing%22) | - |
2020-02-14 11:49:32 +00:00
< / center >
2019-08-05 20:02:48 +00:00
Simple installation from PyPI
2019-03-31 19:32:35 +00:00
```bash
2020-03-27 12:36:50 +00:00
pip install pytorch-lightning
2019-03-31 19:32:35 +00:00
```
2019-03-31 00:50:32 +00:00
2020-06-08 11:22:54 +00:00
From Conda
```bash
conda install pytorch-lightning -c conda-forge
```
2020-03-27 12:36:50 +00:00
## Docs
- [master ](https://pytorch-lightning.readthedocs.io/en/latest )
2020-06-18 11:21:44 +00:00
- [stable ](https://pytorch-lightning.readthedocs.io/en/stable )
2020-07-10 02:00:23 +00:00
- [0.8.5 ](https://pytorch-lightning.readthedocs.io/en/0.8.5/ )
2020-07-01 11:56:10 +00:00
- [0.8.4 ](https://pytorch-lightning.readthedocs.io/en/0.8.4/ )
2020-06-29 11:21:28 +00:00
- [0.8.3 ](https://pytorch-lightning.readthedocs.io/en/0.8.3/ )
2020-06-29 10:34:19 +00:00
- [0.8.1 ](https://pytorch-lightning.readthedocs.io/en/0.8.1/ )
- [0.7.6 ](https://pytorch-lightning.readthedocs.io/en/0.7.6/ )
2019-03-31 19:39:39 +00:00
2020-05-12 11:47:57 +00:00
## Refactoring your PyTorch code + benefits + full walk-through
2020-05-15 12:36:40 +00:00
[](https://www.youtube.com/watch?v=QHww1JH7IDU)
2020-05-12 11:47:57 +00:00
2020-03-27 12:36:50 +00:00
## Demo
2020-05-12 12:46:22 +00:00
Here's a minimal example without a validation or test loop.
```python
# this is just a plain nn.Module with some structure
2020-05-12 12:46:55 +00:00
class LitClassifier(pl.LightningModule):
2020-05-12 12:46:22 +00:00
def __init__ (self):
2020-05-12 12:59:23 +00:00
super().__init__()
2020-05-12 12:46:22 +00:00
self.l1 = torch.nn.Linear(28 * 28, 10)
def forward(self, x):
return torch.relu(self.l1(x.view(x.size(0), -1)))
def training_step(self, batch, batch_nb):
x, y = batch
loss = F.cross_entropy(self(x), y)
tensorboard_logs = {'train_loss': loss}
return {'loss': loss, 'log': tensorboard_logs}
def configure_optimizers(self):
return torch.optim.Adam(self.parameters(), lr=0.02)
# train!
train_loader = DataLoader(MNIST(os.getcwd(), train=True, download=True, transform=transforms.ToTensor()), batch_size=32)
2020-05-12 12:56:32 +00:00
model = LitClassifier()
2020-07-25 15:22:08 +00:00
trainer = pl.Trainer(gpus=8, precision=16)
trainer.fit(model, train_loader)
2020-05-12 12:46:22 +00:00
```
2020-07-25 15:22:08 +00:00
Other examples:
[MNIST hello world ](https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=gEulmrbxwaYL )
[GAN ](https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=P0bSmCw57aV5 )
[BERT ](https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=7uQVI-xv9Ddj )
[DQN ](https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=NWvMLBDySQI5 )
[MNIST on TPUs ](https://colab.research.google.com/drive/1-_LKx4HwAxl5M6xPJmqAAu444LTDQoa3 )
2019-12-08 08:09:03 +00:00
2020-03-02 03:27:37 +00:00
## What is it?
2020-04-26 15:16:48 +00:00
[READ THIS QUICK START PAGE ](https://pytorch-lightning.readthedocs.io/en/stable/new-project.html )
2020-04-26 14:57:26 +00:00
Lightning is a way to organize your PyTorch code to decouple the science code from the engineering.
2020-07-25 15:22:08 +00:00
It's more of a PyTorch style-guide than a framework.
2020-03-02 03:41:39 +00:00
2020-04-03 14:04:54 +00:00
In Lightning, you organize your code into 3 distinct categories:
1. Research code (goes in the LightningModule).
2. Engineering code (you delete, and is handled by the Trainer).
3. Non-essential research code (logging, etc... this goes in Callbacks).
Here's an example of how to refactor your research code into a [LightningModule ](https://pytorch-lightning.readthedocs.io/en/latest/lightning-module.html ).
2019-12-12 18:36:17 +00:00
2020-03-06 16:35:15 +00:00
2020-03-02 03:27:37 +00:00
2020-04-03 14:04:54 +00:00
The rest of the code is automated by the [Trainer ](https://pytorch-lightning.readthedocs.io/en/latest/trainer.html )!
2020-03-06 16:35:15 +00:00
2019-12-12 19:06:20 +00:00
2020-04-03 14:04:54 +00:00
## Testing Rigour
2020-07-25 15:22:08 +00:00
All the automated code by the Trainer is [tested rigorously with every new PR ](https://github.com/PyTorchLightning/pytorch-lightning/tree/master/tests ).
2020-04-03 14:04:54 +00:00
2020-07-25 15:22:08 +00:00
For every PR we test all combinations of:
- PyTorch 1.3, 1.4, 1.5
- Python 3.6, 3.7, 3.8
- Linux, OSX, Windows
- Multiple GPUs
2020-06-09 11:43:33 +00:00
2020-07-25 15:22:08 +00:00
**How does performance compare with vanilla PyTorch?**
We have tests to ensure we get the EXACT same results in under 600 ms difference per epoch. In reality, lightning adds about a 300 ms overhead per epoch.
2020-06-09 11:43:33 +00:00
[Check out the parity tests here ](https://github.com/PyTorchLightning/pytorch-lightning/tree/master/benchmarks ).
2020-04-03 14:04:54 +00:00
Overall, Lightning guarantees rigorously tested, correct, modern best practices for the automated parts.
2020-03-06 16:35:15 +00:00
2020-03-27 12:36:50 +00:00
## How flexible is it?
As you see, you're just organizing your PyTorch code - there's no abstraction.
2020-03-06 16:35:15 +00:00
2020-05-13 16:22:12 +00:00
And for the stuff that the Trainer abstracts out, you can [override any part ](https://pytorch-lightning.readthedocs.io/en/latest/introduction_guide.html#extensibility ) you want to do things like implement your own distributed training, 16-bit precision, or even a custom backward pass.
2020-03-06 16:35:15 +00:00
2020-06-25 20:02:16 +00:00
For example, here you could do your own backward pass without worrying about GPUs, TPUs or 16-bit since we already handle it.
2020-04-03 14:04:54 +00:00
```python
2020-04-03 14:10:16 +00:00
class LitModel(LightningModule):
2020-06-25 20:02:16 +00:00
def optimizer_step(self, current_epoch, batch_idx, optimizer, optimizer_idx,
second_order_closure=None, on_tpu=False, using_native_amp=False, using_lbfgs=False):
2020-04-03 14:10:16 +00:00
optimizer.step()
2020-06-25 20:02:16 +00:00
def optimizer_zero_grad(self, current_epoch, batch_idx, optimizer, opt_idx):
2020-04-03 14:10:16 +00:00
optimizer.zero_grad()
2020-04-03 14:04:54 +00:00
```
2020-03-06 16:35:15 +00:00
For anything else you might need, we have an extensive [callback system ](https://pytorch-lightning.readthedocs.io/en/latest/introduction_guide.html#callbacks ) you can use to add arbitrary functionality not implemented by our team in the Trainer.
## Who is Lightning for?
- Professional researchers
2020-05-13 16:22:12 +00:00
- Ph.D. students
2020-03-06 16:35:15 +00:00
- Corporate production teams
If you're just getting into deep learning, we recommend you learn PyTorch first! Once you've implemented a few models, come back and use all the advanced features of Lightning :)
2019-12-12 18:35:41 +00:00
2020-03-27 12:36:50 +00:00
## What does lightning control for me?
2019-12-12 18:35:41 +00:00
2020-03-27 12:36:50 +00:00
Everything in Blue!
2020-05-13 16:22:12 +00:00
This is how lightning separates the science (red) from engineering (blue).
2019-12-12 18:35:41 +00:00
2020-03-24 18:50:35 +00:00
2019-03-31 20:50:32 +00:00
2019-11-16 16:24:16 +00:00
## How much effort is it to convert?
2020-03-06 16:35:15 +00:00
If your code is not a huge mess you should be able to organize it into a LightningModule in less than 1 hour.
If your code IS a mess, then you needed to clean up anyhow ;)
2020-07-25 15:22:08 +00:00
[Check out this step-by-step guide ](https://towardsdatascience.com/from-pytorch-to-pytorch-lightning-a-gentle-introduction-b371b7caaf09 ).
2020-05-12 11:49:17 +00:00
[Or watch this video ](https://www.youtube.com/watch?v=QHww1JH7IDU ).
2020-02-28 03:38:08 +00:00
2019-11-16 16:22:51 +00:00
2020-03-27 12:36:50 +00:00
## Starting a new project?
[Use our seed-project aimed at reproducibility! ](https://github.com/PytorchLightning/pytorch-lightning-conference-seed )
2019-06-26 23:47:31 +00:00
## Why do I want to use lightning?
2020-03-06 16:35:15 +00:00
Although your research/production project might start simple, once you add things like GPU AND TPU training, 16-bit precision, etc, you end up spending more time engineering than researching. Lightning automates AND rigorously tests those parts for you.
## Support
2020-07-25 15:22:08 +00:00
- [8 core contributors ](https://pytorch-lightning.readthedocs.io/en/latest/governance.html ) who are all a mix of professional engineers, Research Scientists, Ph.D. students from top AI labs.
2020-03-06 16:35:15 +00:00
- 100+ community contributors.
2019-07-25 14:23:51 +00:00
2020-03-06 16:35:15 +00:00
Lightning is also part of the [PyTorch ecosystem ](https://pytorch.org/ecosystem/ ) which requires projects to have solid testing, documentation and support.
2019-06-26 23:47:31 +00:00
2019-09-06 11:30:51 +00:00
---
2020-03-27 12:36:50 +00:00
## README Table of Contents
- [How do I use it ](https://github.com/PytorchLightning/pytorch-lightning#how-do-i-do-use-it )
- [What lightning automates ](https://github.com/PytorchLightning/pytorch-lightning#what-does-lightning-control-for-me )
- [Tensorboard integration ](https://github.com/PytorchLightning/pytorch-lightning#tensorboard )
- [Lightning features ](https://github.com/PytorchLightning/pytorch-lightning#lightning-automates-all-of-the-following-each-is-also-configurable )
- [Examples ](https://github.com/PytorchLightning/pytorch-lightning#examples )
2020-01-14 12:05:26 +00:00
- [Tutorials ](https://github.com/PytorchLightning/pytorch-lightning#tutorials )
2020-02-19 11:26:23 +00:00
- [Asking for help ](https://github.com/PytorchLightning/pytorch-lightning#asking-for-help )
2020-01-14 12:05:26 +00:00
- [Contributing ](https://github.com/PytorchLightning/pytorch-lightning/blob/master/.github/CONTRIBUTING.md )
2020-03-27 12:36:50 +00:00
- [Bleeding edge install ](https://github.com/PytorchLightning/pytorch-lightning#bleeding-edge )
- [Lightning Design Principles ](https://github.com/PytorchLightning/pytorch-lightning#lightning-design-principles )
2020-02-19 11:26:23 +00:00
- [Lightning team ](https://github.com/PytorchLightning/pytorch-lightning#lightning-team )
2020-03-27 12:36:50 +00:00
- [FAQ ](https://github.com/PytorchLightning/pytorch-lightning#faq )
2019-08-05 16:29:19 +00:00
2019-09-06 11:30:51 +00:00
---
2020-03-06 16:35:15 +00:00
## Realistic example
Here's how you would organize a realistic PyTorch project into Lightning.
2019-11-19 19:13:59 +00:00
2020-03-06 16:35:15 +00:00
2019-07-25 16:33:53 +00:00
2020-03-27 12:36:50 +00:00
The LightningModule defines a *system* such as seq-2-seq, GAN, etc...
It can ALSO define a simple classifier.
2020-03-06 16:35:15 +00:00
In summary, you:
1. Define a [LightningModule ](https://pytorch-lightning.rtfd.io/en/latest/lightning-module.html )
```python
class LitSystem(pl.LightningModule):
2020-03-27 12:36:50 +00:00
2019-11-09 05:59:14 +00:00
def __init__ (self):
2020-03-27 12:36:50 +00:00
super().__init__()
2019-11-09 05:59:14 +00:00
# not the best model...
self.l1 = torch.nn.Linear(28 * 28, 10)
2020-03-27 12:36:50 +00:00
2019-11-09 05:59:14 +00:00
def forward(self, x):
return torch.relu(self.l1(x.view(x.size(0), -1)))
2020-03-27 12:36:50 +00:00
2020-03-06 16:35:15 +00:00
def training_step(self, batch, batch_idx):
...
```
2019-07-27 18:28:44 +00:00
2020-03-06 16:35:15 +00:00
2. Fit it with a [Trainer ](https://pytorch-lightning.rtfd.io/en/latest/pytorch_lightning.trainer.html )
```python
from pytorch_lightning import Trainer
2019-07-25 14:11:51 +00:00
2020-03-24 18:48:31 +00:00
model = LitSystem()
2019-07-25 14:11:51 +00:00
2020-03-06 16:35:15 +00:00
# most basic trainer, uses good defaults
2020-03-27 12:36:50 +00:00
trainer = Trainer()
trainer.fit(model)
2020-03-06 16:35:15 +00:00
```
2020-03-27 12:36:50 +00:00
2020-03-06 16:35:15 +00:00
[Check out the COLAB demo here ](https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=HOk9c4_35FKg )
2020-03-27 12:36:50 +00:00
2020-03-06 16:35:15 +00:00
## What types of research works?
Anything! Remember, that this is just organized PyTorch code.
The Training step defines the core complexity found in the training loop.
2019-08-31 07:08:18 +00:00
2020-03-06 16:35:15 +00:00
#### Could be as complex as a seq2seq
2019-06-27 18:44:51 +00:00
2019-06-26 23:44:41 +00:00
```python
# define what happens for training here
2019-12-04 11:57:10 +00:00
def training_step(self, batch, batch_idx):
2019-09-25 23:05:06 +00:00
x, y = batch
2020-03-27 12:36:50 +00:00
2019-06-27 18:43:10 +00:00
# define your own forward and loss calculation
2019-06-29 22:05:17 +00:00
hidden_states = self.encoder(x)
2020-03-27 12:36:50 +00:00
2019-08-07 15:16:02 +00:00
# even as complex as a seq-2-seq + attn model
2019-06-29 22:05:17 +00:00
# (this is just a toy, non-working example to illustrate)
start_token = '< SOS > '
last_hidden = torch.zeros(...)
loss = 0
for step in range(max_seq_len):
attn_context = self.attention_nn(hidden_states, start_token)
2020-03-27 12:36:50 +00:00
pred = self.decoder(start_token, attn_context, last_hidden)
2019-06-29 22:05:17 +00:00
last_hidden = pred
pred = self.predict_nn(pred)
loss += self.loss(last_hidden, y[step])
2020-03-27 12:36:50 +00:00
2019-06-29 22:05:17 +00:00
#toy example as well
loss = loss / max_seq_len
2020-03-27 12:36:50 +00:00
return {'loss': loss}
2019-06-26 23:44:41 +00:00
```
2019-06-27 18:44:51 +00:00
2020-03-27 12:36:50 +00:00
#### Or as basic as CNN image classification
2019-06-26 23:44:41 +00:00
```python
# define what happens for validation here
2020-03-27 12:36:50 +00:00
def validation_step(self, batch, batch_idx):
2019-09-25 23:05:06 +00:00
x, y = batch
2020-03-27 12:36:50 +00:00
2019-06-29 22:05:17 +00:00
# or as basic as a CNN classification
2020-03-27 07:17:56 +00:00
out = self(x)
2019-06-27 00:00:53 +00:00
loss = my_loss(out, y)
2020-03-27 12:36:50 +00:00
return {'loss': loss}
2019-06-26 23:44:41 +00:00
```
2020-03-06 16:35:15 +00:00
And without changing a single line of code, you could run on CPUs
2020-03-27 12:36:50 +00:00
```python
2020-03-06 16:35:15 +00:00
trainer = Trainer(max_epochs=1)
```
2019-06-27 18:44:51 +00:00
2020-03-06 16:35:15 +00:00
Or GPUs
2020-03-27 12:36:50 +00:00
```python
2020-03-06 16:35:15 +00:00
# 8 GPUs
trainer = Trainer(max_epochs=1, gpus=8)
# 256 GPUs
trainer = Trainer(max_epochs=1, gpus=8, num_nodes=32)
2019-06-27 18:44:51 +00:00
```
2019-06-29 22:32:55 +00:00
2020-03-06 16:35:15 +00:00
Or TPUs
2020-03-27 12:36:50 +00:00
```python
2020-05-17 20:30:54 +00:00
# Distributes TPU core training
trainer = Trainer(tpu_cores=8)
# Single TPU core training
trainer = Trainer(tpu_cores=[1])
2020-03-06 16:35:15 +00:00
```
2019-06-29 22:28:11 +00:00
2020-03-06 16:35:15 +00:00
When you're done training, run the test accuracy
```python
trainer.test()
```
## Visualization
Lightning has out-of-the-box integration with the popular logging/visualizing frameworks
2019-06-29 22:32:55 +00:00
2020-03-27 07:32:19 +00:00
- [Tensorboard ](https://pytorch.org/docs/stable/tensorboard.html )
- [MLFlow ](https://mlflow.org/ )
- [Neptune.ai ](https://neptune.ai/ )
- [Comet.ml ](https://www.comet.ml/site/ )
- [Wandb ](https://www.wandb.com/ )
2020-07-25 15:22:08 +00:00
- ...
2019-06-29 21:57:40 +00:00
2020-03-24 18:50:35 +00:00
2019-06-26 23:44:41 +00:00
2020-03-06 16:35:15 +00:00
## Lightning automates 40+ parts of DL/ML research
- GPU training
- Distributed GPU (cluster) training
- TPU training
- EarlyStopping
- Logging/Visualizing
- Checkpointing
- Experiment management
- [Full list here ](https://pytorch-lightning.readthedocs.io/en/latest/#common-use-cases )
2019-03-31 01:47:51 +00:00
2020-03-27 12:36:50 +00:00
2020-06-08 11:26:58 +00:00
## Running speed
2020-06-09 11:43:33 +00:00
Migrating to lightning does not mean compromising on speed! You can expect an overhead of about 300 ms per epoch compared with pure PyTorch.
2020-06-08 11:26:58 +00:00
2020-03-27 12:36:50 +00:00
## Examples
Check out this awesome list of research papers and implementations done with Lightning.
2020-03-06 16:35:15 +00:00
- [Contextual Emotion Detection (DoubleDistilBert) ](https://github.com/PyTorchLightning/emotion_transformer )
- [Generative Adversarial Network ](https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=TyYOdg8g77P0 )
- [Hyperparameter optimization with Optuna ](https://github.com/optuna/optuna/blob/master/examples/pytorch_lightning_simple.py )
2020-07-20 20:24:17 +00:00
- [Hyperparameter optimization with Ray Tune ](https://docs.ray.io/en/master/tune/tutorials/tune-pytorch-lightning.html )
2020-03-06 16:35:15 +00:00
- [Image Inpainting using Partial Convolutions ](https://github.com/ryanwongsa/Image-Inpainting )
- [MNIST on TPU ](https://colab.research.google.com/drive/1-_LKx4HwAxl5M6xPJmqAAu444LTDQoa3#scrollTo=BHBz1_AnamN_ )
- [NER (transformers, TPU, huggingface) ](https://colab.research.google.com/drive/1dBN-wwYUngLYVt985wGs_OKPlK_ANB9D )
- [NeuralTexture (CVPR) ](https://github.com/PyTorchLightning/neuraltexture )
- [Recurrent Attentive Neural Process ](https://github.com/PyTorchLightning/attentive-neural-processes )
- [Siamese Nets for One-shot Image Recognition ](https://github.com/PyTorchLightning/Siamese-Neural-Networks )
- [Speech Transformers ](https://github.com/PyTorchLightning/speech-transformer-pytorch_lightning )
- [Transformers transfer learning (Huggingface) ](https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=yr7eaxkF-djf )
- [Transformers text classification ](https://github.com/ricardorei/lightning-text-classification )
- [VAE Library of over 18+ VAE flavors ](https://github.com/AntixK/PyTorch-VAE )
2020-07-15 09:26:30 +00:00
- [Transformers Question Answering (SQuAD) ](https://github.com/tshrjn/Finetune-QA/ )
2020-06-25 03:13:49 +00:00
- [Pytorch-Lightning + Microsoft NNI with Docker ](https://github.com/davinnovation/pytorch-boilerplate )
2019-06-25 22:40:34 +00:00
2020-03-27 12:36:50 +00:00
## Tutorials
2020-03-06 16:35:15 +00:00
Check out our [introduction guide ](https://pytorch-lightning.readthedocs.io/en/latest/introduction_guide.html ) to get started.
Or jump straight into [our tutorials ](https://pytorch-lightning.readthedocs.io/en/latest/#tutorials ).
2019-08-05 23:34:34 +00:00
2019-11-09 05:59:14 +00:00
---
2020-03-27 12:36:50 +00:00
## Asking for help
Welcome to the Lightning community!
2019-08-19 11:20:10 +00:00
2020-03-27 12:36:50 +00:00
If you have any questions, feel free to:
1. [read the docs ](https://pytorch-lightning.rtfd.io/en/latest/ ).
2. [Search through the issues ](https://github.com/PytorchLightning/pytorch-lightning/issues?utf8=%E2%9C%93&q=my++question ).
3. [Ask on stackoverflow ](https://stackoverflow.com/questions/ask?guided=false ) with the tag pytorch-lightning.
2020-06-17 19:56:19 +00:00
4. [Join our slack ](https://join.slack.com/t/pytorch-lightning/shared_invite/zt-f6bl2l0l-JYMK3tbAgAmGRrlNr00f1A ).
2019-08-18 23:05:13 +00:00
2020-03-27 12:36:50 +00:00
---
2020-06-18 21:54:29 +00:00
2020-03-27 12:36:50 +00:00
## FAQ
2020-07-25 15:22:08 +00:00
**How do I use Lightning for rapid research?**
2020-03-27 12:36:50 +00:00
[Here's a walk-through ](https://pytorch-lightning.readthedocs.io/en/latest/introduction_guide.html )
2019-08-16 01:21:26 +00:00
2020-07-25 15:22:08 +00:00
**Why was Lightning created?**
2019-08-05 23:34:34 +00:00
Lightning has 3 goals in mind:
2020-03-06 16:35:15 +00:00
2020-03-27 12:36:50 +00:00
1. Maximal flexibility while abstracting out the common boilerplate across research projects.
2. Reproducibility. If all projects use the LightningModule template, it will be much much easier to understand what's going on and where to look! It will also mean every implementation follows a standard format.
2020-05-13 16:22:12 +00:00
3. Democratizing PyTorch power-user features. Distributed training? 16-bit? know you need them but don't want to take the time to implement? All good... these come built into Lightning.
2019-08-05 23:34:34 +00:00
2020-07-25 15:22:08 +00:00
**How does Lightning compare with Ignite and fast.ai?**
2020-03-27 12:36:50 +00:00
[Here's a thorough comparison ](https://medium.com/@_willfalcon/pytorch-lightning-vs-pytorch-ignite-vs-fast-ai-61dc7480ad8a ).
2019-08-05 23:34:34 +00:00
2020-07-25 15:22:08 +00:00
**Is this another library I have to learn?**
2020-03-30 14:12:48 +00:00
Nope! We use pure Pytorch everywhere and don't add unnecessary abstractions!
2019-08-05 23:34:34 +00:00
2020-07-25 15:22:08 +00:00
**Are there plans to support Python 2?**
2020-03-27 12:36:50 +00:00
Nope.
2019-08-03 12:21:25 +00:00
2020-07-25 15:22:08 +00:00
**Are there plans to support virtualenv?**
Nope. Please use anaconda or miniconda.
2020-05-12 12:52:11 +00:00
```bash
conda activate my_env
pip install pytorch-lightning
```
2019-08-08 16:43:14 +00:00
2019-09-06 11:30:51 +00:00
## Custom installation
2019-08-14 11:21:45 +00:00
2019-09-06 11:30:51 +00:00
### Bleeding edge
If you can't wait for the next release, install the most up to date code with:
* using GIT (locally clone whole repo with full history)
```bash
2020-01-14 12:05:26 +00:00
pip install git+https://github.com/PytorchLightning/pytorch-lightning.git@master --upgrade
2019-09-06 11:30:51 +00:00
```
* using instant zip (last state of the repo without git history)
```bash
2020-01-14 12:05:26 +00:00
pip install https://github.com/PytorchLightning/pytorch-lightning/archive/master.zip --upgrade
2019-09-06 11:30:51 +00:00
```
2019-08-14 11:19:58 +00:00
2019-09-06 11:30:51 +00:00
### Any release installation
2019-08-08 18:27:47 +00:00
2020-01-20 19:50:31 +00:00
You can also install any past release `0.X.Y` from this repository:
2019-07-25 23:55:22 +00:00
```bash
2020-01-20 19:50:31 +00:00
pip install https://github.com/PytorchLightning/pytorch-lightning/archive/0.X.Y.zip --upgrade
2019-07-27 18:26:08 +00:00
```
2019-11-05 16:52:50 +00:00
2020-06-08 11:22:54 +00:00
## Lightning team
2020-02-19 11:26:23 +00:00
#### Leads
- William Falcon [(williamFalcon) ](https://github.com/williamFalcon ) (Lightning founder)
2020-03-27 12:42:14 +00:00
- Jirka Borovec [(Borda) ](https://github.com/Borda ) (ghost :)
2020-02-19 11:26:23 +00:00
- Ethan Harris [(ethanwharris) ](https://github.com/ethanwharris ) (Torchbearer founder)
2020-02-19 12:49:22 +00:00
- Matthew Painter [(MattPainter01) ](https://github.com/MattPainter01 ) (Torchbearer founder)
2020-03-28 20:09:00 +00:00
- Justus Schock [(justusschock) ](https://github.com/justusschock ) (Former Core Member PyTorch Ignite)
2020-02-19 11:26:23 +00:00
#### Core Maintainers
- Nick Eggert [(neggert) ](https://github.com/neggert )
- Jeff Ling [(jeffling) ](https://github.com/jeffling )
2020-03-27 12:42:14 +00:00
- Jeremy Jordan [(jeremyjordan) ](https://github.com/jeremyjordan )
2020-02-19 11:26:23 +00:00
- Tullie Murrell [(tullie) ](https://github.com/tullie )
2020-04-24 18:46:17 +00:00
- Adrian Wälchli [(awaelchli) ](https://github.com/awaelchli )
2020-05-23 15:38:37 +00:00
- Nicki Skafte [(skaftenicki) ](https://github.com/SkafteNicki )
2020-07-25 15:22:08 +00:00
- Peter Yu [(yukw777) ](https://github.com/yukw777 )
2020-07-25 21:10:09 +00:00
- Rohit Gupta [(rohitgr7) ](https://github.com/rohitgr7 )
2020-02-19 11:26:23 +00:00
2020-06-18 21:54:29 +00:00
---
2020-07-25 15:22:08 +00:00
#### Funding
2020-05-12 15:43:01 +00:00
Building open-source software with only a few part-time people is hard! We've secured funding to make sure we can
hire a full-time staff, attend conferences, and move faster through implementing features you request.
Our goal is to build an incredible research platform and a big supportive community. Many open-source projects
2020-05-12 15:43:53 +00:00
have gone on to fund operations through things like support and special help for big corporations!
2020-05-12 15:43:01 +00:00
If you are one of these corporations, please feel free to reach out to will@pytorchlightning.ai!
2020-06-08 11:22:54 +00:00
## BibTeX
2019-11-05 16:52:50 +00:00
If you want to cite the framework feel free to use this (but only if you loved it 😊):
2020-04-28 03:54:20 +00:00
2020-04-03 13:52:41 +00:00
```bibtex
2020-04-28 03:54:20 +00:00
@article {falcon2019pytorch,
title={PyTorch Lightning},
author={Falcon, WA},
2020-06-18 21:54:29 +00:00
journal={GitHub. Note: https://github.com/PyTorchLightning/pytorch-lightning Cited by},
2020-04-28 03:54:20 +00:00
volume={3},
year={2019}
2019-11-05 16:53:12 +00:00
}
```
