Build and train PyTorch models and connect them to the ML lifecycle using Lightning App templates, without handling DIY infrastructure, cost management, scaling, and other headaches.
Go to file
William Falcon 198190b71d changed read me 2019-06-28 14:14:21 -04:00
docs debugging and gpu guide 2019-06-27 14:33:19 -04:00
examples/new_project_templates changed read me 2019-06-28 13:53:00 -04:00
pytorch_lightning added val loop options 2019-06-27 13:58:13 -04:00
.gitignore updated args 2019-06-25 19:42:15 -04:00
COPYING Add src, docs and other important folders 2019-04-03 22:16:02 +05:30
MANIFEST.in Fix pip install too 2019-04-03 22:47:55 +05:30
README.md changed read me 2019-06-28 14:14:21 -04:00
mkdocs.yml added lightning model docs 2019-06-27 10:04:24 -04:00
pyproject.toml Fix pip install too 2019-04-03 22:47:55 +05:30
requirements.txt initial commit 2019-03-30 20:50:32 -04:00
setup.cfg Fix pip install too 2019-04-03 22:47:55 +05:30
setup.py release v0.11 2019-06-26 18:44:59 -04:00
update.sh beta release to pypi 2019-03-31 15:26:23 -04:00

README.md

Pytorch Lightning

The Keras for ML researchers using PyTorch. More control. Less boilerplate.

PyPI version

pip install pytorch-lightning    

Docs

View the docs here

What is it?

Keras and fast.ai are too abstract for researchers. Lightning abstracts the full training loop but gives you control in the critical points.

Why do I want to use lightning?

Because you don't want to define a training loop, validation loop, gradient clipping, checkpointing, loading, gpu training, etc... every time you start a project. Let lightning handle all of that for you! Just define your data and what happens in the training, testing and validation loop and lightning will do the rest.

To use lightning do 2 things:

  1. Define a Trainer.
  2. Define a LightningModel.

What does lightning control for me?

Everything! Except the following three things:

What happens in the training loop

# define what happens for training here
def training_step(self, data_batch, batch_nb):
    x, y = data_batch
    
    # define your own forward and loss calculation
    out = self.forward(x)
    loss = my_loss(out, y)
    return {'loss': loss} 

What happens in the validation loop

# define what happens for validation here
def validation_step(self, data_batch, batch_nb):    
    x, y = data_batch
    
    # define your own forward and loss calculation
    out = self.forward(x)
    loss = my_loss(out, y)
    return {'loss': loss} 

And what to do with the output of all validation batches

def validation_end(self, outputs):
    """
    Called at the end of validation to aggregate outputs
    :param outputs: list of individual outputs of each validation step
    :return:
    """
    val_loss_mean = 0
    val_acc_mean = 0
    for output in outputs:
        val_loss_mean += output['val_loss']
        val_acc_mean += output['val_acc']

    val_loss_mean /= len(outputs)
    val_acc_mean /= len(outputs)
    tqdm_dic = {'val_loss': val_loss_mean.item(), 'val_acc': val_acc_mean.item()}
    return tqdm_dic

Lightning gives you options to control the following:

Checkpointing

  • Model saving
  • Model loading

Computing cluster (SLURM)

  • Automatic checkpointing
  • Automatic saving, loading
  • Running grid search on a cluster
  • Walltime auto-resubmit

Debugging

Distributed training

Experiment Logging

Training loop

Validation loop

Demo

# install lightning
pip install pytorch-lightning

# clone lightning for the demo
git clone https://github.com/williamFalcon/pytorch-lightning.git
cd examples/new_project_templates/

# run demo (on cpu)
python trainer_gpu_cluster_template.py

Without changing the model AT ALL, you can run the model on a single gpu, over multiple gpus, or over multiple nodes.

# run a grid search on two gpus
python fully_featured_trainer.py --gpus "0;1"

# run single model on multiple gpus
python fully_featured_trainer.py --gpus "0;1" --interactive