Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
lightning/examples/fabric/language_model
History
awaelchli ea89133c65
Rename `fabric run model` to `fabric run` (#19527) 		2024-02-27 11:36:46 -05:00
..
README.md Rename `fabric run model` to `fabric run` (#19527) 2024-02-27 11:36:46 -05:00
train.py Set the generator seed in `random_split` usages in the examples (#18651) 2023-09-27 14:09:05 +02:00

README.md

Transformers

This example contains a simple training loop for next-word prediction with a Transformer model on a subset of the WikiText2 dataset. It is a simplified version of the official PyTorch example.

Train with Fabric

# CPU
fabric run --accelerator=cpu train.py

# GPU (CUDA or M1 Mac)
fabric run --accelerator=gpu train.py

# Multiple GPUs
fabric run --accelerator=gpu --devices=4 train.py