Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Remove lightning_transformers from our docs (#16335)

This commit is contained in:
Carlos Mocholí 2023-01-11 15:45:09 +01:00 committed by GitHub
parent 794685493d
commit 4bc2080c71
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 0 additions and 47 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

47
docs/source-pytorch/ecosystem/transformers.rst
View File

@ -1,47 +0,0 @@
:orphan:
Lightning Transformers
======================
`Lightning Transformers <https://lightning-transformers.readthedocs.io/en/latest/>`_ offers a flexible interface for training and fine-tuning SOTA Transformer models
using the :doc:`PyTorch Lightning Trainer <../common/trainer>`.
.. code-block:: bash
pip install lightning-transformers
In Lightning Transformers, we offer the following benefits:
- Powered by `PyTorch Lightning <https://www.pytorchlightning.ai/>`_ - Accelerators, custom Callbacks, Loggers, and high performance scaling with minimal changes.
- Backed by `HuggingFace Transformers <https://huggingface.co/transformers/>`_ models and datasets, spanning multiple modalities and tasks within NLP/Audio and Vision.
- Task Abstraction for Rapid Research & Experimentation - Build your own custom transformer tasks across all modalities with little friction.
- Powerful config composition backed by `Hydra <https://hydra.cc/>`_ - simply swap out models, optimizers, schedulers task, and many more configurations without touching the code.
- Seamless Memory and Speed Optimizations - Out-of-the-box training optimizations such as `DeepSpeed ZeRO <https://pytorch-lightning.readthedocs.io/en/stable/advanced/model_parallel.html#deepspeed>`_ or `FairScale Sharded Training <https://pytorch-lightning.readthedocs.io/en/stable/advanced/model_parallel.html#fairscale-sharded-training>`_ with no code changes.
-----------------
Using Lightning-Transformers
----------------------------
Lightning Transformers has a collection of tasks for common NLP problems such as `language_modeling <https://lightning-transformers.readthedocs.io/en/latest/tasks/nlp/language_modeling.html#language-modeling>`_,
`translation <https://lightning-transformers.readthedocs.io/en/latest/tasks/nlp/translation.html#translation>`_ and more. To use, simply:
1. Pick a task to train (passed to ``train.py`` as ``task=``)
2. Pick a dataset (passed to ``train.py`` as ``dataset=``)
3. Customize the backbone, optimizer, or any component within the config
4. Add any :doc:`Lightning supported parameters and optimizations <../common/trainer>`
.. code-block:: bash
python train.py \
task=<TASK> \
dataset=<DATASET>
backbone.pretrained_model_name_or_path=<BACKBONE> # Optionally change the HF backbone
optimizer=<OPTIMIZER> # Optionally specify optimizer (Default AdamW)
trainer.<ANY_TRAINER_FLAGS> # Optionally specify Lightning trainer arguments
To learn more about Lightning Transformers, please refer to the `Lightning Transformers documentation <https://lightning-transformers.readthedocs.io/en/latest/>`_.