22 lines
590 B
ReStructuredText
22 lines
590 B
ReStructuredText
|
:orphan:
|
||
|
|
|
||
|
|
.. _gpu_expert:
|
||
|
|
|
||
|
|
GPU training (Expert)
|
||
|
|
=====================
|
||
|
|
**Audience:** Experts creating new scaling techniques such as Deepspeed or FSDP
|
||
|
|
|
||
|
|
----
|
||
|
|
|
||
|
|
Lightning enables experts focused on researching new ways of optimizing distributed training/inference strategies to create new strategies and plug them into Lightning.
|
||
|
|
|
||
|
|
For example, Lightning worked closely with the Microsoft team to develop a Deepspeed integration and with the Facebook(Meta) team to develop a FSDP integration.
|
||
|
|
|
||
|
|
----
|
||
|
|
|
||
|
|
.. include:: ../advanced/strategy_registry.rst
|
||
|
|
|
||
|
|
----
|
||
|
|
|
||
|
|
.. include:: ../extensions/strategy.rst
|