45c2fcb341
Add AttributeDict container for Fabric ( #18943 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
2023-11-18 09:25:26 -05:00
6628d8a73a
docs: switch todo to comment ( #18893 )
2023-11-06 11:50:38 +01:00
f5f4d0a264
Improve map, optimize and StreamingDataset ( #18912 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: thomas <thomas@thomass-MacBook-Pro.local>
2023-11-03 11:28:55 +00:00
ad93f64d35
Bitsandbytes docs improvements ( #18903 )
2023-11-02 15:50:54 +01:00
1e68c50141
ci: restrict build docs on PR ( #18923 )
...
* ci: restrict build docs on PR
* linkcheck_timeout
2023-11-02 09:03:03 -04:00
800b87eb46
Add throughput utilities to Fabric and the Trainer ( #18848 )
2023-10-30 17:10:29 +01:00
e02bb391af
Utility to disable all instances of `PossibleUserWarning` ( #18744 )
...
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
2023-10-10 06:53:32 -04:00
74d402095e
docs: Fix display of navigation tiles in Fabric docs ( #18733 )
2023-10-06 14:36:01 +02:00
5d819c91fb
Remove `fsdp_overlap_step_with_backward` in favor of native solution ( #18726 )
2023-10-06 08:11:41 -04:00
d1f8b0f766
Bitsandbytes docs improvements ( #18681 )
...
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2023-09-30 16:19:11 +02:00
5120ad20f2
Bitsandbytes precision plugin ( #18655 )
...
Co-authored-by: awaelchli <aedu.waelchli@gmail.com>
2023-09-29 19:17:18 +02:00
e3a6ac2436
intro granularity for docs build ( #18675 )
2023-09-29 15:10:45 +02:00
358336268f
enable codespell for docs & fixing +TPU ( #18629 )
...
* precommit/codespell
* run
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* disable
* more fixing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Apply suggestions from code review
* more fixing
* json
* note
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-09-26 11:54:44 -04:00
f83ad093e5
Utility function to check shared filesystem ( #18586 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2023-09-25 15:49:52 -04:00
d579cfed57
precommit: unify formatting with prettier ( #18605 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-09-25 14:34:41 +02:00
57f5268eb3
Improve the suggested `num_workers` warning ( #18591 )
...
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2023-09-21 09:38:25 -04:00
8094855137
Avoid passing process group to enable FSDP's hybrid-shard ( #18583 )
2023-09-19 13:46:24 -04:00
c1ee22a687
Optimize import paths for optional dependencies ( #18561 )
...
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
2023-09-15 17:04:19 -04:00
bda974281d
docs: switch Fabric to lai template ( #18449 )
...
* docs: switch Fabric to lai template
* imports
2023-09-11 18:43:07 +02:00
565046f337
docs: welcome Fabric like Trainer ( #18429 )
...
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
2023-09-08 10:33:44 -04:00
756e481969
Support the TransformerEngine precision plugin with the Trainer ( #18459 )
...
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
2023-09-07 19:21:00 +02:00
d3ee410100
Add dedicated docs page for init-module ( #18416 )
2023-08-28 11:28:38 -04:00
8218097833
docs: updating logos ( #18375 )
2023-08-28 16:49:28 +02:00
f4825e5778
Extend FSDP guide with checkpointing ( #18374 )
2023-08-23 20:23:16 +02:00
547e7aa393
docs: 1/3 enable Sphinx nitpicky [fabric] ( #18069 )
...
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-08-23 10:31:20 +02:00
3f73231d5c
ci/docs: enable not fetch assets ( #18333 )
2023-08-21 20:22:21 +02:00
6df43685ee
Revamp model parallel docs (FSDP) (3/n) ( #18326 )
2023-08-17 15:30:58 -04:00
03ca31c3d3
Avoid updating the device for XLA FSDP in `Fabric.setup()` [TPU] ( #18276 )
2023-08-11 22:00:23 -04:00
c83774a109
Update docs about double precision with complex numbers ( #18269 )
2023-08-10 10:36:55 +02:00
8f29bb561b
bump sphinx to 5.3 ( #18204 )
2023-08-08 15:32:34 +02:00
3fd24f9591
Remove outdated warning about loading full-state checkpoints in FSDP ( #18208 )
2023-08-01 20:06:30 +02:00
6552d29a12
Document limitations of multi-GPU in Jupyter notebooks ( #18132 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-07-24 15:22:16 -04:00
2af425328a
Add FSDP to the docs glossary ( #18121 )
2023-07-20 09:07:37 +02:00
6ab6ab8193
Fabric FSDP documentation guide ( #18109 )
2023-07-19 18:39:07 +02:00
071f85842e
Support NVIDIA's Transformer Engine as a precision plugin ( #17597 )
2023-07-19 18:21:58 +02:00
dab373de54
Support loading a raw PyTorch state-dict checkpoint in Fabric ( #18049 )
...
Co-authored-by: Jirka Borovec <6035284+Borda@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-07-18 14:06:17 -04:00
6b52b84ef8
docs: fallback for restoring right scroll menu ( #18108 )
2023-07-18 14:06:18 +02:00
c8656f1a27
docs: `fabric_methods.rst` using deprecated parameters ( #18015 )
2023-07-07 11:32:09 +02:00
036bd74e47
Disable video controls in Fabric teaser animations ( #17984 )
2023-07-04 13:27:54 +02:00
00a0f2e8f4
docs: fetch external sources ( #17941 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-07-03 20:16:45 +02:00
58d2387e0c
Add `Fabric.save(filter=...)` ( #17845 )
...
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2023-06-20 18:18:59 +00:00
1204006637
Document tensor shape requirements in `all_gather` ( #17816 )
2023-06-15 17:30:02 +02:00
76515140d8
docs: bump sphinx ver. ( #17824 )
...
* docs: bump sphinx ver.
* myst-parser
* myst-parser
* myst-parser
* 6.2
2023-06-15 09:37:04 -04:00
9ff7d7120b
Add `rank_zero_first` utility ( #17784 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-06-12 10:32:32 +00:00
3cfacc7d3e
Document non-strict loading of checkpoints in Fabric ( #17765 )
2023-06-07 22:24:21 +00:00
24a3115995
Support empty weight initialization in `Fabric.init_module()` ( #17627 )
2023-06-07 18:33:53 +00:00
88cd100369
Address feedback for `Fabric.init_module()` (1/4) ( #17721 )
...
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-05-31 14:05:29 +00:00
e470997af9
docs: fix links in fabric ( #17611 )
2023-05-11 22:56:11 +02:00
54e8095a78
Split `init_module` into `init` + `sharded_model` ( #17488 )
2023-05-05 15:54:52 +02:00
6ec9a6bd9e
[TPU] Rename classes to use XLA instead of TPU ( #17383 )
2023-04-28 12:36:22 -04:00
Adrian Wälchli