Rooba
/
lightning
mirror of https://github.com/Lightning-AI/lightning.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

ci: add description how to clean machines (#18553) 

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
This commit is contained in:
Jirka Borovec 2023-09-14 23:57:47 +02:00 committed by GitHub
parent 13760cdee7
commit db37355d31
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 24 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

24
.azure/README.md
View File

@ -44,3 +44,27 @@ done
```bash
ps aux | grep start.sh
```
# Machine maintenance
Since most of our jobs/checks are running in a Docker container, the OS/machine can become polluted and fail to run with errors such as:
```
No space left on device : '/azp/agent-litGPU-21_0,1/_diag/pages/8bb191f4-a8c2-419a-8788-66e3f0522bea_1.log'
```
In such cases, you need to log in to the machine and run `docker system prune`.
## Automated ways
Let's explore adding a cron job for periodically removing all Docker caches:
1. Open your user's cron tab for editing: `crontab -e`
1. Schedule/add the command with the `--force` flag to force pruning without interactive confirmation:
```bash
# every day at 2:00 AM clean docker caches
0 2 * * * docker system prune --force
```
1. Verify the entry: `crontab -l`
Note: You may need to add yourself to the Docker group by running `sudo usermod -aG docker <your_username>` to have permission to execute this command without needing `sudo` and entering the password.