2020-06-12 21:39:16 +00:00
|
|
|
name: Project tests
|
Improve Actions hygiene (#5361)
👋 hello there! I'm a fellow Googler who works on projects that leverage GitHub Actions for CI/CD. Recently I noticed a large increase in our queue time, and I've tracked it down to the [limit of 180 concurrent jobs](https://docs.github.com/en/actions/reference/usage-limits-billing-and-administration) for an organization. To help be better citizens, I'm proposing changes across a few repositories that will reduce GitHub Actions hours and consumption. I hope these changes are reasonable and I'm happy to talk through them in more detail.
- Only run GitHub Actions for pushes and PRs against the main branch of the repository. If your team uses a forking model, this change will not affect you. If your team pushes branches to the repository directly, this changes actions to only run against the primary branches or if you open a Pull Request against a primary branch.
- For long-running jobs (especially tests), I added the "Cancel previous" workflow. This is very helpful to prevent a large queue backlog when you are doing rapid development and pushing multiple commits. Without this, GitHub Actions' default behavior is to run all actions on all commits.
There are other changes you could make, depending on your project (but I'm not an expert):
- If you have tests that should only run when a subset of code changes, consider gating your workflow to particular file paths. For example, we have some jobs that do Terraform linting, but [they only run when Terraform files are changed](https://github.com/google/exposure-notifications-verification-server/blob/c4f59fee71042cf668747e599e7c769fca736554/.github/workflows/terraform.yml#L3-L11).
Hopefully these changes are not too controversial and also hopefully you can see how this would reduce actions consumption to be good citizens to fellow Googlers. If you have any questions, feel free to respond here or ping me on chat. Thank you!
2021-03-11 02:14:23 +00:00
|
|
|
|
2022-03-23 20:09:50 +00:00
|
|
|
permissions:
|
|
|
|
contents: read
|
|
|
|
|
Improve Actions hygiene (#5361)
👋 hello there! I'm a fellow Googler who works on projects that leverage GitHub Actions for CI/CD. Recently I noticed a large increase in our queue time, and I've tracked it down to the [limit of 180 concurrent jobs](https://docs.github.com/en/actions/reference/usage-limits-billing-and-administration) for an organization. To help be better citizens, I'm proposing changes across a few repositories that will reduce GitHub Actions hours and consumption. I hope these changes are reasonable and I'm happy to talk through them in more detail.
- Only run GitHub Actions for pushes and PRs against the main branch of the repository. If your team uses a forking model, this change will not affect you. If your team pushes branches to the repository directly, this changes actions to only run against the primary branches or if you open a Pull Request against a primary branch.
- For long-running jobs (especially tests), I added the "Cancel previous" workflow. This is very helpful to prevent a large queue backlog when you are doing rapid development and pushing multiple commits. Without this, GitHub Actions' default behavior is to run all actions on all commits.
There are other changes you could make, depending on your project (but I'm not an expert):
- If you have tests that should only run when a subset of code changes, consider gating your workflow to particular file paths. For example, we have some jobs that do Terraform linting, but [they only run when Terraform files are changed](https://github.com/google/exposure-notifications-verification-server/blob/c4f59fee71042cf668747e599e7c769fca736554/.github/workflows/terraform.yml#L3-L11).
Hopefully these changes are not too controversial and also hopefully you can see how this would reduce actions consumption to be good citizens to fellow Googlers. If you have any questions, feel free to respond here or ping me on chat. Thank you!
2021-03-11 02:14:23 +00:00
|
|
|
on:
|
|
|
|
pull_request:
|
|
|
|
branches:
|
|
|
|
- master
|
2020-06-12 21:39:16 +00:00
|
|
|
|
|
|
|
jobs:
|
|
|
|
build:
|
|
|
|
runs-on: ubuntu-latest
|
2023-06-13 15:58:08 +00:00
|
|
|
concurrency:
|
|
|
|
group: ${{ github.workflow }}-${{ matrix.sanitizer }}-${{ matrix.engine }}-${{ matrix.architecture }}-${{ github.ref }}
|
|
|
|
cancel-in-progress: true
|
2022-03-23 20:09:50 +00:00
|
|
|
permissions:
|
|
|
|
actions: write
|
2020-06-12 21:39:16 +00:00
|
|
|
strategy:
|
|
|
|
fail-fast: false
|
|
|
|
matrix:
|
|
|
|
engine:
|
|
|
|
- libfuzzer
|
|
|
|
sanitizer:
|
|
|
|
- address
|
|
|
|
- memory
|
|
|
|
- undefined
|
2020-06-25 22:35:44 +00:00
|
|
|
- coverage
|
2020-06-12 21:39:16 +00:00
|
|
|
architecture:
|
|
|
|
- x86_64
|
|
|
|
include:
|
|
|
|
- engine: afl
|
|
|
|
sanitizer: address
|
|
|
|
architecture: x86_64
|
|
|
|
- engine: honggfuzz
|
|
|
|
sanitizer: address
|
|
|
|
architecture: x86_64
|
|
|
|
- engine: libfuzzer
|
|
|
|
sanitizer: address
|
|
|
|
architecture: i386
|
|
|
|
- engine: none
|
|
|
|
sanitizer: address
|
|
|
|
architecture: x86_64
|
2022-07-20 18:36:44 +00:00
|
|
|
- engine: wycheproof
|
|
|
|
sanitizer: none
|
|
|
|
architecture: x86_64
|
2022-09-06 02:34:58 +00:00
|
|
|
- engine: centipede
|
|
|
|
sanitizer: address
|
|
|
|
architecture: x86_64
|
2023-02-06 20:47:29 +00:00
|
|
|
- engine: centipede
|
|
|
|
sanitizer: none
|
|
|
|
architecture: x86_64
|
2020-06-12 21:39:16 +00:00
|
|
|
env:
|
|
|
|
ENGINE: ${{ matrix.engine }}
|
|
|
|
SANITIZER: ${{ matrix.sanitizer }}
|
|
|
|
ARCHITECTURE: ${{ matrix.architecture }}
|
|
|
|
|
|
|
|
steps:
|
2024-07-04 03:44:20 +00:00
|
|
|
- uses: actions/checkout@v4
|
2021-06-21 14:14:41 +00:00
|
|
|
with: # Needed for git diff to work. (get_changed_files)
|
|
|
|
fetch-depth: 0
|
|
|
|
- run: |
|
2020-06-12 21:39:16 +00:00
|
|
|
git symbolic-ref refs/remotes/origin/HEAD refs/remotes/origin/master
|
|
|
|
|
|
|
|
- name: Clear unnecessary files
|
|
|
|
run: |
|
2023-03-02 19:07:26 +00:00
|
|
|
df -h
|
2020-06-12 21:39:16 +00:00
|
|
|
sudo swapoff -a
|
|
|
|
sudo rm -f /swapfile
|
|
|
|
sudo apt clean
|
2024-12-19 09:56:15 +00:00
|
|
|
docker rmi $(docker images -a -q) || true
|
2020-06-12 21:39:16 +00:00
|
|
|
df -h
|
2023-03-02 19:07:26 +00:00
|
|
|
echo "Remove large unused folders, inspired by https://github.com/apache/flink/blame/master/tools/azure-pipelines/free_disk_space.sh"
|
|
|
|
sudo bash -c '(ionice -c 3 nice -n 19 rm -rf /usr/share/dotnet/ /usr/local/graalvm/ /usr/local/.ghcup/ /usr/local/share/powershell /usr/local/share/chromium /usr/local/lib/android /usr/local/lib/node_modules)&'
|
2020-06-12 21:39:16 +00:00
|
|
|
|
|
|
|
- name: Setup python environment
|
2024-07-04 03:44:20 +00:00
|
|
|
uses: actions/setup-python@v5
|
2020-06-12 21:39:16 +00:00
|
|
|
with:
|
2022-12-27 17:40:55 +00:00
|
|
|
python-version: 3.8
|
2023-01-30 17:09:06 +00:00
|
|
|
cache: pip
|
|
|
|
cache-dependency-path: |
|
|
|
|
infra/ci/requirements.txt
|
2020-06-12 21:39:16 +00:00
|
|
|
|
|
|
|
- name: Install dependencies
|
2020-07-15 16:46:24 +00:00
|
|
|
run: |
|
|
|
|
python -m pip install --upgrade pip
|
|
|
|
pip install -r infra/ci/requirements.txt
|
2020-06-12 21:39:16 +00:00
|
|
|
|
|
|
|
- name: Run project tests
|
|
|
|
run: python infra/ci/build.py
|