Make unittests take 20 seconds to run instead of 35.
Make integration tests take 50 seconds to run instead of 6 minutes.
Make CI take 6 minutes instead of 12 minutes.
1. Allow running tests in parallel. Locally this takes the time for running all tests (including integration tests) from 6 minutes to ~50 seconds. We don't do parallel by default since it doesn't really save any time unless running integration tests on my machine (probably due to overhead of starting ~70 processes). This also speeds up CI from about 12 minutes to 6 minutes (since github actions has 2 cores per machine).
2. Fix how we run tests. I'm not exactly sure why, but the method we used for discovering tests, recursing through every directory and passing to unittest caused the build/infra tests to execute twice. Fixing this makes running unittests take ~20 seconds instead of ~35.
This change also uses pytest for running tests since it's easy to use it to run tests in parallel.
This change was made possible by #5113
This will help us catch breaking changes to the base-images.
Unfortunately caching seems to fail here when I expect it to help.
For example, base-builder doesn't build from cache when I do it locally.
This means that every other image I try to build doesn't use the cache.
That means that base-clang would take forever to rebuild.
So to compromise, I don't rebuild base-clang here.
This means that this PR won't catch breaking changes to base-image or base-clang that break in base-builder.
But it will catch breaking changes to base-image that break in base-runner and it will catch breaking changes to base-runner and base-builder.
[CI] Build a canary project on infra/ changes.
Build a specific project, sckms, that does msan, ubsan, asan, i386
builds quickly, when infra/ code is changed. This can let us know
when infra/ changes break proper functioning of OSS-Fuzz.
For this to work more thoroughly we also need to rebuild images.
Coverage builds need to be special cased since they aren't specified
in sanitizers. Instead they are done for all C/C++ projects that
use libFuzzer.
Move all of this special casing to `should_build_coverage` and call that from
`should_build` so we have one place where we decide this.
Add tests as well.
Fixes: https://github.com/google/oss-fuzz/issues/4371
* [util-linux] cover mnt_table_parse_stream
Waiting for https://github.com/karelzak/util-linux/pull/1068
* temporarily point OSS-Fuzz to evverx/util-linux
* make sure it can be built with sanitizer=coverage
jonathanmetzman