* [CIFuzz] Fix diffing + Refactor
Make diffing work in two scenarios where it previously failed:
1. Commit fuzzing. In this case, we diff $COMMIT against
$COMMIT^1 because the intent here is to fuzz the commit.
2. Fuzzing PRs that aren't to master. In this case, we previously
were diffing against origin/master. Instead, diff against the local
version of the base repo. This also has the nice effect of handling
PRs that havent pulled from master recently enough.
Also do refactoring.
1. Move code that is different for differenct CI systems to continuous_integration.py
2. Change how configuration in build_fuzzers is handled.
Previously configuration was gotten in build_fuzzers_entrypoint
and passed as individual params. This made code ugly and hard to
read. Instead, move code dealing with config to it's own module
config_utils. This module implements a config class which can
be used to create objects that are passed around to code that needs
it. Making the code much easier to read.
TODO: Move run_fuzzers code to new config system.
* Get list of changed files from branch head, instead of master.
Fixes https://github.com/google/oss-fuzz/issues/5022
* Add debug with subprocess.call.
* Try again debugginig.
* Try again
* Fix works!
* Project Orbit: Fix Dockerfile
This change is necessary to reflect the recent branch renaming on
github.com/google/orbit from master to main.
* Project Orbit: Fix build.sh
The build is fixed by manually setting flags for building llvm.
* Addressed review comments
* Proof of concept for Bazel support in OSS Fuzz.
* Moved the Bazel rules project under a directory with a "-test" suffix, to better convey intent.
* Added copyright header.
* Update year in copyright note.
* Factor out the '_oss_fuzz' package suffix to bash var.
* Moving 'disabled' attribute at the end of the yaml file.
* Simplified Dockerfile due to base image update.
Building mruby is done with `make clean all -j$(nproc)`, but build fails
randomly because `clean` and `all` run in parallel, and `clean` may be
executed after `all` starts.
Therefore, change it as follows.
* Remove `clean` because it seems to be unneeded in the first place.
* Use `rake` command because mruby uses Rake for builds (`Makefile` is a
limited wrapper for calling `rake`).
* Use `-m` option to run builds in parallel.
* Remove `-j` option because build time seems to be almost the same even
if `-j` option is specified.
The previous approach of only running tests in changed directories is broken.
Tests can fail even when files outside of their directory are modified.
Also blocklist failing tests (see https://github.com/google/oss-fuzz/issues/5025) for why build tests are blocklisted.
Previously region count was used instead of region covered.
This means that unaffected fuzzers only worked when a file wasn't linked
into a fuzzer build (i.e. it was mostly broken).
Add tests to ensure this doesn't happen again.
Fixes: #5013
Also refactor.
1. Create a coverage module and move coverage functionality there.
2. Remove some overly-defensive programming. We probably aren't going to be given an invalid repo dir in coverage module.
3. Convert integration test to unittest.
4. Add helpers for: normalizing paths, getting coverage per file, determining if file is covered (which was done incorrectly before), and getting fuzzer stats dir url to make code easier to understand.
5. Add a class for getting coverage info.
6. Create an affected_fuzz_targets module and move functionality dealing with affected_fuzz_targets there.
7. Add is_fuzz_target_affected helper and log more.
8. Refer to fuzz targets as fuzz targets instead of fuzzers.
9. Move `url_join` to `utils`.
10. Move `GSUTIL_BASE_URL` to `utils`.
11. Add a util function for converting gs:// URLs into https:// urls.
12. Add a util function for removing prefixes (instead of using `.replace` which operates on the whole string not just the prefix).
13. Use more common style of mocking in unittests.
14. Delete unnecessary tests like '' when an invalid fuzzer is already tested.
15. Make constants capitalized in tests.
16. Better variable naming and consistency also reuse variables in tests.
17. Leave TODOs around code that looks suspicious.
18. Cleanup pylint directives.
19. Use single instead of double quotes.
Run lint on all python files. Linting only changed files is incorrect.
This is because an unchanged file can have a linting error from depending on a changed file.
We should do the same for tests as well. But we will skip this for now because:
1. Some tests have weird path hacks and need to be blocklisted.
2. We currently run tests using integration tests which take pretty long (~6 minutes on my desktop, probably longer on CI).
3. Even the unittests are not that fast to run (take about 30 seconds locally, 8 from CIFuzz, ~20 from build).
For now, allow all tests to be run if specified.
Also, lint all files so that presubmit passes.
Fixes https://github.com/google/oss-fuzz/issues/3320
Disable linting of dataflow_tracer.py due to https://github.com/google/oss-fuzz/pull/5024#discussion_r561310796
Florian Mayer