Previously region count was used instead of region covered.
This means that unaffected fuzzers only worked when a file wasn't linked
into a fuzzer build (i.e. it was mostly broken).
Add tests to ensure this doesn't happen again.
Fixes: #5013
Also refactor.
1. Create a coverage module and move coverage functionality there.
2. Remove some overly-defensive programming. We probably aren't going to be given an invalid repo dir in coverage module.
3. Convert integration test to unittest.
4. Add helpers for: normalizing paths, getting coverage per file, determining if file is covered (which was done incorrectly before), and getting fuzzer stats dir url to make code easier to understand.
5. Add a class for getting coverage info.
6. Create an affected_fuzz_targets module and move functionality dealing with affected_fuzz_targets there.
7. Add is_fuzz_target_affected helper and log more.
8. Refer to fuzz targets as fuzz targets instead of fuzzers.
9. Move `url_join` to `utils`.
10. Move `GSUTIL_BASE_URL` to `utils`.
11. Add a util function for converting gs:// URLs into https:// urls.
12. Add a util function for removing prefixes (instead of using `.replace` which operates on the whole string not just the prefix).
13. Use more common style of mocking in unittests.
14. Delete unnecessary tests like '' when an invalid fuzzer is already tested.
15. Make constants capitalized in tests.
16. Better variable naming and consistency also reuse variables in tests.
17. Leave TODOs around code that looks suspicious.
18. Cleanup pylint directives.
19. Use single instead of double quotes.
Move entrypoints for CIFuzz to cifuzz.
This allows us to reduce some complexity by getting rid of
an unnecessary copy in docker and a hack to making importing work.
1. Add a get_affected_fuzzers function.
2. Add comments explaining rationales.
3. Fix logging so that each affected fuzzer is only logged once.
4. Don't put each file affecting each fuzzer on its own line, this makes logs unreadable.
5. Improve logging wording.
1. Fix problem where permissions were being changed to root by non-root test (test was doing this by invoking test_all.py within docker).
2. Mark tests as integration tests so that cifuzz_test.py can be run in a reasonable amount of time.
3. Prevent some unittests from polluting source repo.
4. Add .venv to .gitignore
5. Rename test_test_all.py to the correctly formatted name "test_all_test.py"
Rewrite test_all in python.
Bash is quite annoying to write and test.
One issue with bash is it is even worse than Python for parallelism (which may be causing #4707).
Rewrite test_all in python and optimize base-runner/Dockerfile for fast development.
Also, combine some docker layers.
Fix some coding and style issues.
1. Treat "testcase" as one word. This is consistent with ClusterFuzz.
2. Treat "stacktrace" as one word. This is consistent with ClusterFuzz.
3. Fix spelling of comments and words in code.
4. Use descriptive style docstrings and remove unnecessary clutter
(e.g. "Test the functionality of foo_function function in bar module"
becomes "Tests foo_function."
5. Use JSON for a list needed in testing instead of pickle.
[CIFuzz] Don't assume fuzzer output is ascii
This is wrong because the output can be arbitrary. Instead change
code that deals with the output to deal with bytes. The testcase,
which is derived from the output can be decoded as UTF-8 since it
will be a unix path.
Skia has seen some unhandled connection reset errors.
When we see these errors when downloading old OSS-Fuzz build, retry downloading it, since the error is likely transient.
The issue with 55d9a81 that caused it to be reverted by 3370f8f
was probably due to the build trigger on GCB being stale. I've
updated it so this commit should stick.
Make CIFuzz building accept an env var `MANUAL_SRC_PATH` that points to a manually prepared checkout of the project-under-tests's source code. This allows projects like Skia which are not on OSS-Fuzz and/or need to make changes to the repo after checking out the right commit to use CIFuzz.
Note that for now we aren't supporting this in GitHub, so projects that need to modify the source after checkout can't use it until
we do support it.
Also, use the local copy of OSS-Fuzz when building cifuzz-base instead of cloning it from GitHub. This makes local debugging/development much easier since it allows one to use CIFuzz with local changes.
Temporary fix for sanitizer validation.
Our sanitizer validation is hacky and isn't using pyyaml to actually
parse the project.yaml file. Temporarily work around #3996 by not
validating sanitizers if no sanitizers are specified in project.yaml.
jonathanmetzman