Fix diffing on non-GitHub and improve config
1. Remove obsolete comments.
2. Remove unused function get_pr_ref
3. Correct comment on git_sha
4. Rename commit_sha to git_sha
5. Make base_commit, pr_ref, and base_ref non-github specific and move to
CiEnvironment.
6. Hoist get_diff_base to base class to allow diffing
7. Fix downloading coverage on non-Github.
8. Add TODO so that we don't assume github actions is run on github.com
(enterprise users)
9. Rename repo_url to git_url.
10. Add missing git_url method base class.
11. Clarify what git_url, git_sha and pr_ref are for and leave TODOs about how
we can eliminate them.
12. Fix typos.
Getting the LATEST_BUILD_WINDOW latest commits from HEAD^ is wrong. We
should use the diff base instead.
Also modify the Ci.get_diff_base methods to not include "..." in the
return value. This will be appended in get_changed_code_under_test
instead.
This also potentially fixes a behaviour mismatch with GitHub
PRs and diffing against the latest base branch rather than
git merge-base base-branch HEAD.
Also fix some logging issues:
- Don't output every single stacktrace when determining if a crash is reproducible.
It outputs too many things into the logs and makes it hard to read.
- Log OSS-Fuzz corpus path
- Fix a minor typo and an error log with deleting the xenial image.
The current way adds a lot of ordering assumptions, and doesn't fit too
well with parallel batch fuzzing either. Add a "upload-build" boolean action
input that can be added to "build_fuzzers" to upload latest builds
instead.
Builds are now uploaded by commit hash, rather than a fixed "latest" name.
ClusterFuzzLite's download_latest_build will check the last 3 commits and download the
first available build by git hash.
This should mostly complete support for affected fuzzers for external users.
This implementation is still imperfect however.
It relies on the name of the repo being the same as the name of the directory in $SRC/ that the repo is checked out in
(i.e. repo must be checked out in $SRC/$REPO_NAME).
- Add storage-repo, storage-repo-branch, and
storage-repo-branch-coverage fields to the actions, to indicate that
the Git filestore should be used.
- The CI provided filestore is still used for crashes and builds.
- Replace generic Filestore.upload_directory with typed upload methods
which matches the download methods.
- Rename upload_latest_build to upload_build to make it more generic.
- Make artifact name prefixes an implementation detail of the store.
For #6052.
Previously, the downloaded build was uploaded instead of the new build.
This regression was introduced when the big workspace change was made.
Also, add more logging.
Also rename code uses from `project_name` to `oss_fuzz_project_name`, to
be more explicit, and use it to determine whether or not we're running
an OSS-Fuzz project or not.
For external fuzzing #6051.
1. Make coverage script accept an env var that controls where the generated report is written to. Use this in generate_coverage_report.
2. Rename CoverageGetter to OSSFuzzCoverage
3. Make affected_fuzz_targets get coverage object from clusterfuzz_deployment.
4. Add stubbed FilesystemCoverage class that will be used by non-OSS-Fuzz users.
5. Make ClusterFuzzLite able to return coverage object by downloading old coverage report and returning a FilesystemCoverage object.
6. Add stub to ClusterFuzzLite for uploading coverage.
Related: #6054
Abstract away OSS-Fuzz specific bits into the OSSFuzz implementation
of the ClusterFuzzDeployment class. This will make it easier to implement
support for other deployments of ClusterFuzz (including ClusterFuzzLite).
jonathanmetzman