Implement filestore based on github actions' artifacts feature.
This uses the github api and the github actions API.
Also fix imports in github_actions_toolkit library that were broken by move to third_party directory.
* [CIFuzz] Clean up fuzz_target.py
1. Use CORPUS_DIR env var to set corpus so that corpus can be saved.
2. Clean up is_crash_novel.
* fix
* consistency
* improve logging messages, remove over-cautious check
* fix tests
* fix tests
* Make sure corpus path is mapped
* [CIFuzz] Allow run_fuzzers to use a preexisting corpus.
Also improve comment in cifuzz-base/Dockerfile.
* improve comment
* Fix run_fuzzers
Add --cache option to helper.py's build_image command which
uses the docker image cache when building and use this in
ci/build.py.
Change build_image_impl's argument no_cache to cache for clarity.
Eliminate default argument cache in build_fuzzers_impl since no users
use it.
* [infra][jvm] Add Jazzer UBSan support
* [java-example] Reenable and plant UB
* [docs] Mention support for Java UBSan in docs
Also adds a link to the java-example build.sh to the docs.
1. Moving the _create_config and create_config functions from build_fuzzers_test.py and run_fuzzers_test.py into test_helpers.py (which is now part of cifuzz instead of infra) and share code between them.
2. Rename artifacts_dir to crashes_dir in run_fuzzers.py. "artifacts" is ambiguous.
3. Make some small changes to pytest.ini to improve debugging.
* infra: stop patching llvm-symbolizer
to prevent it from failing to start up with
```
NEW_FUNC[1/1]: llvm-symbolizer: symbol lookup error: /tmp/not-out/instrumented_libraries/lib/x86_64-linux-gnu/libz.so.1: undefined symbol: __msan_va_arg_overflow_size_tls
==45==WARNING: external symbolizer didn't start up correctly!
```
Closes https://github.com/google/oss-fuzz/issues/5349
* infra: reformat patch_build.py with yapf
to make the presubmit check happy
* Exclude Jazzer classes from coverage reports
* Generate file-level coverage data for Java
Adds per-file coverage information to llvm-cov style *.json files for
Java targets.
This provides full CI Fuzz support for "jvm" projects.
For proper support of Bazel's runfiles tree, a shell script wrapper is
also needed for C++ targets, see
https://github.com/bazelbuild/rules_fuzzing/pull/149.
This commit allows shell script fuzz targets for all languages. This
also helps to consolidate the various fuzz target checks in OSS-Fuzz
by removing a dependence on fuzzing_language.
Exposing the mount location for local sources to a command-line
argument makes it possible to reproduce findings with local source
even for umbrella projects with WORKDIR $SRC/.
* [infra] Make `targets_list` detect JVM/Python targets
`targets_list` should not detect `jazzer_driver` and `jazzer_driver_asan` as fuzz targets, but should consider shell scripts that contain `LLVMFuzzerTestOneInput`.
@inferno-chromium This should fix https://github.com/google/oss-fuzz/pull/5770#issuecomment-841851786.
* Fix
Builds of base-runner fail as the Chromium coverage helper script has a
transitive dependency on MarkupSafe, which as of version 2.0 requires
additional python packaging tools to be installed.
This commit installs python3-setuptools and wheels, which fixes the
build.
* Filter non-fuzz target binaries in coverage script
* Filter non-fuzz target binaries in helper.py
* Fix fuzz target executable bit check in helper.py
Python and JVM fuzz target executables created via the docs template
set the exectuable via `chmod u+x` as the root user, which means that
os.access checks in infra/helper.py don't see the exectuable bit if
not run as root locally.
With this commit, the check now looks for any of the three exectuable
bits.
Coverage is collected by running the fuzz targets with the JaCoCo agent
in Jazzer's no instrumentation mode.
Since JaCoCo does not support llvm-cov style coverage reports, a simple
Python helper creates it using the information contained in the JaCoCo
XML report.
As the Java build process does not maintain a mapping between source
files and build artifacts and JaCoCo needs to be passed the root folder
of the package tree, we use the Maven directory layout convention to
heuristically detect these roots.
Fix behavior of build_image --pull (Fixes#5640)
Also refactor helper.py:
1. Change behavior of functions so that most return True on success and False on failure.
2. Only main will return 1 on failure and 0 on success now. Previous behavior was very error prone.
3. Rename _get_output_dir to _get_out_dir.
4. Make function docstrings use descriptive tense.
5. Make helper.py print help when no argument is specified.
* build_image with pull should pull all images.
right now, build_image with pull only pulled the build image,
and not the runner image. this led to
https://github.com/google/oss-fuzz/issues/5480
so, pull all images to match pull_images functionality.
* address review comment.
* Adds swift-protobuf project
* Support swift language
* Fuzz target improvements
* No test for fuzz package
* fixup comments
* rename to binary
* symbolizer
jonathanmetzman