Solves the CI failure in #8690, and uses the chance to add more thorough
tests for the sanitized target binary and the unsanitized target binary
for `Centipede`, which is something we thought about but did not have
the chance to implement:
1. When building sanitized binaries with `helper.py` (i.e., local or
GitHub CI): Unsanitized ones will be built automatically into the same
docker container. Now bad_build_check tests both
* a) If main fuzz targets can run with the auxiliary sanitized binaries,
and
* b) If the auxiliaries are built with the correct sanitizers.
3. When In the Trial build and production build: Two kinds of binaries
will be in separate buckets / docker containers. Now Script
bad_build_check tests either
* a) If the unsanitized binaries can run without the sanitized ones, or
* b) If the sanitized binaries are built with the correct sanitizers.
Co-authored-by: Jonathan Metzman <metzman@chromium.org>
For proper support of Bazel's runfiles tree, a shell script wrapper is
also needed for C++ targets, see
https://github.com/bazelbuild/rules_fuzzing/pull/149.
This commit allows shell script fuzz targets for all languages. This
also helps to consolidate the various fuzz target checks in OSS-Fuzz
by removing a dependence on fuzzing_language.
* retry sequentially if multiprocessing do_bad_build_check detects failures
https://github.com/google/oss-fuzz/issues/5441
The error seen in the build log is:
Whoops, the target binary crashed suddenly, before receiving any input
from the fuzzer!
suggesting that the fuzzer crashed before it got to do anything.
Debugging locally what I tend to see is that
a) in src/afl-forkserver.c afl_fsrv_start the read_s32_timed call
returns 0 and that triggers kill(fsrv->fsrv_pid, fsrv->kill_signal);
(SIGKILL)
b) read_s32_timed returns 0 because *stop_soon_p is non-zero at
restart_read:
c) *stop_soon_p becomes non-zero in handle_stop_sig of
src/afl-fuzz-init.c due to receiving SIGINT
d) that SIGINT is sent by the timeout script used in bad_build_check so
it is that "outer" timeout process which is sending SIGINT which then
triggers afl-forkserver's internal SIGKILL to kill the process
I get improved results if I retry the killed off fuzzers sequentially
* Remove unneeded semicolons to fix presubmit
Co-authored-by: Abhishek Arya <inferno@chromium.org>
compile, bad_build_check, and presubmit.py require small tweaks to
support JVM fuzz targets, most of which are similar to those required
for Python. The following additional changes are required:
* Since the Jazzer driver binary already links in libFuzzer, it should
not be built as a static library.
* It is not clear how to do architecture checks as JVM fuzz targets can
load their native dependencies dynamically at runtime. For now, the
check is disabled.
* The Jazzer binaries are moved into $OUT and need to be skipped over in
find_fuzz_targets.
1. Fix problem where permissions were being changed to root by non-root test (test was doing this by invoking test_all.py within docker).
2. Mark tests as integration tests so that cifuzz_test.py can be run in a reasonable amount of time.
3. Prevent some unittests from polluting source repo.
4. Add .venv to .gitignore
5. Rename test_test_all.py to the correctly formatted name "test_all_test.py"
Rewrite test_all in python.
Bash is quite annoying to write and test.
One issue with bash is it is even worse than Python for parallelism (which may be causing #4707).
Rewrite test_all in python and optimize base-runner/Dockerfile for fast development.
Also, combine some docker layers.
jonathanmetzman