Rewrite test_all in python.
Bash is quite annoying to write and test.
One issue with bash is it is even worse than Python for parallelism (which may be causing #4707).
Rewrite test_all in python and optimize base-runner/Dockerfile for fast development.
Also, combine some docker layers.
* [infra] Skeleton of the changes needed for collecting DFT on the builder (#1632).
* move ENGINE_INFO to the helper as well
* make collect_dft +x
* syntax fixes
* add actual dataflow tracer script
* format
* more refactoring and cleanup
* format
* address Oliver's feedback
* format
* more fixes
* format
* do not redirect stderr to stdout
* add exit at the end of main
* address feedback from Oliver
Whether string_string_checks will be turned on or not is
being discussed in https://github.com/google/oss-fuzz/issues/542.
Until it's settled, it'd probably be better to remove the option
from ASAN_OPTIONS altogether so as not to confuse anybody.
* [infra] Use coverage utils from Chromium and other fixes (follow-up #1547).
* Sort dependencies in an alpha order plus make more readable.
* Re-order arguments passed to coverage_helper script.
* Rename REPORT_DIR variable and put summary.json into platform specific dir.
* Fix -src-root-dir value.
* [infra] Add script for downloading a batch of corpus backups (for #1547).
* Add missing wget dependency and replace a tab with spaces.
* Convert more tabs to spaces.
* [infra] Enable clange code coverage reports generation for local runs.
* Use runner image and move corpus management to the helper.py
.
* Clean up, delete unnecessary stuff, add comments.
* Run fuzz targets in parallel. Do not exit in case of an error.
* Address review feedback, except of the threading thing.
* Fix a typo.
* Use ThreadPool implementatino available in standard python2.7 package.
* Add dry run support + no corpus download option.
* Fix flags handling + add log output in case of an error.
* Append arguments for fuzz target instead of replacing them.
* Remove dry run functionality as it currently errors out after two runs.
* Fix some spacing in the code.
* Update documentation regarding new code coverage script.
* [WIP] Add post build checks for catching and reporting bad builds + test projects.
* Move all type of issues into a single project. Tested all sanitizers.
* First version of the script for performing bad build checks. Still not ready to commit.\
* Added a valid example that should pass all the checks + fixed instrumentation check.
* Use SIGTERM and the handler calling _exit(0) for the startup crash case.
* Small fixes.
* Ignore startup_crash check_for fuzzing engines other than libFuzzer.
* Use "local" for local variables in the script, fix container name.
* Add COPY step for bad_build_check into base-runner Dockerfile.
* Make bad_build_check temporary noop unless EXPERIMENTAL_BAD_BUILD_CHECK is set to 1.
* Temporary remove test binaries step.
* Remove empty line.
* Remove unnecessary env variable.
Separate test_report script runs tests and generates junit-style
reports. The script doesn't fail when fuzzer fails, but marks it as
failed instead.
Fixes#168
jonathanmetzman