* [infra] Add support for dataflow builds to the helper script and build check (#1632).
* Update travis config file.
* Address self-review comments and specify dataflow sanitizer for zstd as well.
* Fix fuzzing_engines in project.yaml
* Fix bad build check for DFSan.
* Use "hasattr" in helper.py to check the sanitizer argument.
* Address more review comments.
* Remove DataFlow config from zstd.
* fix a typo
10% is an absolutely sensible default in general especially for single-purpose
libraries like json-parsers. When large "umbrella" projects (like systemd) are
fuzzed with 30 fuzzers (and counting (hopefully :-)) covering code scattered all
over their repositories it's too easy to introduce a broken fuzzer or break a couple
of fuzzers accidentally even after running `check_build`. Waiting for two to three
days for ClusterFuzz to open an issue isn't ideal from the point of view of large
open-source project maintainers (where generally contributors come and go) so one
solution would be to run something like when PRs are opened
```sh
helper.py check_build ALLOWED_BROKEN_TARGETS_PERCENTAGE=0 ...
```
and catch issues as early as possible (and fix them while the context isn't
completely faded away).
I also considered changing this with `sed` and rebuilding `base-images/base-runner`
locally but it takes too much time, looks too kludgy (even to me) and is likely to be broken
in a week or so :-)
* Fix bad build check.
Bad build checks previously just checked that a file is an executable
ELF. This is insufficient because libraries are also executable ELFs.
Add a check that the file contains LLVMFuzzerTestOneInput, in order
to consider a fuzz target.
Also update coverage and test_report for consistency.
* [infra] Enable clange code coverage reports generation for local runs.
* Use runner image and move corpus management to the helper.py
.
* Clean up, delete unnecessary stuff, add comments.
* Run fuzz targets in parallel. Do not exit in case of an error.
* Address review feedback, except of the threading thing.
* Fix a typo.
* Use ThreadPool implementatino available in standard python2.7 package.
* Add dry run support + no corpus download option.
* Fix flags handling + add log output in case of an error.
* Append arguments for fuzz target instead of replacing them.
* Remove dry run functionality as it currently errors out after two runs.
* Fix some spacing in the code.
* Update documentation regarding new code coverage script.
* [infra] Enable bad build checks once again.
* Minor typo.
* [bad_example] Update build flags for reproducing bad instrumentation scenario.
* [bad_example] split bad/no instrumentation case into two different ones.
* Use new approach for partial instrumentation detection + do that only for libFuzzer.
* Rename bad_example_bad_instrumentation into bad_example_partial_instrumentation.
* Calculate number of broken targets and fail if 10+% are broken.
* Multiprocess madness.
* Always run all checks and store all errors + clean up the code and add comments
* Add special handling for the projects with very small fuzz targets.
* Remove unnecessary semicolon.
* Address review comments.
* Address more review comments, small refactoring.
* [infra] Enable bad_build_checks without test_all step that runs all the fuzzers.
* [infra] Add COPY step for test_all_for_bad_build script.
* [infra] Use SKIP_TEST_TARGET_RUN=1 to avoid running all fuzz targets during test_all.
* [infra] Invern the condition in test_all script.
* [infra] Finally fix the condition in test_all.
* [WIP] Add post build checks for catching and reporting bad builds + test projects.
* Move all type of issues into a single project. Tested all sanitizers.
* First version of the script for performing bad build checks. Still not ready to commit.\
* Added a valid example that should pass all the checks + fixed instrumentation check.
* Use SIGTERM and the handler calling _exit(0) for the startup crash case.
* Small fixes.
* Ignore startup_crash check_for fuzzing engines other than libFuzzer.
* Use "local" for local variables in the script, fix container name.
* Add COPY step for bad_build_check into base-runner Dockerfile.
* Make bad_build_check temporary noop unless EXPERIMENTAL_BAD_BUILD_CHECK is set to 1.
* Temporary remove test binaries step.
* Remove empty line.
* Remove unnecessary env variable.
Separate test_report script runs tests and generates junit-style
reports. The script doesn't fail when fuzzer fails, but marks it as
failed instead.
Fixes#168
Max Moroz