Commit Graph

106 Commits

Author SHA1 Message Date
Evgeny Vereshchagin 3740c751fd [infra] base-runner: remove strict_string_check from ASAN_OPTIONS (#1972)
Whether string_string_checks will be turned on or not is
being discussed in https://github.com/google/oss-fuzz/issues/542.
Until it's settled, it'd probably be better to remove the option
from ASAN_OPTIONS altogether so as not to confuse anybody.
2018-11-26 09:57:09 -08:00
Abhishek Arya 7de7a3afad
Add checksum_fuzzer to bad build check exclusions
This is due to small number of edges in the fuzz target
2018-11-11 07:16:34 -08:00
Evgeny Vereshchagin 3358b336be [infra]: place the findings of fuzzers run with `helper.py run_fuzzer` in $OUT (#1922)
When `libFuzzer` is used as a fuzzing engine it conveniently puts
everything it finds into its working directory, which happens to
be $OUT. When any other engine is used, the output is sent to /tmp,
which disappears as soon as the container stops. That makes it
unnecessarily hard to extract reproducers found by afl for further analysis
without joining the mount namespace of a running container and
almost impossible with `honggfuzz`, that exits immediately after a crash.
2018-11-08 09:27:46 +11:00
Abhishek Arya bbac75cdef
Revert "Revert "[infra] also pass dictionaries when `afl` or `honggfuzz` is used as a fuzzing engine (#1925)" (#1927)" (#1928)
This reverts commit 6c6934dc8b.
2018-11-06 08:30:58 -08:00
Max Moroz 6c6934dc8b
Revert "[infra] also pass dictionaries when `afl` or `honggfuzz` is used as a fuzzing engine (#1925)" (#1927)
This reverts commit 56fc756fc9.
2018-11-06 08:00:43 -08:00
Evgeny Vereshchagin 56fc756fc9 [infra] also pass dictionaries when `afl` or `honggfuzz` is used as a fuzzing engine (#1925)
Currently, dictionaries are taken into account only when `libfuzzer` is
used as a fuzzing engine (and also apparently `none` but I'm not sure what it is).
This patch makes it possible to make use of dictionaries with other fuzzing
engines too. I didn't touch the code handling options passed to libFuzzer
so as not to break anything :-)
2018-11-06 07:52:24 -08:00
Max Moroz 7703e360de
[infra] Remove sancov and rename "profile" to "coverage". (#1839)
* [infra] Remove sancov and rename "profile" to "coverage".

* Bring coverage flags back.

* Update projects files that rely on SANITIZER="profile".
2018-10-01 06:43:21 -07:00
Max Moroz 2331c308cd [infra] Use -merge=1 instead of -runs=0 for coverage (#1743, follow-up #1547). 2018-09-25 17:22:07 -07:00
Max Moroz 8f687dfdad [infra] Code Coverage: update download_corpus script and revert #1798. 2018-09-11 12:26:59 -07:00
Max Moroz 72b82ee08f [infra] Code Coverage: use -close_fd_mask=3 to speed up noisy fuzz targets. 2018-08-31 15:43:07 -07:00
Max Moroz 649d3a7708
[infra] use -src-root-dir=/ and fail hard on errors (follow-up #1547). (#1746) 2018-08-23 08:09:52 -07:00
Max Moroz 17a6cfbd40
[infra] Use coverage utils from Chromium and other fixes (follow-up #1547). (#1741)
* [infra] Use coverage utils from Chromium and other fixes (follow-up #1547).

* Sort dependencies in an alpha order plus make more readable.

* Re-order arguments passed to coverage_helper script.

* Rename REPORT_DIR variable and put summary.json into platform specific dir.

* Fix -src-root-dir value.
2018-08-21 14:02:48 -07:00
Max Moroz 80ea4a427a
[infra] Coverage: handle shared libraries via coverage_helper script. (#1727)
* [infra] Coverage: handle shared libraries and provide per directory view.

* Add coverage_helper script for detecting shared libraries.

* Address review feedback.
2018-08-17 15:15:08 -07:00
Max Moroz f021e50a21
[infra] Coverage: export stats for individual fuzzers (to be uploaded in #1547). (#1722) 2018-08-16 21:23:56 -07:00
Max Moroz e1aa07c491
[infra] Add script for downloading a batch of corpus backups (for #1547). (#1713)
* [infra] Add script for downloading a batch of corpus backups (for #1547).

* Add missing wget dependency and replace a tab with spaces.

* Convert more tabs to spaces.
2018-08-15 06:34:10 -07:00
Max Moroz d747892866
[infra] Update coverage script to support sources white listing. (#1707) 2018-08-12 10:19:40 -07:00
Max Moroz b8a87a862e
[infra] Coverage: always exclude libFuzzer source from the report. (#1697) 2018-08-09 10:45:05 -07:00
Max Moroz 0fe45f59a7
[infra] Re-enable UIO for UBSan in a non-faulting mode (#910). (#1690) 2018-08-07 10:04:25 -07:00
Evgeny Vereshchagin d2d9860ecd [infra] skip the rest of check_instrumentation if grep hasn't found anything (#1689)
When a fuzzer is seriously broken (which happens occasionally during debug),
it's unlikely to start properly let alone provide some meaningful
output. In this case, it seems reasonable to skip some checks and prevent bash
from encountering the following syntax error:
```
/usr/local/bin/bad_build_check: line 68: ((: < 100 : syntax error: operand expected (error token is "< 100 ")
```
2018-08-04 18:14:29 -07:00
Max Moroz 6b761f3826
[infra] Export coverage data in JSON format for picking up by CF. (#1665) 2018-07-27 07:34:02 -07:00
Oliver Chang 2ef8e32bfa Move targets_list to base-runner. 2018-07-27 14:59:02 +10:00
Max Moroz 218f5dc3cf
[infra] Add an option to disable serving coverage report over HTTP (needed for #1547). (#1635) 2018-07-20 15:43:08 -07:00
Max Moroz 645a0375fe
[infra] Support extra arguments for llvm-cov + update the binaries. (#1629)
* [infra] Support extra arguments for llvm-cov + update the binaries.

* Slightly change the doc to be less confusing.
2018-07-19 15:58:58 -07:00
Oliver Chang 40326f9520 Add libcap2 to base-runner image. 2018-07-09 17:22:15 +10:00
Abhishek Arya 12982882b8
Fix #1591 2018-07-03 09:26:51 -07:00
Abhishek Arya 3af21b0252
Don't check for LLVMFuzzerTestOneInput when there is no fuzzing engine. (#1579) 2018-06-28 07:08:57 -07:00
Abhishek Arya 4e87bd622d
Revert "Add support for running perffuzz in infra/helper.py (#1540)" (#1578)
This reverts commit 7f26e2fd92.
2018-06-28 06:11:32 -07:00
jonathanmetzman 9c2fe11666
Fix check for LLVMFuzzerTestOneInput (#1572) 2018-06-26 08:23:56 -07:00
jonathanmetzman 5ea8600f84
Remove test_report since it isn't usable from anywhere (#1569) 2018-06-25 16:59:21 -07:00
jonathanmetzman 36e576467c
Fix bad build check. (#1568)
* Fix bad build check.

Bad build checks previously just checked that a file is an executable
ELF. This is insufficient because libraries are also executable ELFs.
Add a check that the file contains LLVMFuzzerTestOneInput, in order
to consider a fuzz target.

Also update coverage and test_report for consistency.
2018-06-25 16:35:28 -07:00
Max Moroz c05848b347 [infra] Set UBSAN_CALLS_THRESHOLD_FOR_NON_UBSAN_BUILD back to 200 + add an explanation (#1542). 2018-06-20 07:54:20 -07:00
Max Moroz 52fe2acc1a [infra] Fix bad build check for UBSan since we compile engines without sanitizers now (fixes #1542). 2018-06-20 07:21:54 -07:00
Abhishek Arya 7f26e2fd92
Add support for running perffuzz in infra/helper.py (#1540)
* Add support for running perffuzz in infra/helper.py
https://github.com/carolemieux/perffuzz

* Update compile_perffuzz

* Update run_fuzzer
2018-06-19 15:02:00 -07:00
Max Moroz 9874e09f53
[infra] helper.py: support "profile" command for a single fuzz target with arbitrary corpus location. (#1535)
* [infra] helper.py: support "profile" command for a single fuzz target with arbitrary corpus location.

* Address review feedback.

* Update code_coverage.md page.
2018-06-18 14:19:48 -07:00
Max Moroz 883cdcf3b0
[infra] Fix helper.py as per feedback from @evverx + fix issues from #1519. (#1520)
* [infra] Fix helper.py as per feedback from @evverx + fix issues from #1519.

* Remove stuff from local testing

* Suppress unnecessary output from GSUtil.
2018-06-15 10:44:18 -07:00
Max Moroz 20e55ce3b1
[infra] Enable clange code coverage reports generation for local runs. (#1494)
* [infra] Enable clange code coverage reports generation for local runs.

* Use runner image and move corpus management to the helper.py
.

* Clean up, delete unnecessary stuff, add comments.

* Run fuzz targets in parallel. Do not exit in case of an error.

* Address review feedback, except of the threading thing.

* Fix a typo.

* Use ThreadPool implementatino available in standard python2.7 package.

* Add dry run support + no corpus download option.

* Fix flags handling + add log output in case of an error.

* Append arguments for fuzz target instead of replacing them.

* Remove dry run functionality as it currently errors out after two runs.

* Fix some spacing in the code.

* Update documentation regarding new code coverage script.
2018-06-14 15:00:46 -07:00
Abhishek Arya 4557d57b9b
Fix typo in bad_build_check (#1432) 2018-05-20 21:21:09 -07:00
jonathanmetzman 7b9bc817cd
Use "jobs -rp" instead of "jobs -p" to avoid infinite loop. (#1411) 2018-05-10 17:47:05 -07:00
jonathanmetzman e06ca92ee4
Install fonts-dejavu on base-runner (#1392)
Install a small (<10MB) font library on base-runner since some fuzzers
(such as image_deserialize_fuzzer) expect them on a system.
2018-05-02 08:33:08 -07:00
Abhishek Arya 624ea358c7
Match AFL startup crash condition with ClusterFuzz 2018-04-30 14:50:49 -07:00
Robin Richtsfeld 71d91515be Bash (#1382)
* Assigning an array to a string! Assign as array, or use * instead of @ to concatenate.

* Quote the parameter to -name so the shell won't interpret it.
2018-04-30 10:38:48 -07:00
Abhishek Arya 527ef4c7f0
Add AFL bad build checks (instrumentation, startup crash). (#1381)
* Add AFL bad build checks (instrumentation, startup crash).

* Fix incorrect seed corpus unpack for afl bad instrumentation, startup crash test

* Match AFL startup crash check with ClusterFuzz
2018-04-30 09:38:42 -07:00
Abhishek Arya 43fd25f26a
Add check_build command to infra/helper.py (Fix #1356) (#1379)
* Add check_build command to infra/helper.py

* Add exit message.
2018-04-29 18:44:22 -07:00
Alex Gaynor 82ca53f55d Attempt at fixing run_fuzzer for honggfuzz (#1370) 2018-04-27 07:56:18 -07:00
Max Moroz a51ea6df92 [infra] Temporarily disable bad build check with seed corpus. 2018-04-27 07:10:08 -07:00
Oliver Chang 924a786039
Update run_minijail 2018-04-27 13:46:39 +10:00
Max Moroz 69ffa9bfdf
[infra] Invoke run_fuzzer from bad_build_check for regression testing (fixes #1355). (#1357)
* [infra] Invoke run_fuzzer from bad_build_check for regression testing (fixes #1355).

* Address review comments.
2018-04-26 07:17:44 -07:00
Max Moroz 4d4f867b54 [infra] Fix RegEx in bad_build_check as "coverage" builds do not have counters, only guards. 2018-04-20 09:54:54 -07:00
Max Moroz 2519639f73 [infra] Lower down THRESHOLD_FOR_NUMBER_OF_EDGES to 100 (fixes #1333). 2018-04-17 15:28:01 -07:00
Max Moroz 4df2262466
[infra] Enable bad build checks once again. (#838)
* [infra] Enable bad build checks once again.

* Minor typo.

* [bad_example] Update build flags for reproducing bad instrumentation scenario.

* [bad_example] split bad/no instrumentation case into two different ones.

* Use new approach for partial instrumentation detection + do that only for libFuzzer.

* Rename bad_example_bad_instrumentation into bad_example_partial_instrumentation.

* Calculate number of broken targets and fail if 10+% are broken.

* Multiprocess madness.

* Always run all checks and store all errors + clean up the code and add comments

* Add special handling for the projects with very small fuzz targets.

* Remove unnecessary semicolon.

* Address review comments.

* Address more review comments, small refactoring.
2018-04-17 09:31:53 -07:00