Commit Graph

421 Commits

Author SHA1 Message Date
jonathanmetzman 95dd538c60
[infra] Update name of check_instrumentation and error messages (#2414) 2019-05-15 10:35:06 -07:00
Evgeny Vereshchagin 2361c62433 [infra] make it possible to override the percentage of targets that can be broken (#2419)
10% is an absolutely sensible default in general especially for single-purpose
libraries like json-parsers. When large "umbrella" projects (like systemd) are
fuzzed with 30 fuzzers (and counting (hopefully :-)) covering code scattered all
over their repositories it's too easy to introduce a broken fuzzer or break a couple
of fuzzers accidentally even after running `check_build`. Waiting for two to three
days for ClusterFuzz to open an issue isn't ideal from the point of view of large
open-source project maintainers (where generally contributors come and go) so one
solution would be to run something like when PRs are opened
```sh
helper.py check_build ALLOWED_BROKEN_TARGETS_PERCENTAGE=0 ...
```
and catch issues as early as possible (and fix them while the context isn't
completely faded away).

I also considered changing this with `sed` and rebuilding `base-images/base-runner`
locally but it takes too much time, looks too kludgy (even to me) and is likely to be broken
in a week or so :-)
2019-05-15 06:38:09 -07:00
jonathanmetzman 8f9a3fa22b
[infra] Fix check_instrumentation in bad_build_check (#2412) 2019-05-14 10:36:48 -07:00
jonathanmetzman 3976a002af
[i386] Improve linking and bad build check (#2410)
Improve linking by copying i386 libs instead of adding `-L` to `CXXFLAGS`
Get bad_build_check working for i386 binaries and add architecture check.
2019-05-14 10:18:02 -07:00
jonathanmetzman 3df4e14520
Revert "[infra] Roll clang to r360398 (#2405)" (#2408)
This reverts commit 4a9941d9c4.
2019-05-13 15:07:29 -07:00
jonathanmetzman c60fcd7247
Add initial support for i386 builds (#2347)
This feature is not yet officially supported
2019-05-13 15:01:25 -07:00
jonathanmetzman 4a9941d9c4
[infra] Roll clang to r360398 (#2405) 2019-05-13 10:26:40 -07:00
jonathanmetzman 316c1df6a4
Roll clang to r359254 (#2358)
Roll clang to r359254 to uptake fixes for exception handling and MSAN.
2019-04-30 10:19:20 -04:00
jonathanmetzman a047d08b8e
Revert "[infra] Revert the recent clang roll to fix some builds (#2315) (#2316) (#2356)
* Revert "[infra] Revert the recent clang roll to fix some builds (#2315). (#2316)"

This reverts commit 8b34fd89d2.
2019-04-29 16:22:20 -04:00
jonathanmetzman ae9398deef
Change LIB_FUZZING_ENGINE to -fsanitize=fuzzer in libFuzzer builds (#2312) 2019-04-15 10:05:02 -07:00
Max Moroz 8b34fd89d2
[infra] Revert the recent clang roll to fix some builds (#2315). (#2316) 2019-04-12 09:14:09 -07:00
jonathanmetzman 97eca89ed9
Build clang so that it can compile targets with `-fsanitize=fuzzer` (#2311)
Build clang so that it can compile targets with -fsanitize=fuzzer
without adding any runtime dependencies.
2019-04-11 09:51:19 -07:00
Max Moroz 946d0603ba
[infra] Enabling DataFlow build configuration (#1632). (#2292)
* [infra] Enabling DataFlow build configuration (#1632).

* Copy all data flow scripts from the libFuzzer repo.
2019-04-09 07:25:14 -07:00
jonathanmetzman 6625401871
Fix build breakage caused by unnecessary AFL testing (#2291)
Don't test compilation with afl-clang and afl-fuzz
2019-04-04 07:11:40 -07:00
jonathanmetzman 2c9fa6a668
Allow preloading external libFuzzer functions using LD_PRELOAD (#2255) 2019-03-20 16:11:51 -07:00
Sebastian Pop f58939a944 fix build check for aarch64 (#2179) 2019-02-20 21:17:50 -08:00
Sebastian Pop 890c999225 enable builds of base images for AArch64 (#2176)
First step to enable oss-fuzzers to run on native aarch64 as mentioned in
https://github.com/google/oss-fuzz/issues/1754#issuecomment-461926649
2019-02-20 16:16:46 +11:00
Max Moroz 13b6571e2b [infra] Suppress errors from cp command in the coverage build (#2087 and others). 2019-01-14 13:01:16 -08:00
Max Moroz b58fb5dfdb [infra] Remove unused coverage-report-server.py from base-clang image. 2019-01-03 12:44:07 -08:00
Abhishek Arya 5c9cdcfc80
Add tinyxml2 xmltest small target exclusion 2018-12-22 02:19:25 -08:00
Max Moroz 2d1a956fcf [infra] Enable debugging mode for srcmap script to make its failures more obvious. 2018-12-10 10:11:40 -08:00
Max Moroz 9eb9f3bb63 [infra] Follow-up fix for bad_build_check script (#1990). 2018-11-30 12:38:08 -08:00
Max Moroz 7c2f32f748
[infra] Add return value to bad_build_checks script. (#1993) 2018-11-30 11:15:51 -08:00
Oliver Chang 3d662e1bec
Add support for tags field in project.yaml (#1974) (#1980)
* Add support for tags field in project.yaml (#1974)

Also add "sundew" label to libxml2_xml_reader_for_file_fuzzer.

These add additional labels in filed bugs.
2018-11-27 16:46:26 +11:00
Evgeny Vereshchagin 3740c751fd [infra] base-runner: remove strict_string_check from ASAN_OPTIONS (#1972)
Whether string_string_checks will be turned on or not is
being discussed in https://github.com/google/oss-fuzz/issues/542.
Until it's settled, it'd probably be better to remove the option
from ASAN_OPTIONS altogether so as not to confuse anybody.
2018-11-26 09:57:09 -08:00
Abhishek Arya 7de7a3afad
Add checksum_fuzzer to bad build check exclusions
This is due to small number of edges in the fuzz target
2018-11-11 07:16:34 -08:00
Evgeny Vereshchagin 3358b336be [infra]: place the findings of fuzzers run with `helper.py run_fuzzer` in $OUT (#1922)
When `libFuzzer` is used as a fuzzing engine it conveniently puts
everything it finds into its working directory, which happens to
be $OUT. When any other engine is used, the output is sent to /tmp,
which disappears as soon as the container stops. That makes it
unnecessarily hard to extract reproducers found by afl for further analysis
without joining the mount namespace of a running container and
almost impossible with `honggfuzz`, that exits immediately after a crash.
2018-11-08 09:27:46 +11:00
Abhishek Arya bbac75cdef
Revert "Revert "[infra] also pass dictionaries when `afl` or `honggfuzz` is used as a fuzzing engine (#1925)" (#1927)" (#1928)
This reverts commit 6c6934dc8b.
2018-11-06 08:30:58 -08:00
Max Moroz 6c6934dc8b
Revert "[infra] also pass dictionaries when `afl` or `honggfuzz` is used as a fuzzing engine (#1925)" (#1927)
This reverts commit 56fc756fc9.
2018-11-06 08:00:43 -08:00
Evgeny Vereshchagin 56fc756fc9 [infra] also pass dictionaries when `afl` or `honggfuzz` is used as a fuzzing engine (#1925)
Currently, dictionaries are taken into account only when `libfuzzer` is
used as a fuzzing engine (and also apparently `none` but I'm not sure what it is).
This patch makes it possible to make use of dictionaries with other fuzzing
engines too. I didn't touch the code handling options passed to libFuzzer
so as not to break anything :-)
2018-11-06 07:52:24 -08:00
Max Moroz 66f8069ff1 [infra] Use -pthread instead of -Wl,-lpthread (fixes #1886). 2018-10-22 07:24:42 -07:00
Max Moroz 5c263869ae
[infra] Fix missing linking dependencies for coverage build (#1858). (#1863) 2018-10-09 11:04:03 -07:00
pdknsk 7b179a4a24 fix hg revision with uncomitted changes (#1849) 2018-10-02 10:52:49 -07:00
Max Moroz d39e0f7ea0 [infra] Restore srcmap binary back to base-builder image (follow-up #1848). 2018-10-01 23:13:59 -07:00
Max Moroz 7703e360de
[infra] Remove sancov and rename "profile" to "coverage". (#1839)
* [infra] Remove sancov and rename "profile" to "coverage".

* Bring coverage flags back.

* Update projects files that rely on SANITIZER="profile".
2018-10-01 06:43:21 -07:00
Max Moroz 2331c308cd [infra] Use -merge=1 instead of -runs=0 for coverage (#1743, follow-up #1547). 2018-09-25 17:22:07 -07:00
Max Moroz e348738942 [infra] Bump LLVM revision to r342582 to fix MSan issues. 2018-09-19 13:23:42 -07:00
Max Moroz 8f687dfdad [infra] Code Coverage: update download_corpus script and revert #1798. 2018-09-11 12:26:59 -07:00
Max Moroz 72b82ee08f [infra] Code Coverage: use -close_fd_mask=3 to speed up noisy fuzz targets. 2018-08-31 15:43:07 -07:00
Max Moroz c5a9204ae7
[infra] Code Coverage: copy /usr/include headers and upload logs to GCS (follow-up #1547). (#1764) 2018-08-27 08:25:29 -07:00
Max Moroz 649d3a7708
[infra] use -src-root-dir=/ and fail hard on errors (follow-up #1547). (#1746) 2018-08-23 08:09:52 -07:00
Max Moroz d22e0cb352 [infra] Suppress errors from $COPY_SOURCES_CMD (follow-up #1741, #1547). 2018-08-22 07:20:09 -07:00
Max Moroz 17a6cfbd40
[infra] Use coverage utils from Chromium and other fixes (follow-up #1547). (#1741)
* [infra] Use coverage utils from Chromium and other fixes (follow-up #1547).

* Sort dependencies in an alpha order plus make more readable.

* Re-order arguments passed to coverage_helper script.

* Rename REPORT_DIR variable and put summary.json into platform specific dir.

* Fix -src-root-dir value.
2018-08-21 14:02:48 -07:00
Max Moroz 3c6e2bc79a [infra] Bring -ldl flag back and suppress warning messages (follow-up #1547). 2018-08-20 10:56:51 -07:00
Max Moroz 80ea4a427a
[infra] Coverage: handle shared libraries via coverage_helper script. (#1727)
* [infra] Coverage: handle shared libraries and provide per directory view.

* Add coverage_helper script for detecting shared libraries.

* Address review feedback.
2018-08-17 15:15:08 -07:00
Max Moroz f021e50a21
[infra] Coverage: export stats for individual fuzzers (to be uploaded in #1547). (#1722) 2018-08-16 21:23:56 -07:00
Max Moroz 137bc2c9de
[infra] Add UBSAN_OPTIONS=\"silence_unsigned_overflow=1\" to base-builder image (#1715). (#1717) 2018-08-15 15:36:37 -07:00
Max Moroz e1aa07c491
[infra] Add script for downloading a batch of corpus backups (for #1547). (#1713)
* [infra] Add script for downloading a batch of corpus backups (for #1547).

* Add missing wget dependency and replace a tab with spaces.

* Convert more tabs to spaces.
2018-08-15 06:34:10 -07:00
Max Moroz e6bf26c8dd
[infra] Bump LLVM revision to r338452. (#1708) 2018-08-13 07:44:21 -07:00
Max Moroz d747892866
[infra] Update coverage script to support sources white listing. (#1707) 2018-08-12 10:19:40 -07:00