12d6d31e44
[infra] Use gsutil "cp" instead of "rsync" for coverage job ( #2585 ). ( #2625 )
2019-07-23 10:12:37 -07:00
e8df83fc35
[infra] Make travis deal with adding a new project or removing one ( #2566 )
...
This change causes travis to skip building a project when a project does not have build.sh file, a common scenario when a project is added (and only a project.yaml file is added) or removed.
2019-07-02 16:11:37 -07:00
c158db4df2
[infra] Remove debug code breaking bad_build_check #2555 ( #2557 )
2019-06-30 12:52:29 -07:00
5dece89e3c
[infra] Add support for dataflow builds to the helper script and build check ( #1632 ). ( #2501 )
...
* [infra] Add support for dataflow builds to the helper script and build check (#1632 ).
* Update travis config file.
* Address self-review comments and specify dataflow sanitizer for zstd as well.
* Fix fuzzing_engines in project.yaml
* Fix bad build check for DFSan.
* Use "hasattr" in helper.py to check the sanitizer argument.
* Address more review comments.
* Remove DataFlow config from zstd.
* fix a typo
2019-06-12 11:08:15 -07:00
3830c427b2
[infra] Ignore return code when removing seed corpus for dataflow build.
2019-06-10 15:32:51 -07:00
7250d70833
[infra] Run tests on the builder after patching MSan libraries. ( #2497 )
...
* [WORK IN PROGRESS] Debug Travis failure when doing build check for MSan build.
* Change order of "MSan patch libraries" and "build check" steps.
2019-06-10 11:18:59 -07:00
1f0a33c712
[infra] Do not archive seed corpus for dataflow builds.
2019-06-10 10:00:20 -07:00
3be06d5e78
[docs] Remove `-lFuzzingEngine` reference from base-builder image readme.
...
//cc @jonathanmetzman
2019-05-31 16:50:29 -07:00
d1539ce271
[infra] Change auto-generated build.sh template to use $LIB_FUZZING_ENGINE.
...
//cc @jonathanmetzman
2019-05-31 16:46:48 -07:00
4963f6ed93
Use AFL_DRIVER_DONT_DEFER=1 in AFL checks since bad_build_check doesn't support .options ( #2471 )
2019-05-28 10:29:16 -07:00
4d3073292a
Update test_all
2019-05-28 09:09:33 -07:00
e2003e1f39
Remove code for copying dataflow scripts ( #2461 )
...
Dataflow scripts are removed in
51570280af
2019-05-24 13:01:21 -07:00
12fa975bc3
Update regex for clang revision ( #2460 )
2019-05-24 11:19:55 -07:00
91a17744a0
infra: skip files whose names start with afl while looking for fuzz targets ( #2445 )
...
This should help to get rid of the "WARNING: corpus for systemd_afl-showmap not found:"
messages (that are harmless but confusing a bit). In general _get_fuzz_targets
should probably be in sync with ./infra/base-images/base-runner/test_all (where a file
is considered a fuzz target if grep can find "ELF" in it and so on).
2019-05-24 11:06:22 -07:00
bdd9519f44
[i386] Use different bucket for i386 builds. ( #2435 )
2019-05-17 12:47:11 -07:00
7073fb976c
[i386] Do i386 builds of projects that have opted-in ( #2416 )
2019-05-15 14:00:57 -07:00
898fc435c6
[travis] Use job matrix to split up builds to avoid timeout ( #2423 )
2019-05-15 11:01:53 -07:00
95dd538c60
[infra] Update name of check_instrumentation and error messages ( #2414 )
2019-05-15 10:35:06 -07:00
2361c62433
[infra] make it possible to override the percentage of targets that can be broken ( #2419 )
...
10% is an absolutely sensible default in general especially for single-purpose
libraries like json-parsers. When large "umbrella" projects (like systemd) are
fuzzed with 30 fuzzers (and counting (hopefully :-)) covering code scattered all
over their repositories it's too easy to introduce a broken fuzzer or break a couple
of fuzzers accidentally even after running `check_build`. Waiting for two to three
days for ClusterFuzz to open an issue isn't ideal from the point of view of large
open-source project maintainers (where generally contributors come and go) so one
solution would be to run something like when PRs are opened
```sh
helper.py check_build ALLOWED_BROKEN_TARGETS_PERCENTAGE=0 ...
```
and catch issues as early as possible (and fix them while the context isn't
completely faded away).
I also considered changing this with `sed` and rebuilding `base-images/base-runner`
locally but it takes too much time, looks too kludgy (even to me) and is likely to be broken
in a week or so :-)
2019-05-15 06:38:09 -07:00
8f9a3fa22b
[infra] Fix check_instrumentation in bad_build_check ( #2412 )
2019-05-14 10:36:48 -07:00
3976a002af
[i386] Improve linking and bad build check ( #2410 )
...
Improve linking by copying i386 libs instead of adding `-L` to `CXXFLAGS`
Get bad_build_check working for i386 binaries and add architecture check.
2019-05-14 10:18:02 -07:00
3df4e14520
Revert "[infra] Roll clang to r360398 ( #2405 )" ( #2408 )
...
This reverts commit 4a9941d9c4
2019-05-13 15:07:29 -07:00
c60fcd7247
Add initial support for i386 builds ( #2347 )
...
This feature is not yet officially supported
2019-05-13 15:01:25 -07:00
a181b43364
[travis] Add requirements file for travis and fix diff command ( #2406 )
2019-05-13 13:04:23 -07:00
4a9941d9c4
[infra] Roll clang to r360398 ( #2405 )
2019-05-13 10:26:40 -07:00
f9a6870971
[travis] Improve build script ( #2404 )
...
Improve print statements and handling of projects that do not list sanitizers in project.yaml.
2019-05-13 09:55:06 -07:00
3727791e43
[travis] Dont build the same project multiple times ( #2399 )
2019-05-13 09:39:24 -07:00
7ec14aed39
[infra] Create yaml file and build script for travis. ( #2397 )
2019-05-10 15:58:18 -07:00
047eb4be4f
Update requirements.txt
2019-05-02 07:26:36 -07:00
316c1df6a4
Roll clang to r359254 ( #2358 )
...
Roll clang to r359254 to uptake fixes for exception handling and MSAN.
2019-04-30 10:19:20 -04:00
a047d08b8e
Revert "[infra] Revert the recent clang roll to fix some builds ( #2315 ) ( #2316 ) ( #2356 )
...
* Revert "[infra] Revert the recent clang roll to fix some builds (#2315 ). (#2316 )"
This reverts commit 8b34fd89d2
2019-04-29 16:22:20 -04:00
3df307ad59
use immutable default arguments ( #2341 )
2019-04-22 12:28:07 -07:00
c3d7aba005
Prevent misuse of check_build ( #2340 )
...
Don't allow using check_build with builds it doesn't actually support.
2019-04-22 09:50:11 -07:00
ae9398deef
Change LIB_FUZZING_ENGINE to -fsanitize=fuzzer in libFuzzer builds ( #2312 )
2019-04-15 10:05:02 -07:00
8b34fd89d2
[infra] Revert the recent clang roll to fix some builds ( #2315 ). ( #2316 )
2019-04-12 09:14:09 -07:00
97eca89ed9
Build clang so that it can compile targets with `-fsanitize=fuzzer` ( #2311 )
...
Build clang so that it can compile targets with -fsanitize=fuzzer
without adding any runtime dependencies.
2019-04-11 09:51:19 -07:00
946d0603ba
[infra] Enabling DataFlow build configuration ( #1632 ). ( #2292 )
...
* [infra] Enabling DataFlow build configuration (#1632 ).
* Copy all data flow scripts from the libFuzzer repo.
2019-04-09 07:25:14 -07:00
6625401871
Fix build breakage caused by unnecessary AFL testing ( #2291 )
...
Don't test compilation with afl-clang and afl-fuzz
2019-04-04 07:11:40 -07:00
23a5f44b29
Address comments from https://github.com/google/oss-fuzz/pull/2285 . ( #2287 )
2019-04-03 15:29:23 +11:00
994900ab47
Fix builds status script ( #2285 )
...
Get a batch of the last 2000 builds, and filter on those ourselves
instead of calling the cloudbuild API to filter for us. The cloudbuild
API filtering API does not work with our number of builds.
Fixes #2263
2019-04-02 12:00:55 +08:00
7bf55a11df
Use a single tag for builds. ( #2284 )
...
Filtering on multiple tags doesn't seem to work.
2019-04-02 08:19:58 +08:00
497cd398d0
[infra] Fix code coverage builder script (regressed after #2264 ).
2019-03-31 11:33:07 -07:00
aa231dc801
Add a project tag to builds for filtering ( #2263 ). ( #2264 )
2019-03-26 17:02:05 +11:00
2c9fa6a668
Allow preloading external libFuzzer functions using LD_PRELOAD ( #2255 )
2019-03-20 16:11:51 -07:00
66008885e9
Remove image pushing step from build ( #2232 )
...
They aren't used and pushing them appears to be flaky.
2019-03-13 09:48:12 +11:00
322b86a848
[infra] Disable honggfuzz builds by default (follow up for #2227 ).
2019-03-12 14:56:09 -07:00
65037d2ea8
[infra] Bump build timeout from 10h to 12h. ( #2227 )
2019-03-11 07:38:14 -07:00
f58939a944
fix build check for aarch64 ( #2179 )
2019-02-20 21:17:50 -08:00
890c999225
enable builds of base images for AArch64 ( #2176 )
...
First step to enable oss-fuzzers to run on native aarch64 as mentioned in
https://github.com/google/oss-fuzz/issues/1754#issuecomment-461926649
2019-02-20 16:16:46 +11:00
13b6571e2b
[infra] Suppress errors from cp command in the coverage build ( #2087 and others).
2019-01-14 13:01:16 -08:00
Max Moroz