Commit Graph

100 Commits

Author SHA1 Message Date
Max Moroz 4574ff8bdb
[infra] Add the build configuration and other pieces for clang code coverage. (#1547)
* [infra] Add the build configuration and other pieces for clang code coverage.

* Address the initial feedback, still a draft with some pieces missing.

* Trying to test things locally.

* Work in progress, now able to test the build process.

* Rebase and fix a comment.

* Figure out how to upload report, but still have issues with GCS access.

* Remove unnecessary debug prints.

* WIP: adding targets list reading + minor clean up.

* WIP

* WIP: now it works end to end but is very dirty and has limitations.

* WIP: use rsync -r -d to overwrite older reports (if any)

* WIP: trying to use a helper script for bulk corpus download.

* WIP: download corpus in batches passed via args.

* WIP: testing corpus downloading in batches.

* WIP: woohoo, now it works!

* Some cleanup for a high level review.

* Address review comments and add FAQ entry.

* Update FAQ.md as per comments by Oliver.
2018-08-20 06:42:45 -07:00
Max Moroz dbe2731d93
[infra] Print logs URL for base images and msan libs. (#1666) 2018-07-27 10:07:54 -07:00
Oliver Chang 180c87f194 gcb: add env to targets_list step 2018-07-27 15:34:24 +10:00
Oliver Chang bf75778262 pyformat python code in gcb dir. 2018-07-27 13:32:39 +10:00
Oliver Chang 60fb6a8d7b
Generate and upload targets list during build. (#1660) 2018-07-27 13:29:51 +10:00
Abhishek Arya c328ff7518
Speculative fix for firefox build fix. (#1661)
* Speculative fix for firefox build fix.

* Use a yaml attribute.
2018-07-26 19:49:56 -07:00
Oliver Chang 2920beb131 Freeze gcb requirements. 2018-06-01 11:48:42 +10:00
Oliver Chang 5d11b552e8
Update requirements.txt 2018-06-01 11:30:50 +10:00
Oliver Chang d9470966b8
Update builds_status.py 2018-05-30 18:40:19 +10:00
Oliver Chang 24aaf6015a
Update builds_status.py 2018-05-30 18:34:57 +10:00
Max Moroz 69ffa9bfdf
[infra] Invoke run_fuzzer from bad_build_check for regression testing (fixes #1355). (#1357)
* [infra] Invoke run_fuzzer from bad_build_check for regression testing (fixes #1355).

* Address review comments.
2018-04-26 07:17:44 -07:00
Max Moroz 4df2262466
[infra] Enable bad build checks once again. (#838)
* [infra] Enable bad build checks once again.

* Minor typo.

* [bad_example] Update build flags for reproducing bad instrumentation scenario.

* [bad_example] split bad/no instrumentation case into two different ones.

* Use new approach for partial instrumentation detection + do that only for libFuzzer.

* Rename bad_example_bad_instrumentation into bad_example_partial_instrumentation.

* Calculate number of broken targets and fail if 10+% are broken.

* Multiprocess madness.

* Always run all checks and store all errors + clean up the code and add comments

* Add special handling for the projects with very small fuzz targets.

* Remove unnecessary semicolon.

* Address review comments.

* Address more review comments, small refactoring.
2018-04-17 09:31:53 -07:00
Kevin Lubick 33bc7ae7e2 Add timezone and mouseover to build-status page (#1235)
* Add timezone and mouseover to build-status page
2018-03-12 13:25:29 -07:00
Oliver Chang b86f335e6c fix pixman msan blacklist. 2018-02-27 15:35:41 +11:00
Oliver Chang e65e323235 Add base-msan-builder to base images build. 2018-02-21 14:09:17 +11:00
Oliver Chang 92152969c5
[infra] Update upload URL timeout to be the same as build timeout (#1112) 2018-02-01 16:04:11 +11:00
Abhishek Arya 028d077fe5
Update webcomponentsjs (#1065) 2018-01-25 08:54:19 -08:00
Oliver Chang 72c5de97b8 msan-builder: upload libs as zip. 2018-01-19 14:12:44 +11:00
Oliver Chang 5b3485b2d5 msan_builder: fix permission of patch_build.py 2018-01-11 13:28:19 +11:00
Oliver Chang 3d0c96c087
Make MSan builds with instrumented libraries (#608) (#1064) 2018-01-11 12:56:33 +11:00
Max Moroz c069a7c780 Revert of "[infra] Reland profile configuration needed for new code coverage.".
See more information in comments for PR #1050.
2017-12-28 15:10:53 -08:00
Max Moroz add7bbbb2d
[infra] Reland profile configuration needed for new code coverage. (#1050) 2017-12-28 11:10:47 -08:00
Oliver Chang a99a336ecc [infra] fix typo 2017-12-06 19:04:07 -08:00
Oliver Chang 4305d7a1ef Build msan libs on our builders. 2017-12-06 18:58:36 -08:00
Abhishek Arya 259cb3849d
Increase build timeout to 10 hours (fix #978) 2017-11-12 19:40:27 -08:00
Oliver Chang f0d237c4cf
Update build.py 2017-11-01 10:05:46 +11:00
Oliver Chang 312cd47208 Add priliminary support for "engine-less" builds.
Needed for #925.
2017-10-31 15:22:58 +11:00
Oliver Chang 4670c7424d
[infra] Remove builds status hack to work around container build bug 2017-10-30 11:15:51 +11:00
Max Moroz 55c0ea6d97 [infra] Temporarily disable sanitizer=profile build configuration. 2017-10-20 20:34:01 -07:00
Oliver Chang a9f2986db5 Fix workaround for not marking builds as failed when GCS logs are truncated. 2017-10-11 12:50:07 -07:00
Oliver Chang 5148ce1389 Update builds_status.py 2017-10-06 00:45:37 -07:00
Oliver Chang edf077bd04 Update builds_status.py 2017-10-06 00:37:50 -07:00
Oliver Chang e2c1205f54 increase build timeout to 6 hours (#877) 2017-10-05 22:16:45 -07:00
Oliver Chang e9c38132ca Update builds_status.py 2017-10-05 22:11:54 -07:00
Max Moroz 718db9173d [infra] Temporary revert bad_build_check until most of the targets get fixed. (#831)
Need to take a closer look at the following issues:
- https://github.com/google/oss-fuzz/issues/815
- https://github.com/google/oss-fuzz/issues/816
- https://github.com/google/oss-fuzz/issues/817
- https://github.com/google/oss-fuzz/issues/818
- https://github.com/google/oss-fuzz/issues/819
- https://github.com/google/oss-fuzz/issues/820
- https://github.com/google/oss-fuzz/issues/821
- https://github.com/google/oss-fuzz/issues/822
- https://github.com/google/oss-fuzz/issues/823
- https://github.com/google/oss-fuzz/issues/824
- https://github.com/google/oss-fuzz/issues/825
- https://github.com/google/oss-fuzz/issues/826
- https://github.com/google/oss-fuzz/issues/827
- https://github.com/google/oss-fuzz/issues/828
- https://github.com/google/oss-fuzz/issues/829
- https://github.com/google/oss-fuzz/issues/830
2017-09-06 16:59:59 -07:00
Max Moroz 942b5fbbe1 [infra] Enable bad_build_checks without test_all step that runs all the fuzzers. (#814)
* [infra] Enable bad_build_checks without test_all step that runs all the fuzzers.

* [infra] Add COPY step for test_all_for_bad_build script.

* [infra] Use SKIP_TEST_TARGET_RUN=1 to avoid running all fuzz targets during test_all.

* [infra] Invern the condition in test_all script.

* [infra] Finally fix the condition in test_all.
2017-09-06 11:55:07 -07:00
Oliver Chang 656548e737 [infra] upload srcmap first. 2017-09-01 12:52:10 -07:00
Max Moroz 6a6163110a [infra] Add another coverage build using Clang Source-based Code Coverage. (#804)
* [infra] Add another coverage build using Clang Source-based Code Coverage.

* [infra] Remove redundant compile_coverage script.
2017-08-31 13:43:06 -07:00
Oliver Chang 32c3befdf8 Update builds_status.py 2017-08-10 17:19:52 -07:00
Oliver Chang 3798742fa7 [infra] Fix builds query 2017-08-10 16:40:24 -07:00
Oliver Chang 57c5e5c677 [infra] Increase page size of builds list. 2017-08-10 11:47:29 -07:00
Oliver Chang 223733c213 [infra] builds_status: try to reduce 504s. 2017-08-10 11:44:19 -07:00
Max Moroz b604770ee9 [infra] Temporarily disable "test binaries" build step. 2017-08-09 18:26:19 -07:00
Max Moroz e2e0c22845 [infra] Execute test_all step (includes bad_build_check) for all builds. (#766)
* [infra] Execute test_all step (includes bad_build_check) for all builds.

* Checkout master branch as usual.
2017-08-09 15:22:02 -07:00
Max Moroz f81476ca75 [infra] Add post build checks for catching and reporting bad builds + add test projects. (#754)
* [WIP] Add post build checks for catching and reporting bad builds + test projects.

* Move all type of issues into a single project. Tested all sanitizers.

* First version of the script for performing bad build checks. Still not ready to commit.\

* Added a valid example that should pass all the checks + fixed instrumentation check.

* Use SIGTERM and the handler calling _exit(0) for the startup crash case.

* Small fixes.

* Ignore startup_crash check_for fuzzing engines other than libFuzzer.

* Use "local" for local variables in the script, fix container name.

* Add COPY step for bad_build_check into base-runner Dockerfile.

* Make bad_build_check temporary noop unless EXPERIMENTAL_BAD_BUILD_CHECK is set to 1.

* Temporary remove test binaries step.

* Remove empty line.

* Remove unnecessary env variable.
2017-08-08 10:54:53 -07:00
Oliver Chang 0a2a9dad5b [infra] gcb: recognise TIMEOUT. 2017-07-19 08:56:18 -07:00
robertswiecki 268d8052a2 Support honggfuzz as a FUZZING_ENGINE (#636) 2017-06-01 17:55:01 -07:00
Oliver Chang c54ed42cd9 [infra] Build coverage build by default. 2017-05-08 11:31:45 -07:00
Oliver Chang 4896be51cd Create a separate coverage build.
First part of #191.
2017-05-08 11:17:27 -07:00
Oliver Chang 5c0631d83f [infra] build status: use locale date format. 2017-04-18 16:19:41 -07:00