Commit Graph

35 Commits

Author SHA1 Message Date
Abhishek Arya c066a4232f
Fix missing FUZZING_LANGUAGE in coverage build. (#3843) 2020-05-18 08:12:00 -07:00
Max Moroz 8ed2f7af34
[infra] Improve srcmap support for Go projects (#3355, #2714). (#3664)
* [infra] Improve srcmap support for Go projects (#3355, #2714).

* address review feedback
2020-04-16 22:44:01 -07:00
Max Moroz 130710d518
[infra] Change language attribute from "cpp" to "c++" in build scripts. (#3493)
* [infra] Change language attribute from "cpp" to "c++" in coverage job runner.

Should've done this in https://github.com/google/oss-fuzz/pull/3477. Good news is that nothing failed, the coverage job was just skipped for pretty much all projects.

* also remove default value from build_project script
2020-03-12 09:06:46 -07:00
Max Moroz a0ade7d8e5
[infra] Perform "gsutil rm -rf" if we need to overwrite coverage data. (#3404)
* [infra] Perform "gsutil rm -rf" if we need to overwrite coverage data.

* format
2020-02-18 09:25:16 -08:00
Max Moroz 6e8232078a
[infra] Use wildcard in gsutil cp command as it lacks -T option. (#3396) 2020-02-14 13:23:20 -08:00
Max Moroz dfaf951f35
[infra] Upload latest.version metadata file for most recent build. (#3379)
* [infra] Upload latest.version metadata file for most recent build.

* move common functionality into build_lib

* pass content_type to get_signed_url

* format
2020-02-13 07:31:12 -08:00
Max Moroz 4092bebcb5
[infra] Fix broken batching in the corpora download helper (#3351). (#3353)
* [infra] Fix broken batching in the corpora download helper (#3351).

* one more
2020-02-06 15:44:36 -08:00
Max Moroz 8d3d290c61
[infra] Generate full coverage JSONs for dataflow enabled targets (#1632). (#3325) 2020-02-04 17:42:25 -08:00
Max Moroz 7751ab5a87
[infra] Introduce "language" attribute in the project.yaml (#3297). (#3299)
* [infra] Introduce "language" attribute in the project.yaml (#3297).

* follow up

* enable the attribute for more projects

* trailing newline
2020-01-30 15:36:44 -08:00
Max Moroz f6002f6139
[infra] Collect dataflow traces on the builder (#1632). (#3238)
* [infra] Skeleton of the changes needed for collecting DFT on the builder (#1632).

* move ENGINE_INFO to the helper as well

* make collect_dft +x

* syntax fixes

* add actual dataflow tracer script

* format

* more refactoring and cleanup

* format

* address Oliver's feedback

* format

* more fixes

* format

* do not redirect stderr to stdout

* add exit at the end of main

* address feedback from Oliver
2020-01-17 06:24:15 -08:00
Max Moroz 3a9d7381e7
[infra] Print exact reproduction commands in build failures (#3104). (#3236)
* [infra] Print exact reproduction commands in build failures (#3104).

* syntax fixes

* remove redundant &&

* fix a typo

* use named arguments in format strings

* format
2020-01-15 14:12:22 -08:00
Max Moroz 12b5811ad0
[infra] Add parens to ensure correct execution order (#3104). (#3213) 2020-01-09 10:34:08 -08:00
Max Moroz cfe13cb116
[infra] Print additional information for build/coverage failures on GCB (#3104). (#3208)
* [infra] Explicily print failing build config in the GCB log (#3104).

* add trailing "false" command + add a message for misisng coprus backups in code coverage build

* improve the coverage message and behavior

* fix exit command

* use .format() for a single argument format string too

* fix typo
2020-01-09 09:41:58 -08:00
Max Moroz 1463a2c523
[infra] Do not skip coverage build if there is no build.sh file (#3185). (#3189) 2020-01-06 07:45:45 -08:00
Max Moroz 3dd03528b7
[infra] Use more generic approach to skip code coverage for Go projects. (#2953)
* [infra] Use more generic approach to skip code coverage for Go projects.

* address review comments
2019-10-16 07:35:22 -07:00
Max Moroz ccfb73427f
[infra] Skip code coverage job for gofuzz projects (#2714). (#2818) 2019-09-09 08:47:58 -07:00
Max Moroz 12d6d31e44
[infra] Use gsutil "cp" instead of "rsync" for coverage job (#2585). (#2625) 2019-07-23 10:12:37 -07:00
Max Moroz 497cd398d0 [infra] Fix code coverage builder script (regressed after #2264). 2019-03-31 11:33:07 -07:00
Oliver Chang 66008885e9
Remove image pushing step from build (#2232)
They aren't used and pushing them appears to be flaky.
2019-03-13 09:48:12 +11:00
Max Moroz fdd28c42e9
[infra] Add srcmap step to code coverage job (follow-up #1547). (#1868) 2018-10-11 13:52:49 -07:00
Max Moroz 7703e360de
[infra] Remove sancov and rename "profile" to "coverage". (#1839)
* [infra] Remove sancov and rename "profile" to "coverage".

* Bring coverage flags back.

* Update projects files that rely on SANITIZER="profile".
2018-10-01 06:43:21 -07:00
Max Moroz 4867091719 [infra] Code Coverage: fix build_and_run_coverage.py. 2018-09-12 07:01:21 -07:00
Max Moroz 8f687dfdad [infra] Code Coverage: update download_corpus script and revert #1798. 2018-09-11 12:26:59 -07:00
Max Moroz e6e71d0a92 [infra] Do not fail coverage job if corpus download failed (#1798, follow-up #1547). 2018-09-06 13:19:20 -07:00
Max Moroz 60f9023008
[infra] Add report date to the latest report info + fix format strings (follow-up #1547). (#1791) 2018-09-04 18:13:18 -07:00
pdknsk 6ed5fe8ed5 allow folded style for coverage_extra_args (#1786) 2018-09-05 09:27:57 +10:00
Max Moroz 2d09a7e80f
[infra] Code Coverage latest report info to be picked up by CF (follow-up #1547). (#1781)
* [infra] Code Coverage latest report info to be picked up by CF (follow-up #1547).

* Remove unintentional change.

* Remove unnecessary empty lines.
2018-09-03 10:40:23 -07:00
Max Moroz 733c896ce5
[infra] Support extra coverage args in project.yaml (fix #1726, follow-up #1547). (#1774)
* [infra] Support extra coverage args in project.yaml (fix #1726, follow-up #1547).

* Update the documentation page.

* Fix review comments by Jonathan.
2018-08-30 09:46:14 -07:00
Max Moroz c5a9204ae7
[infra] Code Coverage: copy /usr/include headers and upload logs to GCS (follow-up #1547). (#1764) 2018-08-27 08:25:29 -07:00
Max Moroz 23a04ab280
[infra] Use 'coverage' and 'fuzzing' tags to filter builds (follow-up #1547). (#1751)
* [infra] Use 'coverage' and 'fuzzing' tags to filter builds (follow-up #1547).

* Remove unintentional print call.

* Remove a variable that is not used anympre.
2018-08-23 19:14:09 -07:00
Max Moroz 7c476809f3
[infra] Skip code coverage job for projects that are disabled (#1743, #1547). (#1745)
* [infra] Skip code coverage job for projects that are disabled.

* Remove unintentional change.

* Address review feedback.
2018-08-22 14:14:02 -07:00
Max Moroz 204110f6bb
[infra] Skip coverage job for projects w/o fuzz targets (fix #1743, follow-up #1547). (#1744)
* [infra] Skip coverage job for projects w/o fuzz targets (fix #1743, follow-up #1547).

* Use build_id=0 and a more generic message.
2018-08-22 08:55:59 -07:00
Max Moroz 113274a890 [infra] Fix check for qualified target name: include "_" (follow-up #1547). 2018-08-20 10:35:10 -07:00
Max Moroz ab17c5f915 [infra] Do not mark coverage job as failed when there are no targets list (follow-up for #1547). 2018-08-20 10:26:27 -07:00
Max Moroz 4574ff8bdb
[infra] Add the build configuration and other pieces for clang code coverage. (#1547)
* [infra] Add the build configuration and other pieces for clang code coverage.

* Address the initial feedback, still a draft with some pieces missing.

* Trying to test things locally.

* Work in progress, now able to test the build process.

* Rebase and fix a comment.

* Figure out how to upload report, but still have issues with GCS access.

* Remove unnecessary debug prints.

* WIP: adding targets list reading + minor clean up.

* WIP

* WIP: now it works end to end but is very dirty and has limitations.

* WIP: use rsync -r -d to overwrite older reports (if any)

* WIP: trying to use a helper script for bulk corpus download.

* WIP: download corpus in batches passed via args.

* WIP: testing corpus downloading in batches.

* WIP: woohoo, now it works!

* Some cleanup for a high level review.

* Address review comments and add FAQ entry.

* Update FAQ.md as per comments by Oliver.
2018-08-20 06:42:45 -07:00