Commit Graph

955 Commits

Author SHA1 Message Date
Philip Withnall b036df0ce1
glib: Fix build due to Meson option changing type (#3358)
* glib: Fix build due to Meson option changing type

In GLib master commit b220033c we changed the `libmount` option from a
`boolean` to a `feature`, which means it now takes
`enabled`/`disabled`/`auto` rather than `true`/`false`.

Signed-off-by: Philip Withnall <philip@tecnocode.co.uk>

Fixes: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20552

* glib: Fix homepage URI

The FSF page on GLib is not official, loaded with their advertising, and
quite out of date.

Signed-off-by: Philip Withnall <philip@tecnocode.co.uk>

* infra: Add help_url to the valid section names list in Travis checks

It’s documented as valid:
https://google.github.io/oss-fuzz/getting-started/new-project-guide/#help_url

Signed-off-by: Philip Withnall <philip@tecnocode.co.uk>
2020-02-07 05:03:31 -08:00
Max Moroz a32769e89a
[infra] Dump full coverage JSONs separately (#1632). (#3356) 2020-02-06 17:45:57 -08:00
Max Moroz 4092bebcb5
[infra] Fix broken batching in the corpora download helper (#3351). (#3353)
* [infra] Fix broken batching in the corpora download helper (#3351).

* one more
2020-02-06 15:44:36 -08:00
Leo Neat b96d2186ae
[CIFuzz] Go support (#3352)
* Add go support to detecting the main repo

* Go support test update

* Format

* Format

* Removing debug statment

* Format
2020-02-06 15:35:42 -08:00
Leo Neat 460661c700
Fixing entrypoint (#3354) 2020-02-06 14:00:55 -08:00
Leo Neat 5cefaac000
[CIFuzz] Separate build and run actions (#3336)
* Separating actions into build and run

* Formatting

* Formatting

* Removing project-name from run action

* Jonathan comments

* Maxs comments pt.1

* Updating example_main.yml

* Switch 1 0 to true false

* Add base Dockerfile for CIFuzz

* Adding license to dockerfile
2020-02-06 13:39:42 -08:00
Max Moroz 69f75431fe [infra] Dummy change in the base-builder to test something. 2020-02-06 11:11:24 -08:00
Max Moroz 67353e882d [cifuzz] Dummy change to test something. 2020-02-06 10:47:18 -08:00
Leo Neat 52e640af34
[CIFuzz] Fix .strip bug. (#3349) 2020-02-06 09:33:54 -08:00
Leo Neat f0df2d675c
Move detect_repo.py script into /opt (#3337) 2020-02-05 11:32:37 -08:00
Max Moroz c57465770a
[infra] Travis: rely on the presence of Dockerfile to identify buildable projects. (#3328)
* [infra] Travis: rely on the presese of Dockerfile to identify buildable projects.

* fix the comment
2020-02-05 09:14:59 -08:00
jonathanmetzman 27409e0fbc
[infra] Make travis_build presubmit clean (#3334) 2020-02-05 09:10:05 -08:00
Max Moroz 8d3d290c61
[infra] Generate full coverage JSONs for dataflow enabled targets (#1632). (#3325) 2020-02-04 17:42:25 -08:00
Leo Neat 53cb4f7935
[Infra] Update execute function in utils.py (#3319) 2020-02-04 11:51:18 -08:00
Leo Neat da04f5e431
[CIFuzz] Add seed and len control to fuzzer arguments (#3321) 2020-02-04 10:54:28 -08:00
Leo Neat 83d951bd1d
[CIFuzz] Prevent action from failing option (#3315)
* Adding opt to workflow to prevent failures

* fclose bug

* Undoing double negitive

* Undoing double negitive

* Undoing double negitive

* Syntax and formatting

* Olivers comments

* Presubmit fix

* Trying boolean argument

* cleaning

* cleaning fuzz disassembles

* Maxs comments pt.2

* Updating dry_run parse

* Comments added

* Formatting

* Olivers comments
2020-02-03 15:35:04 -08:00
Leo Neat 1a87da68c8
[cifuzz] - commit and pull request hook functionality (#3310) 2020-01-31 15:31:18 -08:00
Leo Neat f2588ea5a8
Moving execute to utils.py (#3313) 2020-01-31 10:19:12 -08:00
Max Moroz 7751ab5a87
[infra] Introduce "language" attribute in the project.yaml (#3297). (#3299)
* [infra] Introduce "language" attribute in the project.yaml (#3297).

* follow up

* enable the attribute for more projects

* trailing newline
2020-01-30 15:36:44 -08:00
Leo Neat 007226e5cc
[infra] unit/integration test updates (#3300)
* Merging build_specified_commit.py and utils.py

* undo dataflow change

* Updating RepoManager tests

* Updated detect_repo_tests

* Build specified commit updated

* Updated test_repos

* updating bisector tests

* formatting updates

* Switch inversions

* Rebase master
2020-01-30 10:27:56 -08:00
Leo Neat 4d5e5016c0
[Infra] Update print statements (#3305) 2020-01-29 14:10:04 -08:00
Leo Neat 8ffc6db00c
[Infra] CIFuzz pipeline complete. (#3281)
* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working fuzzers with out error surface

* Working fuzzers with out error surface

* Working fuzzers with out error surface

* Printing std err

* Adding fuzzer timeout

* Adding fuzzer timeout

* Changing fuzzer timeout to fuzz time

* Formatting and refactoring

* Spelling in fuzz_target.py

* Spelling in fuzz_target.py

* Spelling in fuzz_target.py

* Upload artifact fix

* Upload artifact fix

* Upload artifact fix

* Upload artifact fix

* Upload artifact fix

* Upload artifact fix

* Upload artifact fix

* Refactoring error codes.

* reverting helper.py

* reverting helper.py

* reverting helper.py

* chaning method to static

* moving cifuzz file

* Jonathan changes

* Oliver and Jonathan comments

* Oliver and Jonathan comments

* Oliver and Jonathan comments

* Utils unit tests

* Test formatting and documentation

* Build fuzzer test added

* Changed repo manager errors

* Unit and integration tests complete

* Jonathan comments pt.1

* Jonathan comments pt.1

* Jonathan comments pt.1

* adding cifuzz_test

* Build fuzzer test completed

* Run fuzzers test finished.

* Removed SRC dependency

* Jonathan comments pt.2

* Max comments pt.1

* Max comments pt.2

* removing log specified out stream

* Max comments pt.3

* Adding OSS_FUZZ_HOME env var

* Jonathan comments pt.3

* Formatting

* Olivers comments

* Jonathan comments
2020-01-29 11:03:43 -08:00
jonathanmetzman 2bcc2bfe2b
[cifuzz][bisection] Make BuildData a namedtuple (#3282) 2020-01-23 09:48:09 -08:00
Max Moroz 9918ef3f67
[infra] Make presubmit script handle experimental flag in project.yaml (#3277). (#3280)
* [infra] Make presubmit script handle experimental flag in project.yaml (#3277).

* make Travis happy

* address review feedback

* fix one more comment

* fix yaml file and address review feedback
2020-01-23 07:20:59 -08:00
Max Moroz a37f237ca5
[infra] Add fuzzing_engines section for the project.yaml. (#3267)
* [infra] Add fuzzing_engines section for the project.yaml.

* Update presubmit.py

* Update presubmit.py

* Update presubmit.py
2020-01-21 15:48:09 -08:00
Max Moroz 2c6c6d9785
[infra] Prioritize short files when collecting dataflow traces (#1632). (#3254)
* [infra] Prioritize short files when collecting dataflow traces (#1632).

* remove debug print

* rename files and sizes dict
2020-01-21 13:09:07 -08:00
Max Moroz 5dd3f4cebe
[infra] Add env variable to the DF collection GCB step (#1632). (#3250)
* [infra] Add env variable to the DF collection GCB step (#1632).

* format
2020-01-17 13:06:27 -08:00
Max Moroz f6002f6139
[infra] Collect dataflow traces on the builder (#1632). (#3238)
* [infra] Skeleton of the changes needed for collecting DFT on the builder (#1632).

* move ENGINE_INFO to the helper as well

* make collect_dft +x

* syntax fixes

* add actual dataflow tracer script

* format

* more refactoring and cleanup

* format

* address Oliver's feedback

* format

* more fixes

* format

* do not redirect stderr to stdout

* add exit at the end of main

* address feedback from Oliver
2020-01-17 06:24:15 -08:00
jonathanmetzman db6b6e7fd2 [infra] helper: fix regression introducing attribute error when reproducing crashes (#3246) 2020-01-16 13:54:23 -08:00
Max Moroz f22c0fbaac
[infra] dataflow: use basic block coverage instead of function (#1632). (#3239) 2020-01-15 21:40:58 -08:00
Max Moroz 3a9d7381e7
[infra] Print exact reproduction commands in build failures (#3104). (#3236)
* [infra] Print exact reproduction commands in build failures (#3104).

* syntax fixes

* remove redundant &&

* fix a typo

* use named arguments in format strings

* format
2020-01-15 14:12:22 -08:00
jonathanmetzman bf3592de30
[infra] Make helper.py presubmit-clean (#3237) 2020-01-15 13:37:07 -08:00
Leo Neat 14582175d0 [infra] Scripts for building fuzzers with CIFuzz (#3207) 2020-01-15 13:30:57 -08:00
Max Moroz b5abd91f20
[infra] Presubmit: remove license check for all .yaml files. (#3235) 2020-01-15 09:33:39 -08:00
Abhishek Arya f06afc1ebe
Fix presubmit parsing issue with emails. (#3234)
* Fix presubmit parsing issue with emails.

* Fix
2020-01-15 07:41:53 -08:00
Leo Neat cca04a14f4 [infra] Fix env variable in the helper.py repoduce command (#3230) 2020-01-14 10:50:35 -08:00
Oliver Chang 08c7c706bf
Build honggfuzz by default. (#3228) 2020-01-14 12:35:31 +11:00
Leo Neat 40fa9e5e9c [infra] CIFuzz - Add github action to OSS-Fuzz repo (#3214) 2020-01-13 15:25:12 -08:00
Max Moroz cbdc65515e
[infra] Fix dataflow engine compilation for a speed up (#1632). (#3219) 2020-01-12 11:43:03 -08:00
jonathanmetzman 363d00ba5f
[infra] Add presubmit script (#3196) 2020-01-10 10:19:42 -08:00
Max Moroz 12b5811ad0
[infra] Add parens to ensure correct execution order (#3104). (#3213) 2020-01-09 10:34:08 -08:00
Max Moroz cfe13cb116
[infra] Print additional information for build/coverage failures on GCB (#3104). (#3208)
* [infra] Explicily print failing build config in the GCB log (#3104).

* add trailing "false" command + add a message for misisng coprus backups in code coverage build

* improve the coverage message and behavior

* fix exit command

* use .format() for a single argument format string too

* fix typo
2020-01-09 09:41:58 -08:00
jonathanmetzman cec604f363
Fix issue running AFL targets with run_fuzzer (#3195) 2020-01-08 08:03:29 -08:00
jonathanmetzman 345d49adba
Reland Make bad_build_check more likely to catch hardcoding /out (#3180) (#3205)
This reverts commit 689c3f4b65.
2020-01-08 08:02:45 -08:00
Oliver Chang 84a2de2f84
Bump up rss limit to 2.5GB. (#3201) 2020-01-08 14:50:44 +11:00
jonathanmetzman 689c3f4b65
Revert "Make bad_build_check more likely to catch hardcoding /out (#3180)" (#3197)
Revert so that we don't get two errors in a row.

This reverts commit 14bdd9332c.
2020-01-07 08:23:13 -08:00
Leo Neat 9532d4781a [infra] Fixing bisection to work with more OSS-Fuzz projects (#3152)
* Max comments

* Added bisection tests

* Added test cases

* Docker run command added to build_specific commit

* Infer main repo uses docker image rather than docker file

* Added mores tests for repo infer

* With verbosity

* Formatting

* Bisection fixed

* Formatting updates

* Oliver small comments.

* Script infer main repo

* Detect repo and test module finished

* Detect main repo from script

* Regex searching

* Regex searching

* Max comments + bisector test script

* Oliver comments

* Max's comments

* String concat update

* Jonathan comments
2020-01-06 12:17:26 -08:00
jonathanmetzman 14bdd9332c
Make bad_build_check more likely to catch hardcoding /out (#3180) 2020-01-06 10:48:35 -08:00
Max Moroz 1463a2c523
[infra] Do not skip coverage build if there is no build.sh file (#3185). (#3189) 2020-01-06 07:45:45 -08:00
Oliver Chang 81c4656621
Run apt-get autoremove after removing honggfuzz dependencies. (#3172) 2020-01-02 13:24:21 +11:00