Commit Graph

1147 Commits

Author SHA1 Message Date
Abhishek Arya 65cd35731f
Use symlink instead of broken -path-equivalence (#4610) 2020-11-05 23:18:21 -08:00
jonathanmetzman 8daba1a1a1
[CIFuzz] Don't assume fuzzer output is ASCII (#4537)
[CIFuzz] Don't assume fuzzer output is ascii

This is wrong because the output can be arbitrary. Instead change
code that deals with the output to deal with bytes. The testcase,
which is derived from the output can be decoded as UTF-8 since it
will be a unix path.
2020-10-19 10:10:22 -07:00
mbarbella-chromium 0deeef6e31
Revert "Python SyntaxError: leading zeros in decimal integer literals (#4523)" (#4525)
This reverts commit fa0c1b4c6d.
2020-10-12 08:52:37 -07:00
mbarbella-chromium 2b2f5c504f
Revert "nettle.py: SyntaxError leading zeros in decimal integer literals (#4522)" (#4524)
This reverts commit 6fe03eafb5.
2020-10-12 08:50:38 -07:00
Christian Clauss 6fe03eafb5
nettle.py: SyntaxError leading zeros in decimal integer literals (#4522)
* SyntaxError: leading zeros in decimal integer literals are not permitted; use an 0o prefix for octal integers

* Placate pylint
2020-10-12 08:44:44 -07:00
Christian Clauss fa0c1b4c6d
Python SyntaxError: leading zeros in decimal integer literals (#4523)
Python SyntaxError: leading zeros in decimal integer literals are not permitted; use an 0o prefix for octal integers
2020-10-12 08:41:52 -07:00
jonathanmetzman 449ef28a87
[CIFuzz] Retry HTTP requests on certain errors (#4516)
Skia has seen some unhandled connection reset errors.
When we see these errors when downloading old OSS-Fuzz build, retry downloading it, since the error is likely transient.
2020-10-12 07:50:38 -07:00
Catena cyber f9f99a9604
Enables MSAN for Suricata (#4455)
* Enables MSAN for Suricata

* Move rustup std lib download to base-builder
2020-09-30 07:04:22 -07:00
Oliver Chang 97136f87e4
Set main repo for clamav. (#4466) 2020-09-23 10:41:20 +10:00
Oliver Chang 34751cf016
bisector: early exit when old_commit == new_commit. (#4458)
But do this after computing repo_url as that's still useful information.
2020-09-18 14:10:10 +10:00
Oliver Chang 460a10836b
Update Dockerfile
Fixes #4400
2020-09-09 11:53:44 +10:00
Max Moroz e2cd61f00a
[infra] Re-enable clang/LLVM auto-updating (#4348). (#4426) 2020-09-08 17:56:33 -07:00
Oliver Chang a8263735f5
Add a blackbox field to project.yaml (#4405)
This is needed for CF to determine whether or not to allocate the
project on a higher end bot to run the blackbox fuzzer.

Previously this was keyed on the "none" entry under fuzzing_engines,
which wasn't very descriptive. This change also lets us do both blackbox
and greybox fuzzing in the same project, which will be done for quickjs.
2020-09-03 11:08:09 +10:00
jonathanmetzman db8467bf30
[CI] Don't do coverage builds for engineless fuzzers. (#4374)
Coverage builds need to be special cased since they aren't specified
in sanitizers. Instead they are done for all C/C++ projects that
use libFuzzer.

Move all of this special casing to `should_build_coverage` and call that from
`should_build` so we have one place where we decide this.
Add tests as well.

Fixes: https://github.com/google/oss-fuzz/issues/4371
2020-08-26 11:59:56 -07:00
jonathanmetzman 13f7ba0f63
[presubmit] Allow 'none' as a fuzzing engine (#4370)
'none' is used by projects like spidermonkey that use the JS fuzzer.
2020-08-25 16:31:29 -07:00
Max Moroz eabcccf848
[infra] Fix reproduce command invoking run_fuzzer for a single testcase (#4338). (#4367)
* [infra] Fix reproduce command invoking run_fuzzer for a single testcase (#4338).

* review feedback

* do not introduce new variable, rely on SKIP_SEED_CORPUS only
2020-08-25 11:38:50 -07:00
Abhishek Arya a02946e8f4
Revert chromium clang roll to fix mismatched coverage data error (#4353)
Temporarily fixes #4348
2020-08-21 22:18:24 -07:00
Abhishek Arya 4628ee31a9
Revert "Revert "Compile + install LLVMgold.so so -flto works (#4332)" (#4351)" (#4352)
This reverts commit 7f8f5e0182.
2020-08-21 22:12:46 -07:00
Abhishek Arya 7f8f5e0182
Revert "Compile + install LLVMgold.so so -flto works (#4332)" (#4351)
This reverts commit d1ddd25937.
2020-08-21 16:52:23 -07:00
van Hauser d1ddd25937
Compile + install LLVMgold.so so -flto works (#4332) 2020-08-18 09:55:08 -07:00
Max Moroz d250f4ffbb
[infra] Upgrade reproduce command to re-use run_fuzzer functionality. (#4324)
* [infra] Upgrade reproduce command to re-use run_fuzzer functionality.

* fix a typo

* add missing quotes
2020-08-17 11:34:21 -07:00
Max Moroz a986d76ef4
[infra] Use 16 threads when building LLVM (#4270). (#4315)
* [infra] Use 16 threads when building LLVM (#4270).

* update the comment
2020-08-13 12:58:02 -07:00
Max Moroz cec66e9662
[infra] base-clang: limit the number of build threads to 4 (#4270). (#4311) 2020-08-12 16:44:29 -07:00
Max Moroz 8712207990
[infra] Make bad_build_check to use run_fuzzer when testing startup crash. (#4301)
* [infra] Make bad_build_check to use run_fuzzer when testing startup crash.

* use SKIP_SEED_CORPUS=1
2020-08-12 09:57:15 -07:00
Max Moroz e4d3fb0f8c
[infra][docs] Be more explicit about code coverage being supported for C/C++ only (#4284). (#4303)
* [infra][docs] Be more explicit about code coverage being supported for C/C++ only (#4284).

* fix typos and pass env variable
2020-08-12 09:32:05 -07:00
Oliver Chang 0987ddf994
even more fixes. (#4297) 2020-08-11 18:27:18 +10:00
Oliver Chang ade417ba20
Fix typo in project_sync.py (#4296)
* Fix typo in project_sync.py

* line length

* format
2020-08-11 16:25:08 +10:00
Abhishek Arya 103aca2e27
Add |builds_per_day| to supported section in project.yaml. (#4291) 2020-08-10 08:08:25 -07:00
Max Moroz 941ada694b
[infra] Add support for custom sanitizer options in run_fuzzer. (#4274)
* [infra] Add support for custom sanitizer options in run_fuzzer.

* address review feedback

* fix presubmit warnings
2020-08-07 12:48:23 -07:00
Abhishek Arya 751aa78373
Fix LLVM_REVISION parse regex (#4275)
Fix parse regex as per format change in
e6863f8783%5E%21/#F1
2020-08-06 20:12:58 -07:00
jonathanmetzman 571dbfb39f
Reland 55d9a81. (#4269)
The issue with 55d9a81 that caused it to be reverted by 3370f8f
was probably due to the build trigger on GCB being stale. I've
updated it so this commit should stick.
2020-08-06 14:33:08 -07:00
Evgeny Vereshchagin 3370f8f7e5
Revert "[CIFuzz] Allow supplying a manual source checkout (#4250)" (#4263)
This reverts commit 55d9a81cd7.

Closes https://github.com/google/oss-fuzz/issues/4262
2020-08-06 06:11:59 -07:00
Oliver Chang 65a83e2125
Split up builds status function. (#4261)
- Split into separate invocations for fuzzing, coverage, and badges.
- Also use threads to speed things up.
2020-08-06 13:59:08 +10:00
jonathanmetzman 55d9a81cd7
[CIFuzz] Allow supplying a manual source checkout (#4250)
Make CIFuzz building accept an env var `MANUAL_SRC_PATH` that points to a manually prepared checkout of the project-under-tests's source code. This allows projects like Skia which are not on OSS-Fuzz and/or need to make changes to the repo after checking out the right commit to use CIFuzz.
Note that for now we aren't supporting this in GitHub, so projects that need to modify the source after checkout can't use it until
we do support it.

Also, use the local copy of OSS-Fuzz when building cifuzz-base instead of cloning it from GitHub. This makes local debugging/development much easier since it allows one to use CIFuzz with local changes.
2020-08-05 18:45:12 -07:00
Oliver Chang 909db51a15
More build infra cleanup. (#4252)
- Deleted unused code.
- Consolidate some modules.
2020-08-06 09:36:47 +10:00
Oliver Chang 3f1d43115f
Clean up build infra. (#4251)
- Remove gcb folder. Move the files that we still need into the
  build/functions dir.
- Remove badge_images. They are now stored on GCS.
- Remove jenins configs.
2020-08-05 11:20:48 +10:00
kabeer27 21cd8de8dc
Skipping fuzzing build when project is disabled (#4239)
* Skipping fuzzing build when project is disabled

* Removing skip build and making it better
2020-07-31 17:20:09 +10:00
Oliver Chang 47b57ca1dd
Fix a typo from previous MSan function addition. (#4241) 2020-07-31 16:58:28 +10:00
Oliver Chang 95e66c619e
Move cmake install to base-clang. (#4240)
Clang now needs a newer cmake to build.
2020-07-31 16:55:30 +10:00
kabeer27 8e4c7b92fd
Adding msan builder to gcp (#4234)
* Adding msan builder to gcp

* Formatting changes

* Refactoring and reducing redundancy

* Moving msan builder entry point into base_images
2020-07-31 14:42:09 +10:00
Oliver Chang 5e3348cdb9
Create individual schedulers for requesting coverage builds. (#4237)
The single function for requesting all coverage builds was timing out.
2020-07-31 10:56:18 +10:00
kabeer27 a622daf7f5
better fix for template (#4226) 2020-07-30 18:01:07 +10:00
kabeer27 341b3d836a
Display historical logs ui change (#4197)
* Initial UI Change to display historical logs

* Minor changes

* More formatting changes

* Adding license header

* Changes, take a look Oliver

* Changes for handling empty build history/ no last successful build

* Reverting the logs folder path from /logs/log to /log

* Merged to latest master and other chnages to template

* Changing backend to reflect build history

* Fixing order of query

* Removed last build status

* More changes

* Adding unit tests and fixes found by unit testing

* Fixing lint errors

* Added more unit tests and fixed typos and formatting

* Fixing comment caps

* Minor formatting issue

Co-authored-by: Oliver Chang <oliverchang@users.noreply.github.com>
2020-07-30 16:20:22 +10:00
Oliver Chang ccb5821f51
Fix build status update. (#4224)
Check for valid statuses rather than only excluding the WORKING status.
There are other failure statuses that need to be accounted for (e.g.
EXPIRED).
2020-07-30 15:10:52 +10:00
Oliver Chang 689f3500a2
Add missing dash to build tag. (#4221) 2020-07-30 08:31:01 +10:00
Oliver Chang ac4af2436f
build: Add queueTtl argument to one more place. (#4219) 2020-07-30 08:21:49 +10:00
Tom Ritter 284dad011e
[rnp] Add afl engine (#4212)
* [rnp] Add afl engine

* Add fuzz_dump and fuzz_keyring to the exception list

These binaries - from https://github.com/rnpgp/rnp/tree/master/src/fuzzing -
are very tiny and just call into the library with a single API,
and are not staticly compiled - so they have very few UBSAN calls.
2020-07-29 09:56:23 -07:00
kabeer27 5893c61c77
Fixing missing dash in datastore query (#4213) 2020-07-29 17:32:54 +10:00
Oliver Chang eb19f16419
Add queueTtl argument to cloud build request. (#4211)
To prevent expiry of queued builds.
2020-07-29 14:50:46 +10:00
Oliver Chang 7bb74407dc
Fix an Python 3 exception in build_project.py. (#4210) 2020-07-29 13:04:59 +10:00