Commit Graph

166 Commits

Author SHA1 Message Date
Navidem 490a91a265
Enable Introspector on Python projects (#8945) 2022-11-07 22:48:16 +00:00
Navidem 5b2d1fa9ee
Increase cloud build timeout (#8924) 2022-11-04 05:24:25 +00:00
Navidem 165ee30241
Set introspector env variable in compile (#8878) 2022-10-28 12:06:15 -07:00
jonathanmetzman 998791514d
Get rid of :introspector images. Build introspector in main images (#8868)
Actually install introspector in every image.
Don't do extra introspector build steps.
Fixes https://github.com/google/oss-fuzz/issues/8625
2022-10-27 08:27:12 -04:00
jonathanmetzman 421a866ea7
[build-infra] bad build check using --privileged,--cap-add=all to enable netdriver (#8871)
May fix https://github.com/google/oss-fuzz/issues/8601
2022-10-27 08:19:21 -04:00
jonathanmetzman 16fb6232dc
Fix introspector trial builds (#8867) 2022-10-26 19:38:31 -04:00
Navidem f5e748bb84
Test OSS-Fuzz with introspector images [DO NOT MERGE] (#8628) 2022-10-25 16:31:04 -07:00
jonathanmetzman 871ea5240a
[trial-build] Add sleep to avoid rate limiting (#8819)
Sleep is appropriate because we're going to wait on builds which take
orders of magnitude longer than one second sleeps
2022-10-20 12:52:21 -04:00
jonathanmetzman f8a91ddfe7
Debug "failed to start build" (#8772) 2022-10-14 06:12:02 -04:00
Oliver Chang c278d2a190
Don't use GitHub API for project syncs. (#8589)
Download the zip and unzip it instead. There should be enough memory for
this on the tmpfs environment on Cloud Functions, but bump up the memory
to 4GB just in case.

Create a `OssFuzzRepo` which replicates the GitHub client library to
avoid changing too many things.

Fixes #8565
2022-09-26 17:46:18 +10:00
jonathanmetzman 1ab5e408e5
[trial builds] Support forks (#8519)
Fix https://github.com/google/oss-fuzz/issues/8273
2022-09-18 08:04:33 -04:00
Dongge Liu 2fa71e3c7f
Centipede's CI build, trial build, and build tests (#8422)
Adding CI build, trial build, and build tests.

Co-authored-by: Oliver Chang <oliverchang@users.noreply.github.com>
2022-09-16 09:25:49 +10:00
Dongge Liu 22488896f6
Centipede integration (#8046)
* Add Centipede as a fuzzer

* Specify dictionary param of Centipede

* Update docs

* Mark Centipede as experimental

* More accurate description

* Remove garbage

* Simplify code

* Move mkdir to dockerfile

* Add the weak.c trick

* Install deps with Centipede's script & uninstall new deps

* Fix doc

* Reuse libweak_sancov_stubs.so

* Reorganise flags

* format

* Consistent file type

* Reuse the weak references defined in Centipede

* Replace the shared library of weak symbols with a static one

* Correct the place to call mkdir

* Allow 2G of SHM for Centipede

* Create dirs in run_fuzzer

* Keep Centipede up-to-date

* Avoid duplicating Centipede's binary

* The params of Centipede and their explanations

* The engine info of centipede

* Save the target binary (with san) in a subdir of the project

* Set the target (with san) dir in check_build

* Create the target (with san) first to avoid side-effects

* Fic clone

* Fix format

* Add periods

* Fix comments

* Fix dirs

* Fix parameters

* Adding Centipede as a fuzzing engine for Scarecrow

* Add CI support

* Represent sanitizer with a variable

* Remove the unnecessary definition of FUZZER_OUT

* Reorganise binary directories

* format

* A minor note

* Present issues with dirs that alread exist

* Use os.path.join to join path

* Make a function to get the out/ in check build

* Reusing existing flags in .bazel

* Avoid hardcoding sanitizer, set rss_limit_mb=4096, leave address_space_limit_mb disabled

* Better ways to add bazel build options

* A better way to add bazel flags

* Remove redundant --bazelrc

* Better Cohesion

* Avoid code duplication

* Simplify code

* Exit on crash
2022-09-06 02:34:58 +00:00
jonathanmetzman d7b7141068
Fix ARM manifest building (#8349)
Related #8164
2022-08-31 15:18:21 +00:00
jonathanmetzman 3ce719bba2
[trial-build] Tag with branch name so trials are easier to tell apart. (#8338) 2022-08-23 17:42:59 -04:00
jonathanmetzman bbbbad5c07
Prepare for JavaScript fuzzing (#8325)
Prepare for JavaScript fuzzing
Related: #8324
2022-08-23 17:41:42 -04:00
jonathanmetzman 0fb63dfc7e
fix tests (#8334) 2022-08-23 14:11:30 -04:00
jonathanmetzman a2e5a2d2f6
[arm] Add manifest steps to base-images (#8271)
Related: #8164
2022-08-23 10:54:20 -04:00
Navidem 0b0c48d96a
Increase timeout (#8318) 2022-08-23 02:49:04 +00:00
Navidem 3e430be712
Revert "build(deps): update google-cloud-datastore requirement from <2.0 to <3.0 in /infra/build/functions" (#8316)
Revert "build(deps): update google-cloud-datastore requirement from <2.0 to <3.0 in /infra/build/functions (#7447)"

This reverts commit 7893dde4e7.
2022-08-22 16:54:35 -07:00
dependabot[bot] 7893dde4e7
build(deps): update google-cloud-datastore requirement from <2.0 to <3.0 in /infra/build/functions (#7447)
build(deps): update google-cloud-datastore requirement

Updates the requirements on [google-cloud-datastore](https://github.com/googleapis/python-datastore) to permit the latest version.
- [Release notes](https://github.com/googleapis/python-datastore/releases)
- [Changelog](https://github.com/googleapis/python-datastore/blob/main/CHANGELOG.md)
- [Commits](https://github.com/googleapis/python-datastore/compare/v0.21.0...v2.5.1)

---
updated-dependencies:
- dependency-name: google-cloud-datastore
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-17 14:27:33 -04:00
jonathanmetzman 9f150d831f
Fix trial build for coverage (#8268)
* Fix trial build for coverage

* Fix
2022-08-16 08:59:28 -04:00
jonathanmetzman 1749ae2b0f
Fix failing unittest. (#8262) 2022-08-15 19:34:34 -04:00
Navidem 84ffe92c14
Make sure projects list is sorted on the public index (#8225) 2022-08-11 18:23:56 -07:00
jonathanmetzman ddd25e3c3f
[AArch64] Support building on GCB (#8197)
Implemented using buildx to emulate aarch64 using QEMU
Related: #8164
2022-08-11 16:34:13 -04:00
jonathanmetzman 572a95538a
[trial-build] Test changes to base-runner (#8199)
* [trial-build] Test changes to base-runner

* fix test
2022-08-09 19:26:19 -04:00
jonathanmetzman 8ddcf0df18
Trial builds for fuzz introspector (#8111)
This will make it easier to test changes to introspector and to infra that affects introspector
2022-08-04 20:22:01 +00:00
jonathanmetzman 3bed5727dd
Fix test (#8119) 2022-08-01 21:00:00 -04:00
jonathanmetzman 093f61f727
Tag builds with image suffix to make clearer in UI (#8117) 2022-08-02 00:59:49 +00:00
jonathanmetzman b64ec26f8d
Use test-image (#8118) 2022-08-01 20:43:20 -04:00
jonathanmetzman 04c738d9bc
Fix trial builds (#8114) 2022-08-01 19:04:31 -04:00
jonathanmetzman c3495be8f4
Support concurrent trial builds. (#8113)
Do this by appending the branch to the test image suffix.
2022-08-01 18:47:44 -04:00
jonathanmetzman 68f2340a6a
[wycheproof] Add build infra support (#8041)
* Build infra

* [wycheproof] Add build infra support

* Fix

* undo
2022-07-20 14:34:19 -04:00
jonathanmetzman d50556b758
[NFC] Fix build URL (#8039)
Fix build URL
2022-07-20 14:25:14 -04:00
jonathanmetzman 54e2b96603
Remove build-infra support for DFSan. (#7971)
Related: #7964
2022-07-11 11:43:50 +02:00
jonathanmetzman c3bd295212
Fix dataflow builds (#7955) (#7963)
* Fix dataflow builds (#7955)

* undo fmt
2022-07-07 12:50:16 +02:00
Navidem 8988e2b8d8
Set PROJECT_NAME env and update download_corpora_step() (#7873) 2022-06-16 02:19:08 +00:00
Navidem cb65a0bbcd
Fix build image steps for introspector (#7859) 2022-06-14 21:02:08 +10:00
jonathanmetzman d93c0e5abb
[infra] Increase build timeout. (#7836)
Fixes #7695
2022-06-09 20:16:28 -04:00
jonathanmetzman 4734e40832
[trial_build] Fix bugs (#7764)
* Fix bugs

1. Don't try to build the script name as a project.
2. Add a flag to force builds of projects that previously failed.
3. Make sure we build projects from our PR branch.
4. Wait on all builds not just builds of the last type.
5. Don't use test bucket for corpus or coverage (will corpus work or fail because of creds?)

Add tests for these features.
2022-06-08 21:59:06 -04:00
Navidem 863f16e280
Give introspector builds a timeout same as other builds (#7794)
Give introspector buids same TO
2022-06-02 10:32:01 -05:00
Dongge Liu d75b3c28aa
pin the version of protobuf (#7766)
pin the version of `protobuf` to fix infra test errors.
2022-05-26 16:12:38 +10:00
jonathanmetzman a465bde1f6
Fix trial_build.py (#7752)
1. Make sure we build base images again.
2. Make sure we handle multiple build types properly.
3. Return nonzero on failure.
4. Make a better attempt at introspector builds.
Related: https://github.com/google/oss-fuzz/issues/7726
2022-05-24 17:58:56 -04:00
Oliver Chang 4407c1246e
Fix trial_build for coverage and introspector builds. (#7727)
Fixes #7726.
2022-05-20 05:04:15 +00:00
DavidKorczynski 8d0f7ef8d4
infra: reduce length of line to fix CI (#7722)
* infra: reduce length of line to fix CI

* update indentation
2022-05-19 13:31:55 -05:00
Navidem b22a30dbf4
Schedule introspector builds close to coverage builds (#7661)
* Schedule introspector builds close to coverage builds

* Spacing out the builds
2022-05-02 22:45:25 -05:00
Oliver Chang a75162d931
Fix introspector build from recent refactoring. (#7659) 2022-05-03 09:46:53 +10:00
Oliver Chang 2fe4371409
Restore machine type for base images. (#7652)
* Restore machine type for base images.

This was removed when moving project builds to private pools.

* format
2022-05-02 18:10:33 +10:00
Oliver Chang 03719efa0a
Fix base image builds. (#7651)
Fixes #7650.
2022-05-02 11:26:44 +10:00
jonathanmetzman aab23a8fb8
Roll AFL++ (#7595)
* Roll AFL++

* Adapt to us-central1

* fix

* test faster

* All projects

* all

* All

* fix

* start small

* bigger

* print dbg

* handle sanitizers

* handle sanitizers

* all means all

* handle flake

* Fix

* fix
2022-04-26 23:46:15 +00:00