Commit Graph

1895 Commits

Author SHA1 Message Date
AdamKorcz 099fa2e5f5
infra: bump Go to 1.19 (#8568)
Most projects have upgraded to 1.19 which is the latest version.

This upgrades Go to 1.19 in the base-image.

Signed-off-by: AdamKorcz <adam@adalogics.com>

Signed-off-by: AdamKorcz <adam@adalogics.com>
2022-09-22 11:59:15 -07:00
jonathanmetzman 95d5e613e7
[CFL] Support parallel fuzzing (#8396)
Related: https://github.com/google/clusterfuzzlite/issues/105
2022-09-22 11:21:35 -04:00
John Howard 880dba286e
Always add `testing` package usage in native_go_fuzzer (#8546)
This fixes an issue where the `testing.F` is replaced with a new
package; if this was the only usage of `testing` in the file then we end
up with `testing` being an unused import. A workaround for this is to
just always use `testing` via some placeholder variable.
2022-09-21 10:49:21 -07:00
Dongge Liu 2757ed6601
Pin Centipede to a more recent version (#8543)
A recent commit allows `Centipede` to add timestamps to its log, which
can be very helpful for debugging.
This PR intends to update `Centipede` to add that commit.
2022-09-21 15:02:44 +10:00
Fabian Meumertzheim 7765e4d548
infra: Update Jazzer to fix coverage build failures (#8495)
Fixes #8241
2022-09-20 13:37:22 -04:00
van Hauser c8a9c23666
update afl++ commit id (#8216)
this fixes a bug that affects coverage.

Co-authored-by: jonathanmetzman <31354670+jonathanmetzman@users.noreply.github.com>
2022-09-19 18:48:32 -04:00
Thomas Van Lenten 12c881f772
Use Swift 5.7.0. (#8501)
Swift 5.4.2 was released back in June 2021.

Co-authored-by: jonathanmetzman <31354670+jonathanmetzman@users.noreply.github.com>
2022-09-19 18:40:00 -04:00
jonathanmetzman 2301679632
[CI] Allow projects to opt-out of running tests. (#8518)
Fixes: https://github.com/google/oss-fuzz/issues/8508
2022-09-19 10:21:43 +10:00
jonathanmetzman 1ab5e408e5
[trial builds] Support forks (#8519)
Fix https://github.com/google/oss-fuzz/issues/8273
2022-09-18 08:04:33 -04:00
jonathanmetzman c0f72989f4
[centipede][base-builder] Properly delete bazel cache (#8494)
Fixes #8468

Co-authored-by: Alan32Liu <donggeliu@google.com>
2022-09-16 13:17:26 +10:00
Dongge Liu 97a68b049b
Update Centipede to fix SegFault (#8507)
Update `Centipede` to fix [the SegFault when using `AddressSanitizer`
with
`address_space_limit_mb`](https://github.com/google/centipede/issues/166).
Also set `address_space_limit_mb` as the issue has been fixed.
2022-09-16 12:33:20 +10:00
Navidem 009a098795
Bump Introspector to pick up latest updates (#8506) 2022-09-16 10:57:10 +10:00
Dongge Liu 2fa71e3c7f
Centipede's CI build, trial build, and build tests (#8422)
Adding CI build, trial build, and build tests.

Co-authored-by: Oliver Chang <oliverchang@users.noreply.github.com>
2022-09-16 09:25:49 +10:00
Navidem b13cf7a510
Bump introspector (#8498) 2022-09-15 01:12:21 +00:00
Dongge Liu 46cc569049
Do not remove the symlink as they are used by `Centipede`'s `Bazel` build (#8486) 2022-09-14 02:54:49 +00:00
jonathanmetzman 27b929d05b
Don't install git via ppa (#8470)
Reduce complexity and bloat
2022-09-13 08:37:38 +01:00
DavidKorczynski dc5adbf754
infra: add support for netdriver fuzzing with honggfuzz (#7351)
Adds an example with the mongoose webserver
2022-09-12 11:58:12 +10:00
Catena cyber ede1cc8a60
SystemSan: more logs for arbitrary file open (#8432)
cc @oliverchang 

Log the file trying to be opened and the flags (read or write) for
opening the file

Co-authored-by: Oliver Chang <oliverchang@users.noreply.github.com>
2022-09-12 01:55:03 +00:00
jonathanmetzman d50dacbfb4
Don't install clang-repl (#8471)
It's unneeded and adds 100MB of unneeded bloat.
2022-09-12 09:39:20 +10:00
jonathanmetzman d3e4ba3bb8
[centipede] Fix centipede (#8467)
Pin centipede to a commit that builds.
Also remove 1.3GB of image bloat it adds
CC @ussuri @Alan32Liu 
Fixes #8458
2022-09-11 13:20:02 -04:00
Navidem b8ef6a216d
Bump Introspector (#8436) 2022-09-08 01:16:05 +00:00
Oliver Chang 91acda2611
Centipede fixes. (#8417)
- Replace newlines with spaces in centipede/clang-flags.txt. Build
  systems don't like them.

- Fix a missing newline in output logs.
2022-09-06 07:41:13 +00:00
Dongge Liu 22488896f6
Centipede integration (#8046)
* Add Centipede as a fuzzer

* Specify dictionary param of Centipede

* Update docs

* Mark Centipede as experimental

* More accurate description

* Remove garbage

* Simplify code

* Move mkdir to dockerfile

* Add the weak.c trick

* Install deps with Centipede's script & uninstall new deps

* Fix doc

* Reuse libweak_sancov_stubs.so

* Reorganise flags

* format

* Consistent file type

* Reuse the weak references defined in Centipede

* Replace the shared library of weak symbols with a static one

* Correct the place to call mkdir

* Allow 2G of SHM for Centipede

* Create dirs in run_fuzzer

* Keep Centipede up-to-date

* Avoid duplicating Centipede's binary

* The params of Centipede and their explanations

* The engine info of centipede

* Save the target binary (with san) in a subdir of the project

* Set the target (with san) dir in check_build

* Create the target (with san) first to avoid side-effects

* Fic clone

* Fix format

* Add periods

* Fix comments

* Fix dirs

* Fix parameters

* Adding Centipede as a fuzzing engine for Scarecrow

* Add CI support

* Represent sanitizer with a variable

* Remove the unnecessary definition of FUZZER_OUT

* Reorganise binary directories

* format

* A minor note

* Present issues with dirs that alread exist

* Use os.path.join to join path

* Make a function to get the out/ in check build

* Reusing existing flags in .bazel

* Avoid hardcoding sanitizer, set rss_limit_mb=4096, leave address_space_limit_mb disabled

* Better ways to add bazel build options

* A better way to add bazel flags

* Remove redundant --bazelrc

* Better Cohesion

* Avoid code duplication

* Simplify code

* Exit on crash
2022-09-06 02:34:58 +00:00
jonathanmetzman d7b7141068
Fix ARM manifest building (#8349)
Related #8164
2022-08-31 15:18:21 +00:00
Dongge Liu 7286a718e1
Add `architecture` to `run_fuzzer_parser`'s namespace (#8377)
Add architecture to run_fuzzer's namespace
2022-08-31 08:09:20 -04:00
jonathanmetzman eef03e0cc6
[aarch64] Add ARM support to helper.py (#8332)
This is probably a lot of what is needed for CFL support (though there's probably a lot of other stuff left).
Related: #8164
2022-08-30 16:28:37 -04:00
Oliver Chang 5675ec6958
Rename execSan to SystemSan. (#8369)
* Rename execSan to SystemSan.

All of the bug detectors we've built (or plan to build) relate to system
state.

* fix documentation

* fix more documentation
2022-08-30 08:40:46 +10:00
jonathanmetzman dabee12f18
Roll clang to llvmorg-15-init-1464-gbf7f8d6f (#8313)
* Roll clang to llvmorg-15-init-1464-gbf7f8d6f

This is incremental roll meant to break less than #8108

* Pin some projects

* pin cryptofuzz
2022-08-24 10:31:28 -04:00
jonathanmetzman 3ce719bba2
[trial-build] Tag with branch name so trials are easier to tell apart. (#8338) 2022-08-23 17:42:59 -04:00
jonathanmetzman bbbbad5c07
Prepare for JavaScript fuzzing (#8325)
Prepare for JavaScript fuzzing
Related: #8324
2022-08-23 17:41:42 -04:00
jonathanmetzman a335a00b84
[infra] Add script for pinning images when updating builders. (#8328) 2022-08-23 17:39:33 -04:00
jonathanmetzman 0fb63dfc7e
fix tests (#8334) 2022-08-23 14:11:30 -04:00
jonathanmetzman a2e5a2d2f6
[arm] Add manifest steps to base-images (#8271)
Related: #8164
2022-08-23 10:54:20 -04:00
Navidem 0b0c48d96a
Increase timeout (#8318) 2022-08-23 02:49:04 +00:00
Navidem 3e430be712
Revert "build(deps): update google-cloud-datastore requirement from <2.0 to <3.0 in /infra/build/functions" (#8316)
Revert "build(deps): update google-cloud-datastore requirement from <2.0 to <3.0 in /infra/build/functions (#7447)"

This reverts commit 7893dde4e7.
2022-08-22 16:54:35 -07:00
dependabot[bot] 7893dde4e7
build(deps): update google-cloud-datastore requirement from <2.0 to <3.0 in /infra/build/functions (#7447)
build(deps): update google-cloud-datastore requirement

Updates the requirements on [google-cloud-datastore](https://github.com/googleapis/python-datastore) to permit the latest version.
- [Release notes](https://github.com/googleapis/python-datastore/releases)
- [Changelog](https://github.com/googleapis/python-datastore/blob/main/CHANGELOG.md)
- [Commits](https://github.com/googleapis/python-datastore/compare/v0.21.0...v2.5.1)

---
updated-dependencies:
- dependency-name: google-cloud-datastore
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-08-17 14:27:33 -04:00
Oliver Chang d412c6b08b
Pin Jazzer to older revision (#8275)
All latest builds seem to be broken with:

```
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
INFO: Loaded 112 hooks from com.code_intelligence.jazzer.runtime.TraceCmpHooks
INFO: Loaded 4 hooks from com.code_intelligence.jazzer.runtime.TraceDivHooks
INFO: Loaded 2 hooks from com.code_intelligence.jazzer.runtime.TraceIndirHooks
INFO: Loaded 4 hooks from com.code_intelligence.jazzer.runtime.NativeLibHooks
INFO: Loaded 8 hooks from com.code_intelligence.jazzer.sanitizers.Deserialization
INFO: Loaded 5 hooks from com.code_intelligence.jazzer.sanitizers.ExpressionLanguageInjection
INFO: Loaded 70 hooks from com.code_intelligence.jazzer.sanitizers.LdapInjection
INFO: Loaded 46 hooks from com.code_intelligence.jazzer.sanitizers.NamingContextLookup
INFO: Loaded 1 hooks from com.code_intelligence.jazzer.sanitizers.OsCommandInjection
INFO: Loaded 52 hooks from com.code_intelligence.jazzer.sanitizers.ReflectiveCall
INFO: Loaded 8 hooks from com.code_intelligence.jazzer.sanitizers.RegexInjection
Exception in thread "main" java.lang.ExceptionInInitializerError
  at com.code_intelligence.jazzer.sanitizers.RegexRoadblocks.<clinit>(RegexRoadblocks.java:72)
  at java.base/java.lang.Class.forName0(Native Method)
  at java.base/java.lang.Class.forName(Class.java:398)
  at com.code_intelligence.jazzer.instrumentor.Hooks$Companion$HooksLoader.loadHooks(Hooks.kt:66)
  at com.code_intelligence.jazzer.instrumentor.Hooks$Companion$HooksLoader.load(Hooks.kt:48)
  at com.code_intelligence.jazzer.instrumentor.Hooks$Companion.loadHooks(Hooks.kt:42)
  at com.code_intelligence.jazzer.agent.Agent.premain(Agent.kt:137)
  at com.code_intelligence.jazzer.driver.Driver.start(Driver.java:97)
Caused by: java.lang.NumberFormatException: For input string: "./jazzer_driver"
  at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:66)
  at java.base/java.lang.Long.parseLong(Long.java:680)
  at java.base/java.lang.Integer.parseUnsignedInt(Integer.java:835)
  at java.base/java.lang.Integer.parseUnsignedInt(Integer.java:929)
  at com.code_intelligence.jazzer.api.Jazzer.getLibFuzzerSeed(Jazzer.java:635)
  at com.code_intelligence.jazzer.api.Jazzer.<clinit>(Jazzer.java:38)
  ... 8 more
```
2022-08-17 19:21:53 +10:00
Paulo Gomes 0b416da224
native go fuzzing: Restrict fuzzer lookup to `*.go` (#8238)
native go fuzzing: Restrict fuzzer lookup to *.go
Projects containing words starting with Fuzz in non-go
files break compile_native_go_fuzzer with the error:
sed: -e expression #1, char 15: unknown option to 's'

Signed-off-by: Paulo Gomes <pjbgf@linux.com>

Signed-off-by: Paulo Gomes <pjbgf@linux.com>
2022-08-16 20:03:34 -07:00
jonathanmetzman c113501247
Upgrade to codeqlv2 to avoid warning (#8272)
* Upgrade to codeqlv2 to avoid warning

* debug
2022-08-16 17:08:56 -07:00
jonathanmetzman 9f150d831f
Fix trial build for coverage (#8268)
* Fix trial build for coverage

* Fix
2022-08-16 08:59:28 -04:00
jonathanmetzman 1749ae2b0f
Fix failing unittest. (#8262) 2022-08-15 19:34:34 -04:00
jonathanmetzman 04a35a201b
[arm] Add manifest script (#8223)
Add script to make manifests pointing to docker images for different
platforms.
Related: https://github.com/google/oss-fuzz/issues/8164
2022-08-12 09:04:24 -04:00
Navidem 84ffe92c14
Make sure projects list is sorted on the public index (#8225) 2022-08-11 18:23:56 -07:00
jonathanmetzman ddd25e3c3f
[AArch64] Support building on GCB (#8197)
Implemented using buildx to emulate aarch64 using QEMU
Related: #8164
2022-08-11 16:34:13 -04:00
jonathanmetzman 468e1c08ec
[aarch64] Set random_tags=0 (#8201)
Set random_tags=0
2022-08-10 04:08:18 +00:00
jonathanmetzman 572a95538a
[trial-build] Test changes to base-runner (#8199)
* [trial-build] Test changes to base-runner

* fix test
2022-08-09 19:26:19 -04:00
jonathanmetzman 29848637e2
Fix bad build check breakage (#8198)
fix
2022-08-09 18:52:46 -04:00
jonathanmetzman f02edd2c9a
Support C/C++ projects on AArch64. (#8184)
Related: #8164
2022-08-09 10:18:06 -04:00
jonathanmetzman a4c1193050
Support building base-runner on aarch64. (#8186)
Related: #8164
2022-08-09 04:10:55 +00:00
jonathanmetzman 97133fd942
[Incomplete] Support building targets with HWAsan (#8190)
Note that these targets still fail immediately.
Related: #8164
2022-08-08 22:57:59 -04:00