Commit Graph

361 Commits

Author SHA1 Message Date
jonathanmetzman a27bbce6fb
Set up coverage build properly (#9654) 2023-02-08 21:42:29 -05:00
jonathanmetzman 2be77b9dc2
Reland #9646 (#9652)
It shouldn't fail anymore now that the image is pushed. But just to be
safe, I removed it from the images.
2023-02-08 19:05:19 -05:00
Oliver Chang e59b5e26c4
Revert "Get OSS-Fuzz on demand fuzzing working" (#9651)
Reverts google/oss-fuzz#9646

Reverting this since this is breaking users :/ 

https://github.com/google/oss-fuzz/issues/9647
2023-02-09 10:54:49 +11:00
jonathanmetzman f41ccbb534
Get OSS-Fuzz on demand fuzzing working (#9646) 2023-02-08 13:59:23 -05:00
jonathanmetzman e39ad0f219
Implement building in OSS-Fuzz on Demand (#9633) 2023-02-08 12:34:13 -05:00
jonathanmetzman c2029fd019
[infra][NFC] Silence some useless logging (#9617) 2023-02-07 04:20:31 +00:00
jonathanmetzman f750fddc97
Reduce size of swift image (#9614) 2023-02-06 23:04:26 -05:00
jonathanmetzman a75aab95ec
Fix centipede compilation (#9613)
1. Don't use trace-loads as it is not fully supported in LLVM and breaks
many projects (see
9383870160)
2. Use -Wl for link time options so they aren't unnecessarily used (also
fixes meson based projects).
2023-02-06 22:33:23 -05:00
jonathanmetzman 9bbc9ad236
[NFC] Don't print output when untarring honggfuzz (#9616)
Waste of space in the 5MB limited logs.
2023-02-06 22:21:59 -05:00
jonathanmetzman 54448052e0
Don't use -fsanitize=fuzzer-no-link with centipede (#9610)
It's pointless and has these drawbacks:
1. It add useless instrumentation not used by centipede that slows down
the target
2. It adds instrumentation that expects an implementation at link time
(__sancov_lowest_stack) which can break projects builds.

Fixes https://github.com/google/oss-fuzz/issues/9609
Related: https://github.com/google/oss-fuzz/issues/9299
https://github.com/google/oss-fuzz/issues/9583
2023-02-06 21:19:45 -05:00
Khaled Yakdan dc2f5f4be8
Integrate Jazzer.js (#9466)
This PR enables using Jazzer.js for fuzzing Node.js projects in
OSS-Fuzz.

Part of #8324

---------

Co-authored-by: jonathanmetzman <31354670+jonathanmetzman@users.noreply.github.com>
2023-01-31 11:31:54 -05:00
Dongge Liu e1529c3ee3
Update Centipede to eb91dd2 (#9426)
Update Centipede to [its latest commit
eb91dd2](eb91dd2157),
which added some new features and fixed runtime bugs found in recent
FuzzBench experiments.
2023-01-16 13:14:45 +11:00
jonathanmetzman 11a21c2b6a
[centipede] Turn off warnings during compilation. (#9301)
Related: https://github.com/google/oss-fuzz/issues/9299
2023-01-03 11:44:12 -05:00
AdamKorcz ba8bea455a
native go infra: switch to main branch (#9103)
Fixes:
https://github.com/google/oss-fuzz/pull/8937#discussion_r1021005743

Signed-off-by: AdamKorcz <adam@adalogics.com>

Signed-off-by: AdamKorcz <adam@adalogics.com>
Co-authored-by: Oliver Chang <oliverchang@users.noreply.github.com>
2022-12-20 11:32:37 +11:00
Khaled Yakdan b7ee58b26c
infra: remove temporary build image for Go (#9230)
Upstream Go 1.19 already contains all fuzzing improvements implemented
in the used Go fork.
2022-12-19 13:00:55 +11:00
DavidKorczynski f1593612c7
infra: bump and add java support in introspector (#9208)
Adds introspector support for java builds. Most of this work is
referenced by way of
https://github.com/ossf/fuzz-introspector/issues/536

Also bumps introspector, which comes with a fix for
https://github.com/ossf/fuzz-introspector/issues/679

Signed-off-by: David Korczynski <david@adalogics.com>

Signed-off-by: David Korczynski <david@adalogics.com>
Co-authored-by: Navidem <navid.emamdoost@gmail.com>
2022-12-18 15:51:44 -08:00
Navidem 35d2567c9c
Add analytic tag for FuzzIntrospector. (#9178)
Fixes #9039.
2022-12-11 17:08:49 +11:00
DavidKorczynski ebe05f04ce
infra: add project name to python FI reports (#9174)
Fixes: https://github.com/ossf/fuzz-introspector/issues/674
2022-12-09 11:27:15 -08:00
DavidKorczynski 529a009be5
infra: refine fuzztest work (#8992)
Enable Fuzztest fuzzers for Tensorflow.

This depends on https://github.com/google/fuzztest/pull/79 and
eventually a PR on the tensorflow repo with the (to be refined) diff in
this PR.

Signed-off-by: David Korczynski <david@adalogics.com>

Signed-off-by: David Korczynski <david@adalogics.com>
Co-authored-by: Oliver Chang <oliverchang@users.noreply.github.com>
2022-11-28 18:17:41 +11:00
Navidem 41f9210a29
Revert "[infra] Update Jazzer to latest main" (#9055)
Reverts google/oss-fuzz#9027
per @fmeum request.
2022-11-24 02:21:20 +00:00
Dongge Liu 3ba9eabc27
Update centipede (#9029)
Update `Centipede` to its latest version.
2022-11-23 10:01:38 +11:00
Fabian Meumertzheim 33c5ce4d6f
[infra] Update Jazzer to latest main (#9027)
Reuses the previous Jazzer file names so that no changes to ClusterFuzz
are required.
2022-11-22 09:23:42 -05:00
Dongge Liu 6a4ecab2fb
Disable -Werror on unused-command-line-argument (#9030)
Some projects use `-Werror` to turn all warnings into errors.
This affects `Centipede` as we do not separate build and linking flags
as it expects, which leads to `unused-command-line-argument` warnings.
This PR disables turning that specific warning into errors and keeps the
rest the same.
2022-11-22 21:29:33 +11:00
AdamKorcz 63c99caae9
Refactor infra of Go 1.18 fuzzers (#8937)
`go-118-fuzz-build` is undergoing refactoring to fix some runtime issues
affecting Rekor and other projects. Currently none of Rekors fuzzers
run. This PR has been tested on all of Rekors fuzzers that will run with
this PR.

This PR modifies the infra OSS-Fuzz build script to adapt to the
upstream changes.

Cloning the `dev` branch of `go-118-fuzz-build` is a temporary solution
until it gets merged into `main` upstream.
Cloning go-118-fuzz-build in each build script is also a temporary
measure until `dev` gets merged into `main`.

Signed-off-by: AdamKorcz <adam@adalogics.com>

Signed-off-by: AdamKorcz <adam@adalogics.com>
2022-11-21 17:55:11 +00:00
DavidKorczynski 216ec0b8a4
infra: base-builder: refine fuzz-introspector building (#9004)
- Pass coverage_url to both c/c++ and python report generation.
- Remove a double calling of introspector report generation.

Signed-off-by: David Korczynski <david@adalogics.com>

Signed-off-by: David Korczynski <david@adalogics.com>
2022-11-21 07:21:59 -08:00
jonathanmetzman 3ebdb071dc
Remove centipede's .git (#8986)
It takes up space, is uneeded and causes irrelevant data to be displayed
in revisions.
2022-11-16 07:53:04 -05:00
DavidKorczynski 9452275da3
fuzztest integration (#8784)
Signed-off-by: David Korczynski <david@adalogics.com>

Cross referencing https://github.com/google/fuzztest/pull/69

Signed-off-by: David Korczynski <david@adalogics.com>
Co-authored-by: Oliver Chang <oliverchang@users.noreply.github.com>
2022-11-14 13:34:29 +11:00
DavidKorczynski 8b9cf2d081
infra: bump introspector and update python compile script (#8957)
This is a follow-up to https://github.com/google/oss-fuzz/pull/8952

This bump is needed to have the Python projects work properly. Scan
should be done for all projects now, so have switched it on by default
in fuzz introspector.

Bump also contains
- Fix for util-linux where per-target coverage report links were broken.
- Reduced logging.

Signed-off-by: David Korczynski <david@adalogics.com>

Signed-off-by: David Korczynski <david@adalogics.com>
Co-authored-by: Navidem <navid.emamdoost@gmail.com>
2022-11-09 17:49:17 -08:00
DavidKorczynski 50205ec9e9
infra: bump and refine introspector (#8952)
Bumps fuzz introspector and updates the way python fuzzers are build for
introspector builds.

Signed-off-by: David Korczynski <david@adalogics.com>

Signed-off-by: David Korczynski <david@adalogics.com>
2022-11-08 23:13:31 -08:00
DavidKorczynski 7797279c27
infra: base-builder: compile: wrap find names in quotes (#8943)
Wrap wildcards in quotes as otherwise the script will autocomplete if
there is a e.g. .json file in the current directory. This causes the
wrong files to be moved, which means `all_cov.json` will not be moved to
the introspector folder and thus wrong (no) coverage data will be used.

This error happens in e.g. g-api-auth-library-python
2022-11-09 09:24:41 +11:00
DavidKorczynski 95a8398fb8
infra: fuzz-introspector; install matplotlib from binary (#8912)
This avoids compiling certain parts of matplotlib, which speeds up
runtime of fuzz-introspector runs locally by a significant (~5-10min)
time.

Ref:
https://github.com/ossf/fuzz-introspector/pull/579#issuecomment-1300339783
Ref: https://github.com/ossf/fuzz-introspector/issues/465

Signed-off-by: David Korczynski <david@adalogics.com>

Signed-off-by: David Korczynski <david@adalogics.com>
2022-11-02 12:54:18 -07:00
DavidKorczynski c12585c056
infa: add python fuzz-introspector support (#8865)
Adds necessary logic for running fuzz-introspector's Python frontend.

Signed-off-by: David Korczynski <david@adalogics.com>

Signed-off-by: David Korczynski <david@adalogics.com>
2022-11-01 11:51:50 -07:00
Navidem c57a1712eb
Upgrade Atheris (#8886)
Fixes #8682
2022-10-31 10:27:27 -07:00
Navidem 165ee30241
Set introspector env variable in compile (#8878) 2022-10-28 12:06:15 -07:00
jonathanmetzman 998791514d
Get rid of :introspector images. Build introspector in main images (#8868)
Actually install introspector in every image.
Don't do extra introspector build steps.
Fixes https://github.com/google/oss-fuzz/issues/8625
2022-10-27 08:27:12 -04:00
jonathanmetzman ce2a5a2903
[aflplusplus] Remove 32 MB from image size by deleting .git (#8869) 2022-10-27 12:48:12 +11:00
jonathanmetzman bc02fd0c63
Don't use centipede's install script (#8870)
We can't simply autoremove the packages it installs and it adds 500 MB
to the image size.
Install the only dependency we don't already have libssl-dev since it is very small anyway.
2022-10-26 18:52:22 -04:00
AdamKorcz 099fa2e5f5
infra: bump Go to 1.19 (#8568)
Most projects have upgraded to 1.19 which is the latest version.

This upgrades Go to 1.19 in the base-image.

Signed-off-by: AdamKorcz <adam@adalogics.com>

Signed-off-by: AdamKorcz <adam@adalogics.com>
2022-09-22 11:59:15 -07:00
John Howard 880dba286e
Always add `testing` package usage in native_go_fuzzer (#8546)
This fixes an issue where the `testing.F` is replaced with a new
package; if this was the only usage of `testing` in the file then we end
up with `testing` being an unused import. A workaround for this is to
just always use `testing` via some placeholder variable.
2022-09-21 10:49:21 -07:00
Dongge Liu 2757ed6601
Pin Centipede to a more recent version (#8543)
A recent commit allows `Centipede` to add timestamps to its log, which
can be very helpful for debugging.
This PR intends to update `Centipede` to add that commit.
2022-09-21 15:02:44 +10:00
Fabian Meumertzheim 7765e4d548
infra: Update Jazzer to fix coverage build failures (#8495)
Fixes #8241
2022-09-20 13:37:22 -04:00
van Hauser c8a9c23666
update afl++ commit id (#8216)
this fixes a bug that affects coverage.

Co-authored-by: jonathanmetzman <31354670+jonathanmetzman@users.noreply.github.com>
2022-09-19 18:48:32 -04:00
Thomas Van Lenten 12c881f772
Use Swift 5.7.0. (#8501)
Swift 5.4.2 was released back in June 2021.

Co-authored-by: jonathanmetzman <31354670+jonathanmetzman@users.noreply.github.com>
2022-09-19 18:40:00 -04:00
jonathanmetzman c0f72989f4
[centipede][base-builder] Properly delete bazel cache (#8494)
Fixes #8468

Co-authored-by: Alan32Liu <donggeliu@google.com>
2022-09-16 13:17:26 +10:00
Dongge Liu 97a68b049b
Update Centipede to fix SegFault (#8507)
Update `Centipede` to fix [the SegFault when using `AddressSanitizer`
with
`address_space_limit_mb`](https://github.com/google/centipede/issues/166).
Also set `address_space_limit_mb` as the issue has been fixed.
2022-09-16 12:33:20 +10:00
Dongge Liu 46cc569049
Do not remove the symlink as they are used by `Centipede`'s `Bazel` build (#8486) 2022-09-14 02:54:49 +00:00
jonathanmetzman 27b929d05b
Don't install git via ppa (#8470)
Reduce complexity and bloat
2022-09-13 08:37:38 +01:00
DavidKorczynski dc5adbf754
infra: add support for netdriver fuzzing with honggfuzz (#7351)
Adds an example with the mongoose webserver
2022-09-12 11:58:12 +10:00
jonathanmetzman d3e4ba3bb8
[centipede] Fix centipede (#8467)
Pin centipede to a commit that builds.
Also remove 1.3GB of image bloat it adds
CC @ussuri @Alan32Liu 
Fixes #8458
2022-09-11 13:20:02 -04:00
Oliver Chang 91acda2611
Centipede fixes. (#8417)
- Replace newlines with spaces in centipede/clang-flags.txt. Build
  systems don't like them.

- Fix a missing newline in output logs.
2022-09-06 07:41:13 +00:00