Commit Graph

70 Commits

Author SHA1 Message Date
Navidem d8a7e45314
Remove unnecessary step in fuzz introspector (#7793)
Remove unncessary step in fuzz introspector
2022-06-01 18:00:22 -05:00
Navidem c5be869e4e
Add commands to collect branch coverage (#7709)
* Add commands to collect branch coverage

* Add -g flag

* Switch branch coverage option placement

* Set guarding env variable

Co-authored-by: Oliver Chang <oliverchang@users.noreply.github.com>
2022-05-24 16:10:38 +10:00
DavidKorczynski f9600a44e3
fuzz-introspector: remove use of LDFLAGS (#7573)
* fuzz-introspector: remove use of LDFLAGS

The use of LDFLAGS does not really follow the policy of OSS-Fuzz. This
moves the linker flags into the sanitizer flags.

Ref:
https://github.com/google/oss-fuzz/issues/7540#issuecomment-1094500094
2022-04-15 19:13:24 +01:00
DavidKorczynski 00132fa740
fuzz-introspector: force use of llvm-nm instead of nm (#7533)
This fixes various current build failures across projects that use nm as
part of their build process.
2022-04-11 18:04:38 +01:00
DavidKorczynski 05be069314
fuzz-introspector: add -lto to ldflags (#7547) 2022-04-11 10:00:45 +01:00
DavidKorczynski 0cb820e5af
infra: add Python coverage support (#7298)
* infra: add Python coverage support

* update python coverage helper script

* nits

* switch from commands to python

* pin coverage package

* switch to single quote strings throughout

* nit

* fix style

* fix style

* fix ci

* fix ci
2022-04-06 10:31:34 -04:00
DavidKorczynski 0bcaec68ab
infra: fuzz-introspector: ensure COVERAGE_URL exists (#7502)
* infra: fuzz-introspector: ensure COVERAGE_URL exists

This is to make sure fuzz-introspector can run in local builds.

Ref:
https://github.com/ossf/fuzz-introspector/issues/48#issuecomment-1087513497
Ref:
https://github.com/ossf/fuzz-introspector/issues/67#issuecomment-1087518856

* refactor fuzz-introspector command generation

This is to shorten the long line that runs fuzz-introspector and also in
anticipation that down the line we will have more oss-fuzz specific
commands in fuzz-introspector
2022-04-04 23:38:37 +01:00
DavidKorczynski 4ff34a956e
infra: fuzz-introspector updates and bump (#7497)
* infra: remove use of git_repo_url to fuzz-introspector

* libarchive,fluent-bit: add fuzz-introspector exclusion config

* fuzz-introspector: bump

Co-authored-by: Oliver Chang <oliverchang@users.noreply.github.com>
2022-04-04 11:35:58 +10:00
Navidem e7eb519e22
move environment var to Dockerfile to make it visible to bazel builds and bump FI (#7356) 2022-03-03 16:19:09 +11:00
Navidem b75084c74c
update compile to reflect changes in introspector code (#7341) 2022-03-01 15:57:51 +11:00
Oliver Chang fb6e9fdfef
Fuzz introspector build fixes. (#7211)
- Point `compile` to the right `fuzz-introspector` location (since it's no longer checked out in $SRC).
- Rename build tag to "introspector" to be more consistent with other
  tags.
- Fix bad merge in deploy.sh script.
- Add introspector setup to project sync.
- Enable more logging for project sync cron.
2022-02-01 07:56:59 +11:00
Navidem 1f7c15061f
Adjust to coverage reports dir renaming (#7200) 2022-01-31 11:39:11 +11:00
Navidem 9e39d350e8
apply patches needed for fuzz introspector integration (#7122) 2022-01-20 12:22:27 +11:00
jonathanmetzman 219af13b2b
Copy libc++ to /usr/local/lib instead of /usr/lib (#6493) 2021-09-20 17:00:35 +00:00
jonathanmetzman 20858d392a
[base-builder-new] Fix compile script. (#6340)
In https://github.com/google/oss-fuzz/pull/6322 compile was synced
with the version in base-builder. However, base-builder's compile
assumes that rust and go are installed. This change makes it possible
to run compile without those installed.
2021-08-27 13:00:03 -07:00
jonathanmetzman 3a4e6e4484
[Ubuntu upgrade] Remove support for prebuilt MSAN libraries. (#6280)
This is done in anticipation of the upgrade to Ubuntu 20.04 which wont support this.
We'll do this first so we can handle any breakages caused by this step before needing to handle breakages
caused by the upgrade. However, there shouldn't be any breakages due to #6281, but there may be some projects
we overlooked.
The only exception to this is libcxx.
Related: #6180.
2021-08-25 15:36:05 +00:00
Bitshift 55a5812e8c
Update fuzzers to new Atheris version (#6060)
* Update fuzzers to new Atheris version

* Pin new atheris version

* Change = to ==

* Change path where Atheris fuzzers ld_preload from

* Update hypothesis fuzzer to work with new atheris

* Set version to 2.0.1

* Switch to Atheris 2.0.4

* Correct spelling of pygments_fuzzer decorator

* instrument yaml_reader

* Change to latest Atheris

* Set atheris version to 2.0.6
2021-08-16 08:56:54 -07:00
Fabian Meumertzheim ede1fb13ae
[infra] Enable Jazzer support for UBSan C++ features (#6202)
Jazzer now links in the UBSan C++ runtime and thus support the vptr
and function sanitizer settings.
2021-08-11 17:35:29 +00:00
Fabian Meumertzheim 5426b732a1
[infra][jvm] Fix UBSan C++ builds (#5961) 2021-06-23 08:22:40 -07:00
Fabian Meumertzheim 1eb1f23711
[infra][jvm] Provide jazzer_driver_with_sanitizer for coverage builds (#5948)
Previously, JVM coverage builds for projects with native dependencies would fail: https://oss-fuzz-build-logs.storage.googleapis.com/log-9b66fe39-7e33-4058-812a-c9c52f32e0ea.txt
2021-06-19 10:59:46 -07:00
Fabian Meumertzheim 7c4739add9
[infra][base-builder] Clarify error messages in `compile` (#5912)
@jonathanmetzman
2021-06-11 07:53:28 -07:00
Fabian Meumertzheim 73d78b8879
[infra][jvm] Add Jazzer UBSan support (#5898)
* [infra][jvm] Add Jazzer UBSan support

* [java-example] Reenable and plant UB

* [docs] Mention support for Java UBSan in docs

Also adds a link to the java-example build.sh to the docs.
2021-06-10 07:57:42 -07:00
Evgeny Vereshchagin 1f49a51ce2
infra: show RUSTFLAGS as well (#5867)
It should make it easier to see where all the rustc flags
come from. RUSTFLAGS along with `cargo fuzz build --verbose` should
help to track down weird issues like https://github.com/google/oss-fuzz/pull/5865#issuecomment-852685588

It's a follow-up to ecf3d384fb
2021-06-02 11:51:26 -07:00
Catena cyber 7004ced0d3
Right copy for rust std lib for coverage (#5791) 2021-05-17 18:55:53 +00:00
Catena cyber beee192ed3
Rust coverage: copy stdlib to the referenced directory (#5776)
using rustc commit-hash version
2021-05-16 14:32:33 -07:00
Fabian Meumertzheim 5cfb074ae5
Add Java coverage support (#5733)
Coverage is collected by running the fuzz targets with the JaCoCo agent
in Jazzer's no instrumentation mode.

Since JaCoCo does not support llvm-cov style coverage reports, a simple
Python helper creates it using the information contained in the JaCoCo
XML report.

As the Java build process does not maintain a mapping between source
files and build artifacts and JaCoCo needs to be passed the root folder
of the package tree, we use the Maven directory layout convention to
heuristically detect these roots.
2021-05-10 20:33:07 -07:00
Catena cyber c41e46ffc8
Rust coverage report (for Suricata) (#4697)
* Rust coverage test

* Workaround to get rust coverage for Suricata
2021-03-08 07:05:35 -08:00
Fabian Meumertzheim 481280c650
[jazzer] Adapt infra scripting to JVM fuzz targets (#5176)
compile, bad_build_check, and presubmit.py require small tweaks to
support JVM fuzz targets, most of which are similar to those required
for Python. The following additional changes are required:

* Since the Jazzer driver binary already links in libFuzzer, it should
  not be built as a static library.
* It is not clear how to do architecture checks as JVM fuzz targets can
  load their native dependencies dynamically at runtime. For now, the
  check is disabled.
* The Jazzer binaries are moved into $OUT and need to be skipped over in
  find_fuzz_targets.
2021-02-16 07:06:58 -08:00
Fabian Meumertzheim 427f63c93c
Add Jazzer to base-builder and base-runner (#5166)
Jazzer is built from HEAD using Bazel and the clang toolchain provided
by base-clang. While it could be built with OpenJDK 8, which is
available as a package, JVM fuzz targets should not be forced to be
compatible with Java 8. For this reason, the official binary release of
OpenJDK 15 is pulled into both base-builder and base-runner and set as
JAVA_HOME. It is trimmed down in size by removing src.zip and the jmods
directory.

Jazzer consists of the following four components:

* The API (`jazzer_api_deploy.jar`), which is required for fuzz targets
  that use FuzzedDataProvider or custom method hooks, is made available
  in /usr/local/lib in base-builder.
* The driver (`jazzer_driver`), which links in libFuzzer and is reused
  across fuzz targets. Since it is used to run fuzz targets, it is
  included into base-runner.
* The ASanified driver (`jazzer_driver_asan`), which is obtained from
  `jazzer_driver` by linking in ASan.
* The agent (`jazzer_agent_deploy.jar`), which bundles the runtime
  instrumentation agent with the Jazzer API. It is loaded by the driver
  and thus also included into base-runner.

The changes to the infra scripts required by JVM fuzz targets will be
submitted as a separate PR.
2021-02-16 06:56:36 -08:00
Abhishek Arya ebe4848a52
Properly fix stripping of -fsanitize=fuzzer-no-link for afl++ (#5090)
Fixes several builds - ibmswtpm2, tpm2-tss, etc
2021-02-01 21:16:56 -08:00
Abhishek Arya fba71c70ca
Copy llvm-symbolizer in $OUT, needed for python targets. (#4832) 2020-12-12 18:58:59 -08:00
Abhishek Arya 64b223a8e9
Fix pillow with new atheris changes. (#4826)
* Fix pillow with new atheris changes.

* Fix pillow in ubsan.
2020-12-11 14:35:43 -08:00
Abhishek Arya 0dd7aaca32
Fix atheris integration. (#4824) 2020-12-11 11:38:43 -08:00
Abhishek Arya b4dd4b91de
Enable UBSan for python projects. (#4757)
* Enable UBSan for python projects.

* Update docs.
2020-11-30 13:26:33 -08:00
Abhishek Arya e7921e7795
Python fuzzing fixes - disable leak checking, remove flags in atheris. (#4681)
* Python fuzzing fixes - disable leak checking, remove flags in atheris.

* Fix flags printing.

* Dont export, use directly.

* Try unset.

* Unset flags when python was built.
2020-11-20 11:10:51 -08:00
Abhishek Arya 5e3f47dba6
Fix ujson python fuzzing. (#4667)
* Fix ujson python fuzzing.

* Fix.

* Improve integration.

* Remove preload, stick with wrapper with /bin/sh.

* Update build.sh
2020-11-19 22:41:12 -08:00
Catena cyber 07ea81ba3e
[infra] Add code coverage report generation for Go projects (#3142)
* Golang coverage report

* Enables golang coverage report for gonids and go-dns

* Generates summary for golang coverage reports

* Performance profile for golang projects
2020-11-19 00:14:30 -08:00
Abhishek Arya d70af2fc96
Don't add sanitizer=coverage in RUSTFLAGS 2020-05-25 12:55:58 -07:00
Abhishek Arya f8a00efe4c
Don't add Rust sanitizer flags when using ubsan or i386. (#3862)
Rust sanitizer flags are not supported for these configs.
Fixes Cras and ecc-diff-fuzzer builds.
2020-05-21 17:37:02 -07:00
Abhishek Arya 286badcb3a
Remove unneeded RUSTC_BOOTSTRAP since nightly is used 2020-05-21 10:56:02 -07:00
Abhishek Arya ecf3d384fb
Set rust flags always in compile step. (#3859)
This is needed for projects that mix both rust and c/c++.
2020-05-21 10:16:52 -07:00
Abhishek Arya 4f7cf1b334
Simplify rust project setup. (#3830)
* Simplify rust project setup.

- Add rust and cargo-fuzz in base builder.
- Set RUSTC_BOOSTRAP to make ASan available.
- Set RUSTFLAGS and C,CXXFLAGS properly.
2020-05-17 16:45:54 -07:00
Catena cyber 4231987eee
Adds locally installed includes for coverage build (#3448) 2020-03-02 07:39:12 -08:00
jonathanmetzman c4202c2f48
Revert "Use "x86" instead of "i386" for better name recognition. (#2685)" (#2687)
This reverts commit 0c6cc5cae8.
2019-08-12 10:54:18 -07:00
jonathanmetzman 0c6cc5cae8
Use "x86" instead of "i386" for better name recognition. (#2685)
It seems more people understand x86 than i386, use this term instead.
2019-08-12 10:35:16 -07:00
Max Moroz 5dece89e3c
[infra] Add support for dataflow builds to the helper script and build check (#1632). (#2501)
* [infra] Add support for dataflow builds to the helper script and build check (#1632).

* Update travis config file.

* Address self-review comments and specify dataflow sanitizer for zstd as well.

* Fix fuzzing_engines in project.yaml

* Fix bad build check for DFSan.

* Use "hasattr" in helper.py to check the sanitizer argument.

* Address more review comments.

* Remove DataFlow config from zstd.

* fix a typo
2019-06-12 11:08:15 -07:00
Max Moroz 3830c427b2 [infra] Ignore return code when removing seed corpus for dataflow build. 2019-06-10 15:32:51 -07:00
Max Moroz 1f0a33c712 [infra] Do not archive seed corpus for dataflow builds. 2019-06-10 10:00:20 -07:00
jonathanmetzman 3976a002af
[i386] Improve linking and bad build check (#2410)
Improve linking by copying i386 libs instead of adding `-L` to `CXXFLAGS`
Get bad_build_check working for i386 binaries and add architecture check.
2019-05-14 10:18:02 -07:00
jonathanmetzman c60fcd7247
Add initial support for i386 builds (#2347)
This feature is not yet officially supported
2019-05-13 15:01:25 -07:00