Commit Graph

1484 Commits

Author SHA1 Message Date
Jonathan Metzman 781749884b [cifuzz][github_actions] Create methods/functions for uploading without tar
This can be useful for artifacts we want to upload that we know
do not need to be tarred, such as crashes. It's important not
to tar these because they will need to be viewed by users.
2021-07-23 08:00:35 -07:00
Oliver Chang 1f679f0dd2
Add external actions. (#6086)
This will be used for testing and can be moved elsewhere later.
2021-07-23 08:19:10 +10:00
jonathanmetzman c67d5b8626
[CIFuzz] Fix build uploading. (#6096)
Previously, the downloaded build was uploaded instead of the new build.
This regression was introduced when the big workspace change was made.
Also, add more logging.
2021-07-22 12:52:07 -07:00
jonathanmetzman 24eb7e6668
[NFC][cifuzz] Add tests for github actions. (#6094)
To avoid issues like #6087 before they land.
2021-07-22 12:51:34 -07:00
jonathanmetzman 3d75fc9fb4
[cifuzz][NFC] Use f-strings instead of format function (#6095) 2021-07-22 09:22:29 -07:00
jonathanmetzman cdd78c464b
[cifuzz] Fix corpus usage in coverage reports (#6093)
Fixes #6089
2021-07-22 08:09:49 -07:00
Oliver Chang fdb9c9a0fd
Fix typo (#6087) 2021-07-22 15:02:00 +10:00
jonathanmetzman 2a3a1e903a
[CIFuzz] Add cifuzz- prefix to all artifacts (#6084)
This will prevent conflicts with other artifacts in the repo.
2021-07-21 11:36:11 -07:00
jonathanmetzman 6baa5e3e68
[CIFuzz] Fix bug in tar_directory (#6083)
We need to split the archive_path
2021-07-21 10:19:32 -07:00
jonathanmetzman 67e3a77fd7
[NFC][infra][build] Move test data to test_data directory. (#6081)
Do this for consistency.
2021-07-21 10:07:59 -07:00
jonathanmetzman fde0a30492
[CIFuzz] Support uploading coverage reports (#6078) 2021-07-21 09:19:47 -07:00
jonathanmetzman 6004ddf363
[NFC][cifuzz] Delete debug code and improve naming (#6082) 2021-07-21 09:18:44 -07:00
jonathanmetzman 6c5fffc019
[infra][build] Reset HOME using env instead of bash. (#6079)
Followup on #6069.
2021-07-21 09:09:08 -07:00
Oliver Chang c7cab23c20
Make oss-fuzz-project-name optional. (#6071)
Also rename code uses from `project_name` to `oss_fuzz_project_name`, to
be more explicit, and use it to determine whether or not we're running
an OSS-Fuzz project or not.

For external fuzzing #6051.
2021-07-21 05:32:32 +00:00
jonathanmetzman 857e9c27d0
[CIFuzz][Coverage] Make coverage reports world readable (#6077)
This is needed to upload coverage reports.
Related: #6054
2021-07-20 11:50:27 -07:00
jonathanmetzman 86392ca2f4
[CIFuzz][ClusterFuzzLite] Prepare to support coverage reports for external users (#6074)
1. Make coverage script accept an env var that controls where the generated report is written to. Use this in generate_coverage_report.
2. Rename CoverageGetter to OSSFuzzCoverage
3. Make affected_fuzz_targets get coverage object from clusterfuzz_deployment.
4. Add stubbed FilesystemCoverage class that will be used by non-OSS-Fuzz users.
5. Make ClusterFuzzLite able to return coverage object by downloading old coverage report and returning a FilesystemCoverage object.
6. Add stub to ClusterFuzzLite for uploading coverage.

Related: #6054
2021-07-20 10:04:57 -07:00
jonathanmetzman 0a398c892b
Fix misspelling of Implementation (#6073) 2021-07-20 07:38:16 -07:00
jonathanmetzman d66e974cbe
[infra][build] Set HOME=/root on GCB when doing fuzzer builds. (#6069)
[infra][build] Set HOME=/root on GCB when doing fuzzer builds.
GCB passes HOME as env var to the docker container. It sets
HOME to /builder/home which is persisted accross builds.
This issue causes build breakages in
https://github.com/google/oss-fuzz/issues/6035
and possibly https://github.com/google/oss-fuzz/issues/5317.
Perhaps more insidiuosly it can cause fuzzers to be built with
the wrong instrumentation.
2021-07-20 06:56:50 -07:00
Abhishek Arya ae1ce87c8d
Pin atheris version (#6062) 2021-07-18 13:38:12 -07:00
Oliver Chang b1e5ef81c0
Fix upload_corpus call arguments for batch fuzzing. (#6050) 2021-07-17 07:19:37 +10:00
van Hauser 88c72fd4fc
workaround for #6037 (#6039)
* enhance afl++ performance

* update afl++, higher chance for cmplog

* workaround for afl++ ctx+cmplog issues
2021-07-14 13:57:52 +01:00
Oliver Chang 3022776970
Revert "Bring back build status page fix until quota issues are resolved. (#5997)" (#6008)
This reverts commit e10f1658fa.
2021-07-08 10:32:36 +10:00
van Hauser 0cf9ebe150
update afl++, abuse better cmplog performance (#6006)
* enhance afl++ performance

* update afl++, higher chance for cmplog
2021-07-07 10:26:24 +01:00
Fabian Meumertzheim 360227e3b3
[infra] Do not close file descriptors also for JVM (#6002)
Keeps the JVM coverage libFuzzer args in sync with the change in https://github.com/google/oss-fuzz/pull/5999.
2021-07-06 10:16:48 +01:00
DavidKorczynski 581aa96454
do not close file descriptors in coverage runs. (#5999) 2021-07-05 19:31:56 -07:00
Abhishek Arya e10f1658fa
Bring back build status page fix until quota issues are resolved. (#5997) 2021-07-05 10:15:45 -07:00
jonathanmetzman 535597b038
[build/infra] Build engines in alphabetical order. (#5987)
* [build/infra] Build engines in alphabetical order.

Previously, a project fuzz targets were usually built in the order:
libfuzzer, afl, honggfuzz. This can bias results if one is looking
at which engine finds bugs first.

* fix tests

* lnt
2021-07-01 11:17:18 -07:00
Mitchel Herman faea660c5e
Add CIFuzz external entrypoint for cifuzz base (#5906)
This is an entrypoint for CIFuzz users that can't build two docker images
(e.g. prow users).
Co-authored-by: jonathanmetzman <31354670+jonathanmetzman@users.noreply.github.com>
2021-06-30 17:19:03 -07:00
jonathanmetzman 0672aa4e1f
[CIFuzz] Don't make everything a subdirectory of /out (#5970)
Use different subdirectories of workspace for builds, old builds, coverage reports, corpora and artifacts/testscases.
2021-06-30 07:34:42 -07:00
DavidKorczynski d6970e868f
infra: Remove unused import from update_build_status (#5981)
In this commit 0370821e39 the use of the `time` import was removed. This causes the CI to complain about `infra/build/functions/update_build_status.py:21:0: W0611: Unused import time (unused-import)` here https://github.com/google/oss-fuzz/pull/5980/checks?check_run_id=2941631092
2021-06-29 12:29:35 +00:00
Abhishek Arya 0370821e39
Revert rate limits in update_build_status. 2021-06-28 21:30:04 -07:00
Fabian Meumertzheim 84ea9561f7
[infra] Add support for rules_fuzzing's java_fuzz_test macro (#5960)
* [infra] Add support for rules_fuzzing's java_fuzz_test macro

* [infra] Update bazelisk for rolling release support

* [infra] Fix C++ stdlib mixing for uninstrumented Bazel targets

The build script for Bazel rules_fuzzing tests did not set the C++
stdlib for uninstrumented C++ binaries, which thus use the system
libstdc++ instead of the libc++ built from source.

* [rules_fuzzing] Add test project for rules_fuzzing's java_fuzz_test
2021-06-28 06:33:38 -07:00
Abhishek Arya 53eff02879
Update update_build_status.py 2021-06-25 17:43:38 -07:00
Abhishek Arya a5822bb2ff
Update update_build_status.py 2021-06-25 17:09:42 -07:00
Abhishek Arya 00bbfdb5e1
Update update_build_status.py 2021-06-25 16:56:01 -07:00
Abhishek Arya 7d93e152de
Update update_build_status.py 2021-06-25 13:19:33 -07:00
Abhishek Arya ea06882e92
Update update_build_status.py 2021-06-25 13:16:47 -07:00
Abhishek Arya 77211e836c
Update update_build_status.py 2021-06-25 12:46:09 -07:00
Abhishek Arya 3096b90ca2
Update update_build_status.py 2021-06-25 12:04:37 -07:00
Abhishek Arya ba0a288c6e
Update update_build_status.py 2021-06-25 11:40:44 -07:00
Abhishek Arya 7e61f5509e
Update update_build_status.py 2021-06-25 11:06:47 -07:00
Abhishek Arya 3680920a30
Update update_build_status.py 2021-06-25 10:38:19 -07:00
Abhishek Arya 5fcb499c68
Decrease number of build logs to avoid rate limits. (#5973) 2021-06-25 10:18:38 -07:00
Abhishek Arya 867646da2d
Update update_build_status.py 2021-06-25 09:43:11 -07:00
Abhishek Arya 3b19d9fc18
Update update_build_status.py 2021-06-25 09:29:33 -07:00
Abhishek Arya f367531263
Reduce memory requirement in cloud functions to 2gb 2021-06-25 09:24:34 -07:00
Abhishek Arya db33cc2c38
Update update_build_status.py 2021-06-25 09:04:31 -07:00
Abhishek Arya 13a076b275
Reduce build status workers to see if quota issue resolve. 2021-06-25 08:42:43 -07:00
jonathanmetzman ae5e4228e8
[CIFuzz][ClusterFuzzLite] Implement batch fuzzing (#5963)
[ClusterFuzzLite] Support use of github actions as a filestore
2021-06-25 08:42:10 -07:00
Abhishek Arya dbe6bd6fd3
Try fix for memory exceeded (#5969)
* Try fix for memory exceeded

Should fix https://github.com/google/oss-fuzz/issues/5968

* Update deploy.sh
2021-06-25 08:02:31 -07:00