Commit Graph

27 Commits

Author SHA1 Message Date
jonathanmetzman e96a5f8f40
Allow projects to always upload crashes even if they are "unreportable". (#6998)
Fixes https://github.com/google/clusterfuzzlite/issues/65
2021-12-13 12:04:04 -05:00
jonathanmetzman fb856de70b
[ClusterFuzzLite] Fix fuzz target search for coverage (#6799)
Coverage uses a different mechanism for determining if a file
is a fuzz target: It considers any executables in the top level
of /out as fuzz targets.

Fixes #6768
2021-11-09 07:49:21 -05:00
Oliver Chang a72ef3884f
Don't set seed for batch fuzzing. (#6736)
Fixes #6714.
2021-11-03 14:22:12 +11:00
jonathanmetzman b2fa3547b2
[clusterfuzzlite] Fixes for gsutil (#6683)
1 Fix usage of gsutil tool.
2 Get rid of run_fuzzers_mode and change to mode. Fixes: #6677
3 Install requirements before copying source code to make iterative development of cifuzz code faster.
2021-10-28 15:00:44 -04:00
jonathanmetzman b77a55b9b4
[ClusterFuzzLite] Support GCB and gsutil/gcs as filestore. (#6629)
* add gsutil filestore

* lint

* Fix

* Add build image script

* get gcb fuzzing working

* fmt and fix config_utils_test

* Check that crashes are uploaded

* Add no_filestore

* fix test

* fix tests

* fix

* Print crash URL

* Fix

* fix

* fmt

* lnt

* fix

* fmt
2021-10-27 10:00:04 -04:00
Oliver Chang f460c03c8a
cifuzz: Fix artifacts uploading issues. (#6646) 2021-10-27 12:24:26 +11:00
Oliver Chang 9aa85855b4
cifuzz: Rename run-fuzzers-mode to mode. (#6600)
Also change the default value of "ci" to the more consistent and
descriptive "code-change".
2021-10-20 15:02:17 +11:00
Oliver Chang d178de5e00
CFLite: Fix corpus pruning for git store. (#6434)
Fixes #6376.
2021-09-15 11:49:29 +10:00
Oliver Chang a4bc23909b
Don't upload builds in run_fuzzers. (#6151)
The current way adds a lot of ordering assumptions, and doesn't fit too
well with parallel batch fuzzing either. Add a "upload-build" boolean action
input that can be added to "build_fuzzers" to upload latest builds
instead.

Builds are now uploaded by commit hash, rather than a fixed "latest" name.
ClusterFuzzLite's download_latest_build will check the last 3 commits and download the
first available build by git hash.
2021-08-17 06:36:06 +00:00
jonathanmetzman 94cfc4fe2f
[cifuzz] Add pruning task (#6188)
Fixes: #6064
2021-08-10 11:10:10 -07:00
jonathanmetzman 432105a31a
[cifuzz][NFC] Move Workspace to workspace_utils. (#6158)
In hindsight, it doesn't have a lot to do with the rest of
config_utils. And config_utils is getting crowded.
2021-08-03 17:58:20 -07:00
jonathanmetzman 6f0aa82cd1
[NFC][cifuzz] Move Workspace from docker into config_utils. (#6117)
It doesn't really fit into docker.
2021-07-27 12:11:53 -07:00
Oliver Chang b1e5ef81c0
Fix upload_corpus call arguments for batch fuzzing. (#6050) 2021-07-17 07:19:37 +10:00
jonathanmetzman 0672aa4e1f
[CIFuzz] Don't make everything a subdirectory of /out (#5970)
Use different subdirectories of workspace for builds, old builds, coverage reports, corpora and artifacts/testscases.
2021-06-30 07:34:42 -07:00
jonathanmetzman ae5e4228e8
[CIFuzz][ClusterFuzzLite] Implement batch fuzzing (#5963)
[ClusterFuzzLite] Support use of github actions as a filestore
2021-06-25 08:42:10 -07:00
jonathanmetzman 343e1b2d33
[CIFuzz] Add coverage report generation. (#5937)
Also:
1. Support coverage builds.
2. Add an integration test for coverage builds and reports.
3. Refactor docker code so that there is less duplication in getting arguments for docker run, in particular when handling container (production) vs no container (testing).
4. Rename coverage.py to get_coverage_report.
5. Add tests for untested functions in docker.py
6. Add a test for get_fuzz_target_runner.
2021-06-23 07:30:11 -07:00
jonathanmetzman 26dea63cde
[CIFuzz][NFC] Replace format strings with f-strings. (#5933)
Also replace % formatting.
2021-06-17 11:26:42 -07:00
jonathanmetzman 8c4ad095e9
[CIFuzz] Refactor in preparation for filestore (#5841)
1. Moving the _create_config and create_config functions from build_fuzzers_test.py and run_fuzzers_test.py into test_helpers.py (which is now part of cifuzz instead of infra) and share code between them.
2. Rename artifacts_dir to crashes_dir in run_fuzzers.py. "artifacts" is ambiguous.
3. Make some small changes to pytest.ini to improve debugging.
2021-05-26 09:45:22 -07:00
jonathanmetzman 4cc3aa6343
[CIFuzz] Print crash stacktraces no matter what (#5474)
Even if we don't want to report them, print the stacktrace.
Related: https://github.com/google/oss-fuzz/issues/5461
2021-03-31 06:55:30 -07:00
jonathanmetzman 62fce2a587
[CIFuzz] Give testcases unique names. (#5424) 2021-03-17 10:21:49 -07:00
jonathanmetzman 3465403f30
[CIFuzz] Add functionality to save diskspace (#5342)
* [CIFuzz] Add functionality to save diskspace.

Add a LOW_DISK_SPACE env/config var. When this is specified
(always true for Github actions) run_fuzzers will delete
base-builder and the project builder image before fuzzing.
After it finishes fuzzing with a target, it will also
delete the targets, its seed corpus and its corpus.

Related: #4879
2021-03-12 07:27:07 -08:00
jonathanmetzman e93f222966
[CIFuzz] Fix handling of sanitizer artifacts (#5182)
Fixes #5175
1. Put sanitizer in artifact name.
2. Fix parsing of non-ASAN stacks.
2021-02-18 09:57:34 -08:00
jonathanmetzman 21b47a7a22
[cifuzz][NFC] Handle TODOs (#5104)
Handle some TODOs
1. Get rid of multiple return values and replace with a more sensible
return value.
2. Eliminate some useless TODOs.
2021-02-04 07:15:51 -08:00
jonathanmetzman 7f150fe75e
[cifuzz] Abstract-away OSS-Fuzz specific bits (#5088)
Abstract away OSS-Fuzz specific bits into the OSSFuzz implementation
of the ClusterFuzzDeployment class. This will make it easier to implement
support for other deployments of ClusterFuzz (including ClusterFuzzLite).
2021-02-03 12:46:19 -08:00
jonathanmetzman 95d3905ec9
[cifuzz] Support a batch fuzzing mode (#5073)
In this mode, CIFuzz will keep fuzzing until the time limit is reached, even if a crash was found.
2021-02-01 10:49:33 -08:00
jonathanmetzman f2756d7321
[CIFuzz] Move run_fuzzers to new config system (#5063)
Also, decide is_github based on something not used/faked by Skia.
2021-01-29 09:49:03 +11:00
jonathanmetzman d6ff0bfcdc
[CIFuzz] Seperate code for running fuzzers into own module: run_fuzzers.py (#5031)
TODO: Rename cifuzz.py to build_fuzzers.py
2021-01-26 08:32:41 -08:00