Rooba
/
oss-fuzz
mirror of https://github.com/google/oss-fuzz.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Switch docs to new structure (#2663)

This commit is contained in:
Abhishek Arya 2019-08-07 07:37:16 -07:00 committed by GitHub
parent c070f7fc7d
commit cf4af86972
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
29 changed files with 829 additions and 521 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

128
README.md
View File

@ -1,22 +1,5 @@
# OSS-Fuzz - Continuous Fuzzing for Open Source Software # OSS-Fuzz - Continuous Fuzzing for Open Source Software
> *Status*: Stable. We are accepting applications from widely-used open source projects.
[FAQ](docs/faq.md)
| [Integration Rewards](https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html)
| [New Project Guide](docs/new_project_guide.md)
| [Ideal Fuzzing Integration](docs/ideal_integration.md)
| [Accessing Corpora](docs/corpora.md)
| [Reproducing Bugs](docs/reproducing.md)
| [Current Projects](projects)
| [Projects Issue Tracker](https://bugs.chromium.org/p/oss-fuzz/issues/list)
| [Glossary](docs/glossary.md)
[Create New Issue](https://github.com/google/oss-fuzz/issues/new) for questions or feedback about OSS-Fuzz.
Not sure where to start? Look at [New Project Guide](docs/new_project_guide.md).
## Introduction ## Introduction
[Fuzz testing](https://en.wikipedia.org/wiki/Fuzz_testing) is a well-known [Fuzz testing](https://en.wikipedia.org/wiki/Fuzz_testing) is a well-known
@ -34,105 +17,30 @@ distributed execution.
We support [libFuzzer](http://llvm.org/docs/LibFuzzer.html) and [AFL](http://lcamtuf.coredump.cx/afl/) as fuzzing engines We support [libFuzzer](http://llvm.org/docs/LibFuzzer.html) and [AFL](http://lcamtuf.coredump.cx/afl/) as fuzzing engines
in combination with [Sanitizers](https://github.com/google/sanitizers). in combination with [Sanitizers](https://github.com/google/sanitizers).
[ClusterFuzz](docs/clusterfuzz.md) [ClusterFuzz](https://google.github.io/oss-fuzz/furthur-reading/clusterfuzz)
provides a distributed fuzzer execution environment and reporting. You can provides a distributed fuzzer execution environment and reporting. You can
checkout ClusterFuzz [here](https://github.com/google/clusterfuzz). checkout ClusterFuzz [here](https://github.com/google/clusterfuzz).
Currently OSS-Fuzz supports C and C++ code (other languages supported by [LLVM](http://llvm.org) may work too). Currently OSS-Fuzz supports C and C++ code (other languages supported by [LLVM](http://llvm.org) may work too).
## Process Overview ## Documentation
You can find detailed documentation [here](https://google.github.io/oss-fuzz).
![diagram](docs/images/process.png?raw=true)
The following process is used for projects in OSS-Fuzz:
- A maintainer of an opensource project or an outside volunteer creates
one or more [fuzz targets](http://libfuzzer.info/#fuzz-target)
and [integrates](docs/ideal_integration.md) them with the project's build and test system.
- The project is [accepted to OSS-Fuzz](#accepting-new-projects).
- When [ClusterFuzz](docs/clusterfuzz.md) finds a bug, an issue is automatically
reported in the OSS-Fuzz [issue tracker](https://bugs.chromium.org/p/oss-fuzz/issues/list)
([example](https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=9)).
([Why use a different tracker?](docs/faq.md#why-do-you-use-a-different-issue-tracker-for-reporting-bugs-in-oss-projects)).
Project owners are CC-ed to the bug report.
- The project developer fixes the bug upstream and credits OSS-Fuzz for the discovery (commit message should contain
the string **'Credit to OSS-Fuzz'**).
- [ClusterFuzz](docs/clusterfuzz.md) automatically verifies the fix, adds a comment and closes the issue ([example](https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=53#c3)).
- 30 days after the fix is verified or 90 days after reporting (whichever is earlier), the issue becomes *public*
([guidelines](#bug-disclosure-guidelines)).
<!-- NOTE: this anchor is referenced by oss-fuzz blog post -->
## Accepting New Projects
To be accepted to OSS-Fuzz, an open-source project must
have a significant user base and/or be critical to the global IT infrastructure.
To submit a new project:
- [Create a pull request](https://help.github.com/articles/creating-a-pull-request/) with new
`projects/<project_name>/project.yaml` file ([example](projects/libarchive/project.yaml)) giving at least the following information:
* project homepage.
* e-mail of the engineering contact person to be CCed on new issues. It should:
* belong to an established project committer (according to VCS logs). If this is not you or the email address differs from VCS, an informal e-mail verification will be required.
* be associated with a Google account ([why?](docs/faq.md#why-do-you-require-a-google-account-for-authentication)). If you use an alternate email address [linked to a Google Account](https://support.google.com/accounts/answer/176347?hl=en), it will ONLY give you access to filed bugs in [issue tracker](https://bugs.chromium.org/p/oss-fuzz/issues/list) and NOT to [ClusterFuzz](clusterfuzz.md) dashboard (due to appengine api limitations).
* Note that `project_name` can only contain alphanumeric characters, underscores(_) or dashes(-).
- Once accepted by an OSS-Fuzz project member, follow the [New Project Guide](docs/new_project_guide.md)
to configure your project.
## Bug Disclosure Guidelines
Following [Google's standard disclosure policy](https://googleprojectzero.blogspot.com/2015/02/feedback-and-data-driven-updates-to.html)
OSS-Fuzz will adhere to following disclosure principles:
- **Deadline**. After notifying project authors, we will open reported
issues to the public in 90 days, or 30 days after the fix is released
(whichever comes earlier).
- **Weekends and holidays**. If a deadline is due to expire on a weekend,
the deadline will be moved to the next normal work day.
- **Grace period**. We have a 14-day grace period. If a 90-day deadline
expires but the upstream engineers let us know before the deadline that a
patch is scheduled for release on a specific day within 14 days following
the deadline, the public disclosure will be delayed until the availability
of the patch.
## More Documentation
* [Glossary](docs/glossary.md) describes the common terms used in OSS-Fuzz.
* [New Project Guide](docs/new_project_guide.md) walks through the steps necessary to add new projects to OSS-Fuzz.
* [Ideal Integration](docs/ideal_integration.md) describes the steps to integrate fuzz targets with your project.
* [Accessing corpora](docs/corpora.md) describes how to access the corpora we use for fuzzing.
* [Fuzzer execution environment](docs/fuzzer_environment.md) documents the
environment under which your fuzzers will be run.
* [Projects](projects) lists OSS projects currently analyzed by OSS-Fuzz.
* [Chrome's Efficient Fuzzing Guide](https://chromium.googlesource.com/chromium/src/testing/libfuzzer/+/HEAD/efficient_fuzzing.md)
while containing some Chrome-specific bits, is an excellent guide to making your fuzzer better.
* Blog posts:
* 2016-12-01 ([1](https://opensource.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html),
[2](https://testing.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html),
[3](https://security.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html))
* 2017-05-08 ([1](https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html),
[2](https://testing.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html),
[3](https://security.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html))
* 2018-11-06 ([1](https://security.googleblog.com/2018/11/a-new-chapter-for-oss-fuzz.html))
## Build Status
[This page](https://oss-fuzz-build-logs.storage.googleapis.com/index.html)
gives the latest build logs for each project.
(Internal only) [Builds dashboard](https://builder.oss-fuzz.com).
## Web Interface
* The main page: [oss-fuzz.com](https://oss-fuzz.com)
* [ClusterFuzz for OSS-Fuzz documentation](docs/clusterfuzz.md)
## Trophies ## Trophies
As of August 2019, OSS-Fuzz has found [~14,000] bugs in over [200] open source
projects.
[~14,000]: https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=-status%3AWontFix%2CDuplicate+-Infra
[200]: https://github.com/google/oss-fuzz/tree/master/projects
## Blog posts
* 2016-12-01 ([1](https://opensource.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html),
[2](https://testing.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html),
[3](https://security.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html))
* 2017-05-08 ([1](https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html),
[2](https://testing.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html),
[3](https://security.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html))
* 2018-11-06 ([1](https://security.googleblog.com/2018/11/a-new-chapter-for-oss-fuzz.html))
[This page](https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=status%3AFixed%2CVerified+Type%3ABug%2CBug-Security+-component%3AInfra+)
gives a list of publicly-viewable fixed bugs found by OSS-Fuzz.
## References
* [libFuzzer documentation](http://libfuzzer.info)
* [libFuzzer tutorial](http://tutorial.libfuzzer.info)
* [libFuzzer workshop](https://github.com/Dor1s/libfuzzer-workshop)
* [Structure-Aware Fuzzing with libFuzzer](https://github.com/google/fuzzer-test-suite/blob/master/tutorial/structure-aware-fuzzing.md)
* [Chromium Fuzzing Page](https://chromium.googlesource.com/chromium/src/testing/libfuzzer/)
* [ClusterFuzz](https://github.com/google/clusterfuzz)

9
docs/advanced-topics/advanced_topics.md Normal file
View File

@ -0,0 +1,9 @@
---
layout: default
title: Advanced topics
has_children: true
nav_order: 3
permalink: /advanced-topics/
---
# Advanced topics

13
docs/code_coverage.md → docs/advanced-topics/code_coverage.md
View File

@ -1,8 +1,20 @@
---
layout: default
title: Code coverage
parent: Advanced topics
nav_order: 2
permalink: /advanced-topics/code-coverage
---
# Code Coverage # Code Coverage
You can generate code coverage report for your project using [Clang Source-based You can generate code coverage report for your project using [Clang Source-based
Code Coverage]. Code Coverage].
- TOC
{:toc}
---
## Pull the latest Docker images ## Pull the latest Docker images
@ -107,7 +119,6 @@ reports for your project, add the arguments into `project.yaml` file as follows:
coverage_extra_args: -ignore-filename-regex=.*crc.* -ignore-filename-regex=.*adler.* <other_extra_args> coverage_extra_args: -ignore-filename-regex=.*crc.* -ignore-filename-regex=.*adler.* <other_extra_args>
``` ```
[Clang Source-based Code Coverage]: https://clang.llvm.org/docs/SourceBasedCodeCoverage.html [Clang Source-based Code Coverage]: https://clang.llvm.org/docs/SourceBasedCodeCoverage.html
[gsutil tool]: https://cloud.google.com/storage/docs/gsutil_install [gsutil tool]: https://cloud.google.com/storage/docs/gsutil_install
[llvm-cov tool]: https://llvm.org/docs/CommandGuide/llvm-cov.html [llvm-cov tool]: https://llvm.org/docs/CommandGuide/llvm-cov.html

67
docs/advanced-topics/corpora.md Normal file
View File

@ -0,0 +1,67 @@
---
layout: default
title: Corpora
parent: Advanced topics
nav_order: 3
permalink: /advanced-topics/corpora
---
# Accessing Corpora
If you would like to access the corpora that we are using for your fuzz targets
(synthesized by the fuzzing engines), please follow these steps.
- TOC
{:toc}
---
## Obtain access
In order to get access to a project's corpora, you must first be listed as the
primary contact or an auto cc in the project's `project.yaml` file as described
in the [New Project Guide]({{ site.baseurl }}/getting-started/new-project-guide/#projectyaml).
If you don't do this, most of the links below won't work.
## Install Google Cloud SDK
The corpora for fuzz targets are stored on
[Google Cloud Storage](https://cloud.google.com/storage/). To access them, you
will need to [install](https://cloud.google.com/storage/docs/gsutil_install) the
gsutil tool, which is part of the Google Cloud SDK.
Follow the instructions on the installation page to login with a Google account
listed in your project's `project.yaml` file.
## Viewing the corpus for a fuzz target
The fuzzer statistics page for your project on
[ClusterFuzz]({{ site.baseurl }}/furthur-reading/clusterfuzz)
will contain a link to the Google Cloud console for your corpus under the
"corpus_size" column. You can browse and download individual test inputs in the
corpus here.
![viewing_corpus](https://raw.githubusercontent.com/google/oss-fuzz/master/docs/images/viewing_corpus.png)
## Downloading the corpus
If you would like to download the entire corpus, from the cloud console link,
copy the bucket path highlighted here:
![corpus_path](https://raw.githubusercontent.com/google/oss-fuzz/master/docs/images/corpus_path.png)
And then run the following command to copy the corpus to a directory on your
machine.
```bash
gsutil -m rsync gs://<bucket_path> <local_directory>
```
Following the expat example above, this would be:
```bash
gsutil -m rsync gs://expat-corpus.clusterfuzz-external.appspot.com/libFuzzer/expat_parse_fuzzer <local_directory>
```
## Corpus backups
We also keep daily zipped backups of your corpora. These can be accessed from
the `corpus_backup` column of the fuzzer statistics page. Downloading these can
also be significantly faster than `gsutil -m rsync` on the corpus bucket.

21
docs/debugging.md → docs/advanced-topics/debugging.md
View File

@ -1,4 +1,18 @@
## Debugging Build Scripts ---
layout: default
title: Debugging
parent: Advanced topics
nav_order: 4
permalink: /advanced-topics/debugging
---
# Debugging issues
- TOC
{:toc}
---
## Debugging build scripts
While developing your build script, it may be useful to run bash within the While developing your build script, it may be useful to run bash within the
container: container:
@ -8,7 +22,7 @@ $ python infra/helper.py shell $PROJECT_NAME # runs /bin/bash within container
$ compile # run compilation manually $ compile # run compilation manually
``` ```
## Debugging Fuzzers with GDB ## Debugging fuzzers with GDB
If you wish to debug a fuzz target with gdb, you may use the base-runner-debug If you wish to debug a fuzz target with gdb, you may use the base-runner-debug
image: image:
@ -25,5 +39,6 @@ $ gdb --args /out/$PROJECT_NAME/$FUZZ_TARGET_NAME /out/$PROJECT_NAME/testcase
Note that the base-runner-debug image does not have access to your sources, so Note that the base-runner-debug image does not have access to your sources, so
you will not be able to do source code level debugging. We recommend integrating you will not be able to do source code level debugging. We recommend integrating
your fuzz target upstream as part of [ideal integration](ideal_integration.md) your fuzz target upstream as part of
[ideal integration]({{ site.baseurl }}/advanced-topics/ideal-integration/)
for debugging purposes. for debugging purposes.

105
docs/ideal_integration.md → docs/advanced-topics/ideal_integration.md
View File

@ -1,25 +1,43 @@
---
layout: default
title: Ideal integration
parent: Advanced topics
nav_order: 1
permalink: /advanced-topics/ideal-integration
---
# Ideal integration with OSS-Fuzz # Ideal integration with OSS-Fuzz
OSS projects have different build and test systems. So, we can not expect them OSS projects have different build and test systems. So, we can not expect them
to have a unified way of implementing and maintaining fuzz targets and integrating to have a unified way of implementing and maintaining fuzz targets and
them with OSS-Fuzz. However, we will still try to give recommendations on the preferred ways. integrating them with OSS-Fuzz. However, we will still try to give
recommendations on the preferred ways.
Here are several features (starting from the easiest) that will make automated fuzzing Here are several features (starting from the easiest) that will make automated
simple and efficient, and will allow to catch regressions early on in the development cycle. fuzzing simple and efficient, and will allow to catch regressions early on in
the development cycle.
- TOC
{:toc}
---
## TL;DR ## TL;DR
Every [fuzz target](http://libfuzzer.info/#fuzz-target): Every [fuzz target](http://libfuzzer.info/#fuzz-target):
* Is [maintained by code owners](#fuzz-target) in their RCS (Git, SVN, etc). * Is [maintained by code owners](#fuzz-target) in their RCS (Git, SVN, etc).
* Is [built with the rest of the tests](#build-support) - no bit rot! * Is [built with the rest of the tests](#build-support) - no bit rot!
* Has a [seed corpus](#seed-corpus) with good [code coverage](#coverage). * Has a [seed corpus](#seed-corpus) with good [code coverage](#coverage).
* Is [continuously tested on the seed corpus](#regression-testing) with [ASan/UBSan/MSan](https://github.com/google/sanitizers) * Is [continuously tested on the seed corpus](#regression-testing) with
* Is [fast and has no OOMs](#performance) [ASan/UBSan/MSan](https://github.com/google/sanitizers).
* Is [fast and has no OOMs](#performance).
* Has a [fuzzing dictionary](#fuzzing-dictionary), if applicable * Has a [fuzzing dictionary](#fuzzing-dictionary), if applicable
## Fuzz Target ## Fuzz Target
The code of the [fuzz target(s)](http://libfuzzer.info/#fuzz-target) should be part of the project's source code repository. The code of the [fuzz target(s)](http://libfuzzer.info/#fuzz-target) should be
All fuzz targets should be easily discoverable (e.g. reside in the same directory, or follow the same naming pattern, etc). part of the project's source code repository.
All fuzz targets should be easily discoverable (e.g. reside in the same
directory, or follow the same naming pattern, etc).
This makes it easy to maintain the fuzzers and minimizes breakages that can arise as source code changes over time. This makes it easy to maintain the fuzzers and minimizes breakages that can
arise as source code changes over time.
Make sure to fuzz the target locally for a small period of time to ensure that Make sure to fuzz the target locally for a small period of time to ensure that
it does not crash, hang, or run out of memory instantly. it does not crash, hang, or run out of memory instantly.
@ -44,10 +62,11 @@ A plethora of different build systems exist in the open-source world.
And the less OSS-Fuzz knows about them, the better it can scale. And the less OSS-Fuzz knows about them, the better it can scale.
An ideal build integration for OSS-Fuzz would look like this: An ideal build integration for OSS-Fuzz would look like this:
* For every fuzz target `foo` in the project, there is a build rule that builds `foo_fuzzer`, * For every fuzz target `foo` in the project, there is a build rule that
a binary that contains the fuzzing entry point (`LLVMFuzzerTestOneInput`) builds `foo_fuzzer`, a binary that contains the fuzzing entry point
and all the code it depends on, and that uses the `main()` function from `$LIB_FUZZING_ENGINE` (`LLVMFuzzerTestOneInput`) and all the code it depends on, and that uses the
(env var [provided](new_project_guide.md) by OSS-Fuzz environment). `main()` function from `$LIB_FUZZING_ENGINE`
(env var [provided]({{ site.baseurl }}/getting-started/new-project-guide/) by OSS-Fuzz environment).
* The build system supports changing the compiler and passing extra compiler * The build system supports changing the compiler and passing extra compiler
flags so that the build command for a `foo_fuzzer` looks similar to this: flags so that the build command for a `foo_fuzzer` looks similar to this:
@ -57,23 +76,26 @@ flags so that the build command for a `foo_fuzzer` looks similar to this:
$ make_or_whatever_other_command foo_fuzzer $ make_or_whatever_other_command foo_fuzzer
``` ```
This will allow to have minimal OSS-Fuzz-specific configuration and thus be more robust. This will allow to have minimal OSS-Fuzz-specific configuration and thus be
more robust.
There is no point in hardcoding the exact compiler flags in the build system because they There is no point in hardcoding the exact compiler flags in the build system
a) may change and b) are different depending on the fuzzing engine and the sanitizer being used. because they a) may change and b) are different depending on the fuzzing engine
and the sanitizer being used.
## Seed Corpus ## Seed Corpus
The *corpus* is a set of inputs for the fuzz target (stored as individual files). The *corpus* is a set of inputs for the fuzz target (stored as individual files).
When starting the fuzzing process, one should have a "seed corpus", When starting the fuzzing process, one should have a "seed corpus",
i.e. a set of inputs to "seed" the mutations. i.e. a set of inputs to "seed" the mutations.
The quality of the seed corpus has a huge impact on fuzzing efficiency as it allows the fuzzer The quality of the seed corpus has a huge impact on fuzzing efficiency as it
to discover new code paths more easily. allows the fuzzer to discover new code paths more easily.
The ideal corpus is a minimal set of inputs that provides maximal code coverage. The ideal corpus is a minimal set of inputs that provides maximal code coverage.
For better OSS-Fuzz integration, For better OSS-Fuzz integration,
the seed corpus should be available in revision control (can be same or different as the source code). the seed corpus should be available in revision control (can be same or
It should be regularly extended with the inputs that (used to) trigger bugs and/or touch new parts of the code. different as the source code). It should be regularly extended with the inputs
that (used to) trigger bugs and/or touch new parts of the code.
Examples: Examples:
[boringssl](https://github.com/google/boringssl/tree/master/fuzz), [boringssl](https://github.com/google/boringssl/tree/master/fuzz),
@ -81,11 +103,13 @@ Examples:
[nss](https://github.com/mozilla/nss-fuzzing-corpus) (corpus in a separate repo). [nss](https://github.com/mozilla/nss-fuzzing-corpus) (corpus in a separate repo).
## Regression Testing ## Regression Testing
The fuzz targets should be regularly tested (not necessarily fuzzed!) as a part of the project's regression testing process. The fuzz targets should be regularly tested (not necessarily fuzzed!) as a part
of the project's regression testing process.
One way to do so is to link the fuzz target with a simple standalone driver One way to do so is to link the fuzz target with a simple standalone driver
(e.g. [this one](https://github.com/llvm-mirror/compiler-rt/tree/master/lib/fuzzer/standalone)) (e.g. [this one](https://github.com/llvm-mirror/compiler-rt/tree/master/lib/fuzzer/standalone))
that runs the provided inputs and use this driver with the seed corpus created in previous step. that runs the provided inputs and use this driver with the seed corpus created
It is recommended to use [sanitizers](https://github.com/google/sanitizers) during regression testing. in previous step. It is recommended to use
[sanitizers](https://github.com/google/sanitizers) during regression testing.
Examples: [SQLite](https://www.sqlite.org/src/artifact/d9f1a6f43e7bab45), Examples: [SQLite](https://www.sqlite.org/src/artifact/d9f1a6f43e7bab45),
[openssl](https://github.com/openssl/openssl/blob/master/fuzz/test-corpus.c) [openssl](https://github.com/openssl/openssl/blob/master/fuzz/test-corpus.c)
@ -100,25 +124,32 @@ Ideally, a dictionary should be maintained alongside the fuzz target.
The syntax is described [here](http://libfuzzer.info/#dictionaries). The syntax is described [here](http://libfuzzer.info/#dictionaries).
## Coverage ## Coverage
For a fuzz target to be useful, it must have good coverage in the code that it is testing. You can view the coverage For a fuzz target to be useful, it must have good coverage in the code that it
for your fuzz targets by looking at the [fuzzer stats](clusterfuzz.md#fuzzer-stats) dashboard on ClusterFuzz, as well as is testing. You can view the coverage for your fuzz targets by looking at the
[coverage reports](clusterfuzz.md#coverage-reports). [fuzzer stats]({{ site.baseurl }}/furthur-reading/clusterfuzz#fuzzer-stats)
dashboard on ClusterFuzz, as well as
[coverage reports]({{ site.baseurl }}/furthur-reading/clusterfuzz#coverage-reports).
To generate an aggregated code coverage report for your project, please see To generate an aggregated code coverage report for your project, please see
[code coverage](code_coverage.md) [code coverage]({{ site.baseurl }}/advanced-topics/code-coverage)
documentation page. documentation page.
Coverage can often be improved by adding dictionaries, more inputs for seed corpora, and fixing Coverage can often be improved by adding dictionaries, more inputs for seed
timeouts/out-of-memory bugs in your targets. corpora, and fixing timeouts/out-of-memory bugs in your targets.
## Performance ## Performance
Fuzz targets should also be performant, as high memory usage and/or slow execution speed can slow the down Fuzz targets should also be performant, as high memory usage and/or slow
the growth of coverage and finding of new bugs. ClusterFuzz provides a execution speed can slow the down the growth of coverage and finding of new
[performance analyzer](clusterfuzz.md) bugs. ClusterFuzz provides a
for each fuzz target that shows problems that are impacting the performance of the fuzz target. [performance analyzer]({{ site.baseurl }}/furthur-reading/clusterfuzz)
for each fuzz target that shows problems that are impacting the performance of
the fuzz target.
## Example ## Example
You may look at a simple [example](../projects/example/my-api-repo) that covers most of the items above.
You may look at a simple
[example](https://github.com/google/oss-fuzz/tree/master/projects/example/my-api-repo)
that covers most of the items above.
## Not a project member? ## Not a project member?
@ -128,8 +159,10 @@ and the project maintainers are not interested in helping.
In such cases, we can host the fuzz targets, dictionaries, etc in OSS-Fuzz's In such cases, we can host the fuzz targets, dictionaries, etc in OSS-Fuzz's
repository and mention them in the Dockerfile. repository and mention them in the Dockerfile.
Examples: [libxml2](../projects/libxml2), [c-ares](../projects/c-ares), [expat](../projects/expat). Examples: [libxml2](https://github.com/google/oss-fuzz/tree/master/projects/libxml2),
[c-ares](https://github.com/google/oss-fuzz/tree/master/projects/c-ares), [expat](https://github.com/google/oss-fuzz/tree/master/projects/expat).
This is far from ideal because the fuzz targets will not be continuously tested This is far from ideal because the fuzz targets will not be continuously tested
and hence may quickly bitrot. and hence may quickly bitrot.
If you are not a project maintainer, we may not be able to CC you to security bugs found by OSS-Fuzz. If you are not a project maintainer, we may not be able to CC you to security
bugs found by OSS-Fuzz.

127
docs/advanced-topics/reproducing.md Normal file
View File

@ -0,0 +1,127 @@
---
layout: default
title: Reproducing
parent: Advanced topics
nav_order: 5
permalink: /advanced-topics/reproducing
---
# Reproducing OSS-Fuzz issues
You've been CC'ed on an OSS-Fuzz issue
([examples](https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=Type%3ABug%2CBug-Security)),
now what? Before attempting to fix the bug, you should be able to reliably
reproduce it.
- TOC
{:toc}
---
## Fuzz target bugs
Every issue has a [reproducer]({{ site.baseurl }}/reference/glossary/#reproducer)
(aka "testcase") file attached.
Download it. If the issue is not public, you will need to login using your
[Google account](https://support.google.com/accounts/answer/176347?hl=en)
([why?]({{ site.baseurl }}/faq/#why-do-you-require-a-google-account-for-authentication))
that the bug report CCs.
This file contains the bytes that were fed to the [Fuzz Target](http://libfuzzer.info/#fuzz-target).
If you have already
[integrated]({{ site.baseurl }}/advanced-topics/ideal-integration/)
the fuzz target with your build and test system, all you do is run:
```bash
$ ./fuzz_target_binary <testcase_path>
```
If this is a timeout bug, add the **-timeout=25** argument.
If this is an OOM bug, add the **-rss_limit_mb=2048** argument.
Read more on how timeouts and OOMs are handed
[here]({{ site.baseurl }}/faq/#how-do-you-handle-timeouts-and-ooms).
Depending on the nature of the bug, the fuzz target binary needs to be built
with the appropriate [sanitizer](https://github.com/google/sanitizers)
(e.g. if this is a buffer overflow, with
[AddressSanitizer](http://clang.llvm.org/docs/AddressSanitizer.html)).
If you are not sure how to build the fuzzer using the project's build system,
you may also use Docker
([how?]({{ site.baseurl }}/getting-started/new-project-guide/#prerequisites),
[why?]({{ site.baseurl }}/faq/#why-do-you-use-docker)) commands
to replicate the exact build steps used by OSS-Fuzz and then feed the reproducer
input to the fuzz target.
## Building using Docker
### Pull the latest Docker images
```bash
$ python infra/helper.py pull_images
```
Docker images get regularly updated with a newer version of build tools, build
configurations, scripts, and other changes. In some cases, a particular issue
can be reproduced only with a fresh image being used.
### Build the image and the fuzzers
```bash
$ python infra/helper.py build_image $PROJECT_NAME
$ python infra/helper.py build_fuzzers --sanitizer <address/memory/undefined> $PROJECT_NAME
```
## Reproducing bugs
```bash
$ python infra/helper.py reproduce $PROJECT_NAME <fuzz_target_name> <testcase_path>
```
Find the type of sanitizer used in the report using the value in the
**Sanitizer** column. It is one of the following:
* **address** for AddressSanitizer
* **memory** for MemorySanitizer
* **undefined** for UndefinedBehaviorSanitizer
E.g. for building [libxml2](https://github.com/google/oss-fuzz/tree/master/projects/libxml2)
project with UndefinedBehaviorSanitizer (undefined) instrumentation and
reproduce a crash testcase for a fuzzer named `libxml2_xml_read_memory_fuzzer`,
it will be:
```bash
$ python infra/helper.py build_image libxml2
$ python infra/helper.py build_fuzzers --sanitizer undefined libxml2
$ python infra/helper.py reproduce libxml2 libxml2_xml_read_memory_fuzzer ~/Downloads/testcase
```
## Reproduce using local source checkout
```bash
$ python infra/helper.py build_fuzzers --sanitizer <address/memory/undefined> $PROJECT_NAME <source_path>
$ python infra/helper.py reproduce $PROJECT_NAME <fuzz_target_name> <testcase_path>
```
This is essentially the previous command that additionally mounts local sources
into the running container.
- *Fix issue*. Write a patch to fix the issue in your local checkout and then
use the previous command to verify the fix (i.e. no crash occurred).
[Use gdb]({{ site.baseurl }}/advanced-topics/debugging/#debugging-fuzzers-with-gdb)
if needed.
- *Submit fix*. Submit the fix in the project's repository. ClusterFuzz will
automatically pick up the changes, recheck the testcase and will close the
issue (in &lt; 1 day).
- *Improve fuzzing support*. Consider
[improving fuzzing support]({{ site.baseurl }}/advanced-topics/ideal-integration/)
in your project's build and test system.
## Reproducing build failures
Our infrastructure runs some sanity tests to make sure that your build was
correctly configured, even if it succeeded. To reproduce these locally, run:
```bash
$ python infra/helper.py build_image $PROJECT_NAME
$ python infra/helper.py build_fuzzers --sanitizer <address/memory/undefined> --engine <libfuzzer/afl/honggfuzz> $PROJECT_NAME
$ python infra/helper.py check_build --sanitizer <address/memory/undefined> --engine <libfuzzer/afl/honggfuzz> $PROJECT_NAME <fuzz_target_name>
```
For reproducing a `coverage` build failure, follow
[Code Coverage page]({{ site.baseurl }}/advanced-topics/code-coverage) to build
your project and generate a code coverage report.

52
docs/clusterfuzz.md
View File

@ -1,52 +0,0 @@
# ClusterFuzz
[ClusterFuzz](https://github.com/google/clusterfuzz) is the distributed fuzzing infrastructure behind OSS-Fuzz. It was initially built
for fuzzing Chrome at scale.
## Web interface
ClusterFuzz provides a [web interface](https://oss-fuzz.com)
to view statistics about your fuzz targets, as well as current crashes.
*Note*: Access is restricted to project developers who we auto CC on new bug reports.
## Testcase reports
ClusterFuzz will automatically de-duplicate and file reproducible crashes into
our [bug tracker](https://bugs.chromium.org/p/monorail). We provide a crash
report page that gives you the stack trace, a link to the crashing testcase, and
regression ranges where the bug was most likely introduced.
![report](images/pcre2_testcase.png?raw=true)
## Fuzzer stats
You can view statistics about your fuzz targets (e.g. speed, coverage information,
memory usage) on our fuzzer statistics dashboard.
![stats](images/freetype_stats_graphs.png?raw=true)
![stats](images/freetype_stats_table.png?raw=true)
## Coverage reports
We provide coverage reports, where we highlight the parts of source code that are being
reached by your fuzz target. Make sure to look at the uncovered code marked in red and
add appropriate fuzz targets to cover those usecases.
![coverage_1](images/freetype_coverage_1.png?raw=true)
![coverage_2](images/freetype_coverage_2.png?raw=true)
## Performance analyzer
You can view performance issues that your fuzz target is running into (e.g. leaks, timeouts,
etc) by clicking on `Performance` link on our fuzzer statistics dashboard. Make sure to fix
all cited issues, so as to keep your fuzz target running efficiently and finding new bugs.
![performance_analyzer](images/expat_performance_analyzer.png?raw=true)
## Crash stats
You can view statistics of crashes over time on our crash statistics dashboard.
![crash_stats](images/crash_stats.png?raw=true)

39
docs/corpora.md
View File

@ -1,39 +0,0 @@
# Accessing Corpora
If you would like to access the corpora that we are using for your fuzz targets (synthesized by the fuzzing engines), please follow these steps.
## Obtain access
In order to get access to a project's corpora, you must first be listed as the primary contact or an auto cc in the project's `project.yaml` file as described in the [New Project Guide](new_project_guide.md#projectyaml). If you don't do this, most of the links below won't work.
## Install Google Cloud SDK
The corpora for fuzz targets are stored on [Google Cloud Storage](https://cloud.google.com/storage/). To access them, you will need to [install](https://cloud.google.com/storage/docs/gsutil_install) the gsutil tool, which is part of the Google Cloud SDK.
Follow the instructions on the installation page to login with a Google account listed in your project's `project.yaml` file.
## Viewing the corpus for a fuzz target
The fuzzer statistics page for your project on [ClusterFuzz](clusterfuzz.md) will contain a link to the Google Cloud console for your corpus under the "corpus_size" column. You can browse and download individual test inputs in the corpus here.
![viewing_corpus](https://raw.githubusercontent.com/google/oss-fuzz/master/docs/images/viewing_corpus.png)
## Downloading the corpus
If you would like to download the entire corpus, from the cloud console link, copy the bucket path highlighted here:
![corpus_path](https://raw.githubusercontent.com/google/oss-fuzz/master/docs/images/corpus_path.png)
And then run the following command to copy the corpus to a directory on your machine.
```bash
gsutil -m rsync gs://<bucket_path> <local_directory>
```
Following the expat example above, this would be:
```bash
gsutil -m rsync gs://expat-corpus.clusterfuzz-external.appspot.com/libFuzzer/expat_parse_fuzzer <local_directory>
```
## Corpus backups
We also keep daily zipped backups of your corpora. These can be accessed from the `corpus_backup` column of the fuzzer statistics page. Downloading these can also be significantly faster than `gsutil -m rsync` on the corpus bucket.

117
docs/faq.md
View File

@ -1,15 +1,29 @@
---
layout: default
title: FAQ
has_children: true
nav_order: 7
permalink: /faq/
---
# Frequently Asked Questions # Frequently Asked Questions
- TOC
{:toc}
---
## What kind of projects are you accepting? ## What kind of projects are you accepting?
We accept established projects that have a critical impact on infrastructure and user security. We accept established projects that have a critical impact on infrastructure and
We will consider each request on a case-by-case basis, but some things we keep in mind are: user security. We will consider each request on a case-by-case basis, but some
things we keep in mind are:
- Exposure to remote attacks (e.g. libraries that are used to process untrusted input). - Exposure to remote attacks (e.g. libraries that are used to process
untrusted input).
- Number of users/other projects depending on this project. - Number of users/other projects depending on this project.
We hope to relax this requirement in the future though, so keep an eye out even if we are not able We hope to relax this requirement in the future though, so keep an eye out even
to accept your project at this time! if we are not able to accept your project at this time!
## How can I find potential fuzz targets in my open source project? ## How can I find potential fuzz targets in my open source project?
@ -17,8 +31,10 @@ You should look for places in your code that:
- consume un-trusted data from users or from the network. - consume un-trusted data from users or from the network.
- consume complex data input or event if it's 'trusted'. - consume complex data input or event if it's 'trusted'.
- use an algorithm that has two or more implementations (to verify their equivalence). - use an algorithm that has two or more implementations
- look for existing fuzz target [examples](https://github.com/google/oss-fuzz/tree/master/projects) and find similarities. (to verify their equivalence).
- look for existing fuzz target [examples](https://github.com/google/oss-fuzz/tree/master/projects)
and find similarities.
## My project is not open source. Can I use OSS-Fuzz? ## My project is not open source. Can I use OSS-Fuzz?
@ -40,18 +56,23 @@ We will reconsider the GitHub issue tracker once the
## Why do you require a Google account for authentication? ## Why do you require a Google account for authentication?
Our [ClusterFuzz](clusterfuzz.md) fuzzing infrastructure and [issue tracker](https://bugs.chromium.org/p/oss-fuzz/issues/list) require a Google account for authentication. Note that an alternate email address associated with a Google account does not work due to appengine api limitations. Our [ClusterFuzz]({{ site.baseurl }}/furthur-reading/clusterfuzz) fuzzing
infrastructure and [issue tracker](https://bugs.chromium.org/p/oss-fuzz/issues/list)
require a Google account for authentication. Note that an alternate email
address associated with a Google account does not work due to appengine api
limitations.
## Why do you use Docker? ## Why do you use Docker?
Building fuzzers requires building your project with a fresh Clang compiler and special compiler flags. Building fuzzers requires building your project with a fresh Clang compiler and
An easy-to-use Docker image is provided to simplify toolchain distribution. This also simplifies our special compiler flags. An easy-to-use Docker image is provided to simplify
support for a variety of Linux distributions and provides a reproducible and secure environment for fuzzer toolchain distribution. This also simplifies our support for a variety of Linux
distributions and provides a reproducible and secure environment for fuzzer
building and execution. building and execution.
## How do you handle timeouts and OOMs? ## How do you handle timeouts and OOMs?
If a single input to a [fuzz target](glossary.md#fuzz-target) If a single input to a [fuzz target]({{ site.baseurl }}/reference/glossary/#fuzz-target)
requires more than **~25 seconds** or more than **2GB RAM** to process, we requires more than **~25 seconds** or more than **2GB RAM** to process, we
report this as a timeout or an OOM (out-of-memory) bug report this as a timeout or an OOM (out-of-memory) bug
(examples: [timeouts](https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=%22Crash+Type%3A+Timeout%22), (examples: [timeouts](https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=%22Crash+Type%3A+Timeout%22),
@ -72,15 +93,15 @@ Currently we do not offer ways to change the memory and time limits.
## Can I launch an additional process (e.g. a daemon) from my fuzz target? ## Can I launch an additional process (e.g. a daemon) from my fuzz target?
No. In order to get all the benefits of in-process, coverage-guided fuzz testing, No. In order to get all the benefits of in-process, coverage-guided fuzz testing,
it is required to run everything inside a single process. Any child processes created it is required to run everything inside a single process. Any child processes
outside the main process introduces heavy launch overhead and is not monitored for created outside the main process introduces heavy launch overhead and is not
code coverage. monitored for code coverage.
Another rule of thumb is: "the smaller fuzz target is, the better it is". It is Another rule of thumb is: "the smaller fuzz target is, the better it is". It is
expected that your project will have many fuzz targets to test different components, expected that your project will have many fuzz targets to test different
instead of a single fuzz target trying to cover everything. Think of fuzz target components, instead of a single fuzz target trying to cover everything.
as a unit test, though it is much more powerful since it helps to test millions Think of fuzz target as a unit test, though it is much more powerful since it
of data permutations rather than just one. helps to test millions of data permutations rather than just one.
## What if my fuzz target finds a bug in another project (dependency) ? ## What if my fuzz target finds a bug in another project (dependency) ?
@ -89,20 +110,32 @@ Using that, you can debug the root cause and see which category the bug falls in
- If this is a bug is due to an incorrect usage of the dependent project's API - If this is a bug is due to an incorrect usage of the dependent project's API
in your project, then you need to fix your usage to call the API correctly. in your project, then you need to fix your usage to call the API correctly.
- If this is a real bug in the dependent project, then you should CC the maintainers - If this is a real bug in the dependent project, then you should CC the
of that project on the bug. Once CCed, they will get automatic access to all the maintainers of that project on the bug. Once CCed, they will get automatic
information necessary to reproduce the issue. If this project is maintained in OSS-Fuzz, access to all the information necessary to reproduce the issue. If this project
you can search for contacts in the respective project.yaml file. is maintained in OSS-Fuzz, you can search for contacts in the respective
project.yaml file.
## What if my fuzzer does not find anything? ## What if my fuzzer does not find anything?
If your fuzz target is running for many days and does not find bugs or new coverage, it may mean several things: If your fuzz target is running for many days and does not find bugs or new
- We've covered all reachable code. In order to cover more code we need more fuzz targets. coverage, it may mean several things:
- The [seed corpus](new_project_guide.md#seed-corpus) is not good enough and the fuzzing engine(s) are not able to go deeper based on the existing seeds. Need to add more seeds. - We've covered all reachable code. In order to cover more code we need more
- There is some crypto/crc stuff in the code that will prevent any fuzzing engine from going deeper, in which case the crypto should be disabled in [fuzzing mode](http://libfuzzer.info#fuzzer-friendly-build-mode). Examples: [openssl](https://github.com/openssl/openssl/tree/master/fuzz#reproducing-issues), [boringssl](https://boringssl.googlesource.com/boringssl/+/HEAD/FUZZING.md#Fuzzer-mode) fuzz targets.
- It is also possible that the fuzzer is running too slow (you may check the speed of your targets at https://oss-fuzz.com/) - The [seed corpus]({{ site.baseurl }}/getting-started/new-project-guide#seed-corpus) is not good enough and the
fuzzing engine(s) are not able to go deeper based on the existing seeds.
Need to add more seeds.
- There is some crypto/crc stuff in the code that will prevent any fuzzing
engine from going deeper, in which case the crypto should be disabled in
[fuzzing mode](http://libfuzzer.info#fuzzer-friendly-build-mode).
Examples: [openssl](https://github.com/openssl/openssl/tree/master/fuzz#reproducing-issues),
[boringssl](https://boringssl.googlesource.com/boringssl/+/HEAD/FUZZING.md#Fuzzer-mode)
- It is also possible that the fuzzer is running too slow
(you may check the speed of your targets at https://oss-fuzz.com/)
In either case, look at the [coverage reports](clusterfuzz.md#coverage-reports) for your target(s) and figure out why some parts of the code are not covered. In either case, look at the
[coverage reports]({{ site.baseurl }}/furthur-reading/clusterfuzz#coverage-reports)
for your target(s) and figure out why some parts of the code are not covered.
## Why are code coverage reports public? ## Why are code coverage reports public?
@ -111,19 +144,33 @@ possible. We believe that public code coverage reports do not put users at risk,
as they do not indicate the presence of bugs or lack thereof. as they do not indicate the presence of bugs or lack thereof.
## What happens when I rename a fuzz target ? ## What happens when I rename a fuzz target ?
If you rename your fuzz targets, the existing bugs for those targets will get closed and fuzzing will start from scratch from a fresh corpora (seed corpus only). Similar corpora will get accumulated over time depending on the number of cpu cycles that original fuzz target has run. If this is not desirable, make sure to copy the accumulated corpora from the original fuzz target (instructions to download [here](corpora.md#downloading-the-corpus)) and restore it to the new GCS location later (instruction to find the new location [here](corpora.md#viewing-the-corpus-for-a-fuzz-target)). If you rename your fuzz targets, the existing bugs for those targets will get
closed and fuzzing will start from scratch from a fresh corpora
(seed corpus only). Similar corpora will get accumulated over time depending on
the number of cpu cycles that original fuzz target has run. If this is not
desirable, make sure to copy the accumulated corpora from the original fuzz
target (instructions to download
[here]({{ site.baseurl }}/advanced-topics/corpora/#downloading-the-corpus)) and
restore it to the new GCS location later (instruction to find the
new location [here]({{ site.baseurl }}/advanced-topics/corpora/#viewing-the-corpus-for-a-fuzz-target)).
## Does OSS-Fuzz support AFL? ## Does OSS-Fuzz support AFL?
OSS-Fuzz *uses* [AFL](http://lcamtuf.coredump.cx/afl/) as one of its [fuzzing engines](glossary.md#fuzzing-engine) but this is an implementation detail. Just follow the [ideal integration guide](ideal_integration.md) and OSS-Fuzz will use all its fuzzing engines on your code. OSS-Fuzz *uses* [AFL](http://lcamtuf.coredump.cx/afl/) as one of its
[fuzzing engines]({{ site.baseurl }}/reference/glossary/#fuzzing-engine) but this is an implementation
detail. Just follow the
[ideal integration guide]({{ site.baseurl }}/advanced-topics/ideal-integration/)
and OSS-Fuzz will use all its fuzzing engines on your code.
## Does OSS-Fuzz support Honggfuzz? ## Does OSS-Fuzz support Honggfuzz?
OSS-Fuzz (the service) does not use honggfuzz to find bugs. OSS-Fuzz (the service) does not use honggfuzz to find bugs.
That is because [ClusterFuzz](https://github.com/google/clusterfuzz) (the infrastructure OSS-Fuzz runs on) does not support honggfuzz. That is because [ClusterFuzz](https://github.com/google/clusterfuzz)
However, OSS-Fuzz supports **building** fuzz targets for use with honggfuzz, similar to the way it (the infrastructure OSS-Fuzz runs on) does not support honggfuzz.
supports [AFL](#does-oss-fuzz-support-afl). However, OSS-Fuzz supports **building** fuzz targets for use with honggfuzz,
similar to the way it supports [AFL](#does-oss-fuzz-support-afl).
Currently OSS-Fuzz builders do builds for libFuzzer, AFL, and honggfuzz. Currently OSS-Fuzz builders do builds for libFuzzer, AFL, and honggfuzz.
## What are the specs on your machines? ## What are the specs on your machines?
OSS-Fuzz builders have 32CPU/28.8GB RAM. OSS-Fuzz builders have 32CPU/28.8GB RAM.
Fuzzing machines only have a single core and fuzz targets should not use more than 2GB of RAM. Fuzzing machines only have a single core and fuzz targets should not use more
than 2GB of RAM.

67
docs/furthur-reading/clusterfuzz.md Normal file
View File

@ -0,0 +1,67 @@
---
layout: default
title: ClusterFuzz
parent: Furthur reading
nav_order: 1
permalink: /furthur-reading/clusterfuzz/
---
# ClusterFuzz
[ClusterFuzz](https://github.com/google/clusterfuzz) is the distributed fuzzing
infrastructure behind OSS-Fuzz. It was initially built for fuzzing Chrome at
scale.
- TOC
{:toc}
---
## Web interface
ClusterFuzz provides a [web interface](https://oss-fuzz.com)
to view statistics about your fuzz targets, as well as current crashes.
*Note*: Access is restricted to project developers who we auto CC on new bug
reports.
## Testcase reports
ClusterFuzz will automatically de-duplicate and file reproducible crashes into
our [bug tracker](https://bugs.chromium.org/p/oss-fuzz/issues/list). We provide
a crash report page that gives you the stack trace, a link to the crashing
testcase, and regression ranges where the bug was most likely introduced.
![report]({{ site.baseurl }}/images/pcre2_testcase.png?raw=true)
## Fuzzer stats
You can view statistics about your fuzz targets (e.g. speed, coverage
information, memory usage) on our fuzzer statistics dashboard.
![stats]({{ site.baseurl }}/images/freetype_stats_graphs.png?raw=true)
![stats]({{ site.baseurl }}/images/freetype_stats_table.png?raw=true)
## Coverage reports
We provide coverage reports, where we highlight the parts of source code that
are being reached by your fuzz target. Make sure to look at the uncovered code
marked in red and add appropriate fuzz targets to cover those usecases.
![coverage_1]({{ site.baseurl }}/images/freetype_coverage_1.png?raw=true)
![coverage_2]({{ site.baseurl }}/images/freetype_coverage_2.png?raw=true)
## Performance analyzer
You can view performance issues that your fuzz target is running into (e.g.
leaks, timeouts, etc) by clicking on `Performance` link on our fuzzer statistics
dashboard. Make sure to fix all cited issues, so as to keep your fuzz target
running efficiently and finding new bugs.
![performance_analyzer]({{ site.baseurl }}/images/expat_performance_analyzer.png?raw=true)
## Crash stats
You can view statistics of crashes over time on our crash statistics dashboard.
![crash_stats]({{ site.baseurl }}/images/crash_stats.png?raw=true)

9
docs/furthur-reading/furthur_reading.md Normal file
View File

@ -0,0 +1,9 @@
---
layout: default
title: Furthur reading
has_children: true
nav_order: 4
permalink: /furthur-reading/
---
# Furthur reading

46
docs/fuzzer_environment.md → docs/furthur-reading/fuzzer_environment.md
View File

@ -1,14 +1,28 @@
---
layout: default
title: Fuzzer environment
parent: Furthur reading
nav_order: 2
permalink: /furthur-reading/fuzzer-environment/
---
# Fuzzer environment on ClusterFuzz # Fuzzer environment on ClusterFuzz
Your fuzz targets will be run on a [Google Compute Engine](https://cloud.google.com/compute/) VM (Linux) with some [security restrictions](https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-runner/run_minijail). Your fuzz targets will be run on a
[Google Compute Engine](https://cloud.google.com/compute/) VM (Linux) with some
[security restrictions](https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-runner/run_minijail).
- TOC
{:toc}
---
## Runtime Dependencies ## Runtime Dependencies
You should not make any assumptions on the availability of dependent packages You should not make any assumptions on the availability of dependent packages
in the execution environment. Packages that are installed via in the execution environment. Packages that are installed via
[Dockerfile](new_project_guide.md#dockerfile) [Dockerfile]({{ site.baseurl }}/getting-started/new-project-guide/#dockerfile)
or built as part of or built as part of
[build.sh](new_project_guide.md#buildsh) [build.sh]({{ site.baseurl }}/getting-started/new-project-guide/#buildsh)
are not available on the bot runtime environment (where the fuzz targets run). are not available on the bot runtime environment (where the fuzz targets run).
If you need these dependencies in the runtime environment, you can either If you need these dependencies in the runtime environment, you can either
@ -17,22 +31,24 @@ If you need these dependencies in the runtime environment, you can either
and then link statically against them and then link statically against them
([example](https://github.com/google/oss-fuzz/blob/master/projects/tor/build.sh#L40)) ([example](https://github.com/google/oss-fuzz/blob/master/projects/tor/build.sh#L40))
- Or build the dependencies statically in - Or build the dependencies statically in
[build.sh](new_project_guide.md#buildsh) [build.sh]({{ site.baseurl }}/getting-started/new-project-guide/#buildsh)
([example](https://github.com/google/oss-fuzz/blob/master/projects/ffmpeg/build.sh#L26)). ([example](https://github.com/google/oss-fuzz/blob/master/projects/ffmpeg/build.sh#L26)).
All build artifacts needed during fuzz target execution should be inside the `$OUT` All build artifacts needed during fuzz target execution should be inside the
directory. Only those artifacts are archived and used on the bots. Everything else `$OUT` directory. Only those artifacts are archived and used on the bots.
is ignored (e.g. artifacts in `$WORK`, `$SRC`, etc) and hence is not available Everything else is ignored (e.g. artifacts in `$WORK`, `$SRC`, etc) and hence
in the execution environment. is not available in the execution environment.
You should ensure that the fuzz target works correctly by using `run_fuzzer` command You should ensure that the fuzz target works correctly by using `run_fuzzer`
(see instructions [here](new_project_guide.md#testing-locally)). This command uses command (see instructions
a clean base-runner docker container and not the base-builder docker container [here]({{ site.baseurl }}/getting-started/new-project-guide/#testing-locally)).
created during build-time. This command uses a clean base-runner docker container and not the base-builder
docker container created during build-time.
## argv[0] ## argv[0]
You must not modify `argv[0]`. It is required for certain things to work correctly. You must not modify `argv[0]`. It is required for certain things to work
correctly.
## Current working directory ## Current working directory
@ -42,8 +58,8 @@ directory where your fuzz target executable is located.
## File system ## File system
Everything except `/tmp` is read-only, including the directory that your fuzz target Everything except `/tmp` is read-only, including the directory that your fuzz
executable lives in. target executable lives in.
`/dev` is also unavailable. `/dev` is also unavailable.

38
docs/getting-started/accepting_new_projects.md Normal file
View File

@ -0,0 +1,38 @@
---
layout: default
title: Accepting new projects
parent: Getting started
nav_order: 1
permalink: /getting-started/accepting-new-projects/
---
## Accepting New Projects
To be accepted to OSS-Fuzz, an open-source project must
have a significant user base and/or be critical to the global IT infrastructure.
To submit a new project:
* [Create a pull request](https://help.github.com/articles/creating-a-pull-request/)
with new `projects/<project_name>/project.yaml` file
([example](https://github.com/google/oss-fuzz/tree/master/projects/libarchive/project.yaml))
giving at least the following information:
* project homepage.
* e-mail of the engineering contact person to be CCed on new issues. It should:
* belong to an established project committer (according to VCS logs).
If this is not you or the email address differs from VCS, an informal
e-mail verification will be required.
* be associated with a Google account
([why?]({{ site.baseurl }}/faq/#why-do-you-require-a-google-account-for-authentication)).
If you use an alternate email address
[linked to a Google Account](https://support.google.com/accounts/answer/176347?hl=en),
it will ONLY give you access to filed bugs in
[issue tracker](https://bugs.chromium.org/p/oss-fuzz/issues/list) and
NOT to [ClusterFuzz]({{ site.baseurl }}/furthur-reading/clusterfuzz)
dashboard (due to appengine api limitations).
* Note that `project_name` can only contain alphanumeric characters,
underscores(_) or dashes(-).
* Once accepted by an OSS-Fuzz project member, follow the
[New Project Guide]({{ site.baseurl }}/getting-started/new-project-guide/)
to configure your project.

23
docs/getting-started/bug_disclosure_guidelines.md Normal file
View File

@ -0,0 +1,23 @@
---
layout: default
title: Bug disclosure guidelines
parent: Getting started
nav_order: 4
permalink: /getting-started/bug-disclosure-guidelines/
---
## Bug Disclosure Guidelines
Following [Google's standard disclosure policy](https://googleprojectzero.blogspot.com/2015/02/feedback-and-data-driven-updates-to.html),
OSS-Fuzz will adhere to following disclosure principles:
- **Deadline**. After notifying project authors, we will open reported
issues to the public in 90 days, or 30 days after the fix is released
(whichever comes earlier).
- **Weekends and holidays**. If a deadline is due to expire on a weekend,
the deadline will be moved to the next normal work day.
- **Grace period**. We have a 14-day grace period. If a 90-day deadline
expires but the upstream engineers let us know before the deadline that a
patch is scheduled for release on a specific day within 14 days following
the deadline, the public disclosure will be delayed until the availability
of the patch.

11
docs/getting-started/getting_started.md Normal file
View File

@ -0,0 +1,11 @@
---
layout: default
title: Getting started
has_children: true
nav_order: 2
permalink: /getting-started/
---
# Getting started
These pages walk you through the process of integrating your open source project
with OSS-Fuzz.

13
docs/getting-started/integration_rewards.md Normal file
View File

@ -0,0 +1,13 @@
---
layout: default
title: Integration rewards
parent: Getting started
nav_order: 3
permalink: /getting-started/integration-rewards/
---
# Integration rewards
We encourage you to apply for integration rewards (upto **$20,000**) once your project is
successfully integrated with OSS-Fuzz. Please see the details in our blog post
[here](https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html).

79
docs/new_project_guide.md → docs/getting-started/new_project_guide.md
View File

@ -1,8 +1,16 @@
---
layout: default
title: New project guide
parent: Getting started
nav_order: 2
permalink: /getting-started/new-project-guide/
---
# Setting up a New Project # Setting up a New Project
## Prerequisites ## Prerequisites
- [Integrate](ideal_integration.md) one or more [Fuzz Targets](glossary.md#fuzz-target) - [Integrate]({{ site.baseurl }}/advanced-topics/ideal-integration/) one or more [Fuzz Targets]({{ site.baseurl }}/reference/glossary/#fuzz-target)
with the project you want to fuzz.<BR> with the project you want to fuzz.
Examples: Examples:
[boringssl](https://github.com/google/boringssl/tree/master/fuzz), [boringssl](https://github.com/google/boringssl/tree/master/fuzz),
[SQLite](https://www.sqlite.org/src/artifact/ad79e867fb504338), [SQLite](https://www.sqlite.org/src/artifact/ad79e867fb504338),
@ -14,21 +22,29 @@
[pcre2](http://vcs.pcre.org/pcre2/code/trunk/src/pcre2_fuzzsupport.c?view=markup), [pcre2](http://vcs.pcre.org/pcre2/code/trunk/src/pcre2_fuzzsupport.c?view=markup),
[ffmpeg](https://github.com/FFmpeg/FFmpeg/blob/master/tools/target_dec_fuzzer.c). [ffmpeg](https://github.com/FFmpeg/FFmpeg/blob/master/tools/target_dec_fuzzer.c).
- [Install Docker](installing_docker.md). ([Why Docker?](faq.md#why-do-you-use-docker)) - Install Docker using the instructions
[here](https://docs.docker.com/engine/installation).
Googlers: [go/installdocker](https://goto.google.com/installdocker).
[Why Docker?]({{ site.baseurl }}/faq/#why-do-you-use-docker)
*NOTE: (Optional) If you want to run `docker` without `sudo`, follow the
[Create a docker group](https://docs.docker.com/engine/installation/linux/ubuntulinux/#/create-a-docker-group) section.*
*NOTE: Docker images can consume significant disk space. Run*
*[docker-cleanup](https://gist.github.com/mikea/d23a839cba68778d94e0302e8a2c200f)*
*periodically to garbage collect unused images.*
## Overview ## Overview
To add a new OSS project to OSS-Fuzz, you need a project subdirectory To add a new OSS project to OSS-Fuzz, you need a project subdirectory
inside the [`projects/`](../projects) directory in [OSS-Fuzz repository](https://github.com/google/oss-fuzz). inside the [`projects/`](https://github.com/google/oss-fuzz/tree/master/projects) directory in [OSS-Fuzz repository](https://github.com/google/oss-fuzz).
Example: [boringssl](https://github.com/google/boringssl) project is located in Example: [boringssl](https://github.com/google/boringssl) project is located in
[`projects/boringssl`](../projects/boringssl). [`projects/boringssl`](https://github.com/google/oss-fuzz/tree/master/projects/boringssl).
The project directory needs to contain the following three configuration files: The project directory needs to contain the following three configuration files:
* `projects/<project_name>/project.yaml` - provides metadata about the project. * `projects/<project_name>/project.yaml` - provides metadata about the project.
* `projects/<project_name>/Dockerfile` - defines the container environment with information * `projects/<project_name>/Dockerfile` - defines the container environment with information
on dependencies needed to build the project and its [fuzz targets](glossary.md#fuzz-target). on dependencies needed to build the project and its [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target).
* `projects/<project_name>/build.sh` - build script that executes inside the container and * `projects/<project_name>/build.sh` - build script that executes inside the container and
generates project build. generates project build.
@ -42,7 +58,7 @@ $ export PROJECT_NAME=<project_name>
$ python infra/helper.py generate $PROJECT_NAME $ python infra/helper.py generate $PROJECT_NAME
``` ```
It is preferred to keep and maintain [fuzz targets](glossary.md#fuzz-target) in your own source code repository. If this is not possible due to various reasons, you can store them inside the OSS-Fuzz's project directory created above. It is preferred to keep and maintain [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) in your own source code repository. If this is not possible due to various reasons, you can store them inside the OSS-Fuzz's project directory created above.
## project.yaml ## project.yaml
@ -55,7 +71,7 @@ Project's homepage.
Primary contact and CCs list. These people get access to ClusterFuzz Primary contact and CCs list. These people get access to ClusterFuzz
which includes crash reports, fuzzer statistics, etc and are auto-cced on newly filed bugs in OSS-Fuzz which includes crash reports, fuzzer statistics, etc and are auto-cced on newly filed bugs in OSS-Fuzz
tracker. To get full access to these artifacts, you should use a [Google account](https://support.google.com/accounts/answer/176347?hl=en) tracker. To get full access to these artifacts, you should use a [Google account](https://support.google.com/accounts/answer/176347?hl=en)
here ([why?](faq.md#why-do-you-require-a-google-account-for-authentication)). here ([why?]({{ site.baseurl }}/faq/#why-do-you-require-a-google-account-for-authentication)).
### sanitizers (optional) ### sanitizers (optional)
List of sanitizers to use. By default, it will use the default list of supported List of sanitizers to use. By default, it will use the default list of supported
@ -74,7 +90,7 @@ You may opt-in by adding "memory" to this list.
If you want to test a particular sanitizer (e.g. memory) and see what crashes it generates without filing If you want to test a particular sanitizer (e.g. memory) and see what crashes it generates without filing
them in the issue tracker, you can set the experimental flag. The crashes can be accessed on [ClusterFuzz them in the issue tracker, you can set the experimental flag. The crashes can be accessed on [ClusterFuzz
homepage](clusterfuzz.md#web-interface). Example: homepage]({{ site.baseurl }}/furthur-reading/clusterfuzz#web-interface). Example:
``` ```
sanitizers: sanitizers:
@ -88,7 +104,7 @@ Example: [boringssl](https://github.com/google/oss-fuzz/blob/master/projects/bor
### help_url ### help_url
Link to a custom help URL in bug reports instead of the Link to a custom help URL in bug reports instead of the
[default OSS-Fuzz guide to reproducing crashes](reproducing.md). This can be useful if you assign [default OSS-Fuzz guide to reproducing crashes]({{ site.baseurl }}/advanced-topics/reproducing/). This can be useful if you assign
bugs to members of your project unfamiliar with OSS-Fuzz or if they should follow a different workflow for bugs to members of your project unfamiliar with OSS-Fuzz or if they should follow a different workflow for
reproducing and fixing bugs than standard one outlined in the reproducing guide. reproducing and fixing bugs than standard one outlined in the reproducing guide.
@ -96,7 +112,7 @@ Example: [skia](https://github.com/google/oss-fuzz/blob/master/projects/skia/pro
### experimental ### experimental
A boolean (either True or False) that indicates whether this project is in evaluation mode. This allows a project to be A boolean (either True or False) that indicates whether this project is in evaluation mode. This allows a project to be
fuzzed and generate crash findings, but not file them in the issue tracker. The crashes can be accessed on [ClusterFuzz homepage](clusterfuzz.md#web-interface). This should be only used if you are not a maintainer of the project and have fuzzed and generate crash findings, but not file them in the issue tracker. The crashes can be accessed on [ClusterFuzz homepage]({{ site.baseurl }}/furthur-reading/clusterfuzz#web-interface). This should be only used if you are not a maintainer of the project and have
less confidence in the efficacy of your fuzz targets. Example: less confidence in the efficacy of your fuzz targets. Example:
``` ```
@ -124,20 +140,20 @@ RUN git clone <git_url> <checkout_dir> # checkout all sources needed to build
WORKDIR <checkout_dir> # current directory for build script WORKDIR <checkout_dir> # current directory for build script
COPY build.sh fuzzer.cc $SRC/ # copy build script and other fuzzer files in src dir COPY build.sh fuzzer.cc $SRC/ # copy build script and other fuzzer files in src dir
``` ```
Expat example: [expat/Dockerfile](../projects/expat/Dockerfile) Expat example: [expat/Dockerfile](https://github.com/google/oss-fuzz/tree/master/projects/expat/Dockerfile)
In the above example, the git clone will check out the source to `$SRC/<checkout_dir>`. In the above example, the git clone will check out the source to `$SRC/<checkout_dir>`.
## build.sh ## build.sh
This file describes how to build binaries for [fuzz targets](glossary.md#fuzz-target) in your project. This file describes how to build binaries for [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) in your project.
The script will be executed within the image built from `Dockerfile`. The script will be executed within the image built from `Dockerfile`.
In general, this script will need to: In general, this script will need to:
1. Build the project using your build system *with* correct compiler and its flags provided as 1. Build the project using your build system *with* correct compiler and its flags provided as
*environment variables* (see below). *environment variables* (see below).
2. Build the [fuzz targets](glossary.md#fuzz-target), linking your project's build and libFuzzer. 2. Build the [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target), linking your project's build and libFuzzer.
Resulting binaries should be placed in `$OUT`. Resulting binaries should be placed in `$OUT`.
*Note*: *Note*:
@ -145,7 +161,7 @@ In general, this script will need to:
1. Please don't assume that the fuzzing engine is libFuzzer and hardcode in your build scripts. 1. Please don't assume that the fuzzing engine is libFuzzer and hardcode in your build scripts.
We generate builds for both libFuzzer and AFL fuzzing engine configurations. We generate builds for both libFuzzer and AFL fuzzing engine configurations.
So, link the fuzzing engine using $LIB_FUZZING_ENGINE, see example below. So, link the fuzzing engine using $LIB_FUZZING_ENGINE, see example below.
2. Please make sure that the binary names for your [fuzz targets](glossary.md#fuzz-target) contain only 2. Please make sure that the binary names for your [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) contain only
alphanumeric characters, underscore(_) or dash(-). Otherwise, they won't run on our infrastructure. alphanumeric characters, underscore(_) or dash(-). Otherwise, they won't run on our infrastructure.
3. Please don't remove source code files. They are needed for code coverage. 3. Please don't remove source code files. They are needed for code coverage.
@ -199,7 +215,7 @@ You *must* use `$CXX` as a linker, even if your project is written in pure C.
Most well-crafted build scripts will automatically use these variables. If not, Most well-crafted build scripts will automatically use these variables. If not,
pass them manually to the build tool. pass them manually to the build tool.
See [Provided Environment Variables](../infra/base-images/base-builder/README.md#provided-environment-variables) section in See [Provided Environment Variables](https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-builder/README.md#provided-environment-variables) section in
`base-builder` image documentation for more details. `base-builder` image documentation for more details.
## Disk space restrictions ## Disk space restrictions
@ -210,12 +226,12 @@ In addition to this, please keep the size of the build (everything copied to `$O
## Fuzzer execution environment ## Fuzzer execution environment
[This page](fuzzer_environment.md) gives information about the environment that [This page]({{ site.baseurl }}/furthur-reading/fuzzer-environment/) gives information about the environment that
your [fuzz targets](glossary.md#fuzz-target) will run on ClusterFuzz, and the assumptions that you can make. your [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) will run on ClusterFuzz, and the assumptions that you can make.
## Testing locally ## Testing locally
Use the helper script to build docker image and [fuzz targets](glossary.md#fuzz-target). Use the helper script to build docker image and [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target).
```bash ```bash
$ cd /path/to/oss-fuzz $ cd /path/to/oss-fuzz
@ -241,10 +257,10 @@ $ python infra/helper.py run_fuzzer $PROJECT_NAME <fuzz_target>
``` ```
If everything works locally, then it should also work on our automated builders and ClusterFuzz. If everything works locally, then it should also work on our automated builders and ClusterFuzz.
If it fails, check out [this](fuzzer_environment.md#dependencies) entry. If it fails, check out [this]({{ site.baseurl }}/furthur-reading/fuzzer-environment/#dependencies) entry.
It's recommended to look at code coverage as a sanity check to make sure that It's recommended to look at code coverage as a sanity check to make sure that
[fuzz target](glossary.md#fuzz-target) gets to the code you expect. [fuzz target]({{ site.baseurl }}/reference/glossary/#fuzz-target) gets to the code you expect.
```bash ```bash
$ python infra/helper.py build_fuzzers --sanitizer coverage $PROJECT_NAME $ python infra/helper.py build_fuzzers --sanitizer coverage $PROJECT_NAME
@ -257,8 +273,8 @@ of the supported build configurations with the above commands (build_fuzzers ->
## Debugging Problems ## Debugging Problems
[Debugging](debugging.md) document lists ways to debug your build scripts or [Debugging]({{ site.baseurl }}/advanced-topics/debugging/) document lists ways to debug your build scripts or
[fuzz targets](glossary.md#fuzz-target) [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target)
in case you run into problems. in case you run into problems.
@ -282,7 +298,7 @@ if (size < kMinInputLength || size > kMaxInputLength)
return 0; return 0;
``` ```
For out of tree [fuzz targets](glossary.md#fuzz-target), you will likely add options file using docker's For out of tree [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target), you will likely add options file using docker's
`COPY` directive and will copy it into output in build script. `COPY` directive and will copy it into output in build script.
(example: [woff2](https://github.com/google/oss-fuzz/blob/master/projects/woff2/convert_woff2ttf_fuzzer.options)). (example: [woff2](https://github.com/google/oss-fuzz/blob/master/projects/woff2/convert_woff2ttf_fuzzer.options)).
@ -290,10 +306,10 @@ For out of tree [fuzz targets](glossary.md#fuzz-target), you will likely add opt
### Seed Corpus ### Seed Corpus
OSS-Fuzz uses evolutionary fuzzing algorithms. Supplying seed corpus consisting OSS-Fuzz uses evolutionary fuzzing algorithms. Supplying seed corpus consisting
of good sample inputs is one of the best ways to improve [fuzz target](glossary.md#fuzz-target)'s coverage. of good sample inputs is one of the best ways to improve [fuzz target]({{ site.baseurl }}/reference/glossary/#fuzz-target)'s coverage.
To provide a corpus for `my_fuzzer`, put `my_fuzzer_seed_corpus.zip` file next To provide a corpus for `my_fuzzer`, put `my_fuzzer_seed_corpus.zip` file next
to the [fuzz target](glossary.md#fuzz-target)'s binary in `$OUT` during the build. Individual files in this to the [fuzz target]({{ site.baseurl }}/reference/glossary/#fuzz-target)'s binary in `$OUT` during the build. Individual files in this
archive will be used as starting inputs for mutations. The name of each file in the corpus is the sha1 checksum (which you can get using the `sha1sum` or `shasum` comand) of its contents. You can store the corpus archive will be used as starting inputs for mutations. The name of each file in the corpus is the sha1 checksum (which you can get using the `sha1sum` or `shasum` comand) of its contents. You can store the corpus
next to source files, generate during build or fetch it using curl or any other next to source files, generate during build or fetch it using curl or any other
tool of your choice. tool of your choice.
@ -303,7 +319,7 @@ Seed corpus files will be used for cross-mutations and portions of them might ap
in bug reports or be used for further security research. It is important that corpus in bug reports or be used for further security research. It is important that corpus
has an appropriate and consistent license. has an appropriate and consistent license.
See also [Accessing Corpora](corpora.md) for information about getting access to the corpus we are currently using for your fuzz targets. See also [Accessing Corpora]({{ site.baseurl }}/advanced-topics/corpora/) for information about getting access to the corpus we are currently using for your fuzz targets.
### Dictionaries ### Dictionaries
@ -321,7 +337,7 @@ in .options file:
dict = dictionary_name.dict dict = dictionary_name.dict
``` ```
It is common for several [fuzz targets](glossary.md#fuzz-target) It is common for several [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target)
to reuse the same dictionary if they are fuzzing very similar inputs. to reuse the same dictionary if they are fuzzing very similar inputs.
(example: [expat](https://github.com/google/oss-fuzz/blob/master/projects/expat/parse_fuzzer.options)). (example: [expat](https://github.com/google/oss-fuzz/blob/master/projects/expat/parse_fuzzer.options)).
@ -358,10 +374,11 @@ If you are porting a fuzz target from Chromium, keep the original Chromium licen
## The end ## The end
Once your change is merged, your project and fuzz targets should be automatically built and run on Once your change is merged, your project and fuzz targets should be automatically built and run on
ClusterFuzz after a short while (&lt; 1 day)!<BR><BR> ClusterFuzz after a short while (&lt; 1 day)!
Check your project's build status [here](https://oss-fuzz-build-logs.storage.googleapis.com/index.html).<BR>
Use [ClusterFuzz](clusterfuzz.md) web interface [here](https://oss-fuzz.com/) to checkout the following items: Check your project's build status [here](https://oss-fuzz-build-logs.storage.googleapis.com/index.html).
Use [ClusterFuzz]({{ site.baseurl }}/furthur-reading/clusterfuzz) web interface [here](https://oss-fuzz.com/) to checkout the following items:
* Crashes generated * Crashes generated
* Code coverage statistics * Code coverage statistics
* Fuzzer statistics * Fuzzer statistics

55
docs/glossary.md
View File

@ -1,55 +0,0 @@
# Glossary
For general fuzzing terms, see the [glossary] from [google/fuzzing] project,
[glossary]: https://github.com/google/fuzzing/blob/master/docs/glossary.md
[google/fuzzing]: https://github.com/google/fuzzing
## OSS-Fuzz specific terms
### ClusterFuzz
A scalable fuzzing infrastructure that is used for OSS-Fuzz backend.
[ClusterFuzz] is also used to fuzz Chrome and many other projects. A quick
overview of ClusterFuzz user interface is available on this [page].
[page]: clusterfuzz.md
[ClusterFuzz]: https://github.com/google/clusterfuzz
### Fuzz Target
In addition to its general definition, in OSS-Fuzz a fuzz target can be used to
[reproduce bug reports](reproducing.md). It is recommended to use it for
regression testing as well (see [ideal integration](ideal_integration.md)).
### Job type
Or **Fuzzer Build**.
This refers to a build that contains all the [fuzz targets] for a given
[project](#project), is run with a specific [fuzzing engine], in a specific
build mode (e.g. with enabled/disabled assertions), and optionally combined
with a [sanitizer].
For example, we have a "libfuzzer_asan_sqlite" job type, indicating a build of
all sqlite3 [fuzz targets] using [libFuzzer](http://libfuzzer.info) and
[ASan](http://clang.llvm.org/docs/AddressSanitizer.html).
### Project
A project is an open source software project that is integrated with OSS-Fuzz.
Each project has a single set of configuration files
(example: [expat](https://github.com/google/oss-fuzz/tree/master/projects/expat))
and may have one or more [fuzz targets]
(example: [openssl](https://github.com/openssl/openssl/blob/master/fuzz/)).
### Reproducer
Or a **testcase**.
A [test input] that causes a specific bug to reproduce.
[fuzz targets]: https://github.com/google/fuzzing/blob/master/docs/glossary.md#fuzz-target
[fuzzing engine]: https://github.com/google/fuzzing/blob/master/docs/glossary.md#fuzzing-engine
[sanitizer]: https://github.com/google/fuzzing/blob/master/docs/glossary.md#sanitizer
[test input]: https://github.com/google/fuzzing/blob/master/docs/glossary.md#test-input

2
docs/index.md
View File

@ -29,7 +29,7 @@ distributed execution.
We support [libFuzzer](http://llvm.org/docs/LibFuzzer.html) and We support [libFuzzer](http://llvm.org/docs/LibFuzzer.html) and
[AFL](http://lcamtuf.coredump.cx/afl/) as fuzzing engines in combination with [AFL](http://lcamtuf.coredump.cx/afl/) as fuzzing engines in combination with
[Sanitizers](https://github.com/google/sanitizers). [Sanitizers](https://github.com/google/sanitizers).
[ClusterFuzz](docs/clusterfuzz.md) [ClusterFuzz]({{ site.baseurl }}/furthur-reading/clusterfuzz)
provides a distributed fuzzer execution environment and reporting. You can provides a distributed fuzzer execution environment and reporting. You can
checkout ClusterFuzz [here](https://github.com/google/clusterfuzz). checkout ClusterFuzz [here](https://github.com/google/clusterfuzz).

14
docs/installing_docker.md
View File

@ -1,14 +0,0 @@
# Installing Docker
Follow the [Official Guide](https://docs.docker.com/engine/installation).
Googlers: [go/installdocker](https://goto.google.com/installdocker).
[Why Docker?](faq.md#why-do-you-use-docker)
*NOTE: (Optional) If you want to run `docker` without `sudo`, follow the [Create a docker group](https://docs.docker.com/engine/installation/linux/ubuntulinux/#/create-a-docker-group) section.*
*NOTE: Docker images can consume significant disk space. Run*
*[docker-cleanup](https://gist.github.com/mikea/d23a839cba68778d94e0302e8a2c200f)*
*periodically to garbage collect unused images.*

33
docs/oss-fuzz/architecture.md Normal file
View File

@ -0,0 +1,33 @@
---
layout: default
title: Architecture
permalink: /architecture/
nav_order: 1
parent: OSS-Fuzz
---
# Architecture
![diagram]({{ site.baseurl }}/images/process.png?raw=true)
The following process is used for projects in OSS-Fuzz:
- A maintainer of an opensource project or an outside volunteer creates
one or more [fuzz targets](http://libfuzzer.info/#fuzz-target)
and [integrates]({{ site.baseurl }}/advanced-topics/ideal-integration/) them
with the project's build and test system.
- The project is [accepted to OSS-Fuzz]({{ site.baseurl }}/getting-started/accepting-new-projects/).
- When [ClusterFuzz]({{ site.baseurl }}/furthur-reading/clusterfuzz) finds a
bug, an issue is automatically reported in the OSS-Fuzz
[issue tracker](https://bugs.chromium.org/p/oss-fuzz/issues/list)
([example](https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=9)).
([Why use a different tracker?]({{ site.baseurl }}/faq/#why-do-you-use-a-different-issue-tracker-for-reporting-bugs-in-oss-projects)).
Project owners are CC-ed to the bug report.
- The project developer fixes the bug upstream and credits OSS-Fuzz for the
discovery (commit message should contain the string **'Credit to OSS-Fuzz'**).
- [ClusterFuzz]({{ site.baseurl }}/furthur-reading/clusterfuzz) automatically
verifies the fix, adds a comment and closes the issue
([example](https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=53#c3)).
- 30 days after the fix is verified or 90 days after reporting (whichever is
earlier), the issue becomes *public*
([guidelines]({{ site.baseurl }}/getting-started/bug-disclosure-guidelines/)).

30
docs/reference.md
View File

@ -1,30 +0,0 @@
# Reference
## Sanitizers
Fuzzers are usually built with one or more [sanitizer](https://github.com/google/sanitizers) enabled.
You can select sanitizer configuration by specifying `$SANITIZER` build environment variable using `-e` option:
```bash
python infra/helper.py build_fuzzers --sanitizer undefined json
```
Supported sanitizers:
| `$SANITIZER` | Description
| ------------ | ----------
| `address` *(default)* | [Address Sanitizer](https://github.com/google/sanitizers/wiki/AddressSanitizer) with [Leak Sanitizer](https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer).
| `undefined` | [Undefined Behavior Sanitizer](http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html).
| `memory` | [Memory Sanitizer](https://github.com/google/sanitizers/wiki/MemorySanitizer).<br/>*NOTE: It is critical that you build __all__ the code in your program (including libraries it uses) with Memory Sanitizer. Otherwise, you will see false positive crashes due to an inability to see initializations in uninstrumented code.*
| `profile` | Used for generating code coverage reports. See [Code Coverage doc](code_coverage.md).
Compiler flag values for predefined configurations are specified in the [Dockerfile](../infra/base-images/base-builder/Dockerfile).
These flags can be overridden by specifying `$SANITIZER_FLAGS` directly.
You can choose which configurations to automatically run your fuzzers with in `project.yaml` file (e.g. [sqlite3](../projects/sqlite3/project.yaml)):
```yaml
sanitizers:
- address
- undefined
```

96
docs/reference/glossary.md Normal file
View File

@ -0,0 +1,96 @@
---
layout: default
title: Glossary
nav_order: 1
permalink: /reference/glossary/
parent: Reference
---
# Glossary
For general fuzzing terms, see the [glossary] from [google/fuzzing] project,
[glossary]: https://github.com/google/fuzzing/blob/master/docs/glossary.md
[google/fuzzing]: https://github.com/google/fuzzing
- TOC
{:toc}
---
## OSS-Fuzz specific terms
### ClusterFuzz
A scalable fuzzing infrastructure that is used for OSS-Fuzz backend.
[ClusterFuzz] is also used to fuzz Chrome and many other projects. A quick
overview of ClusterFuzz user interface is available on this [page].
[page]: {{ site.baseurl }}/furthur-reading/clusterfuzz
[ClusterFuzz]: https://github.com/google/clusterfuzz
### Fuzz Target
In addition to its general definition, in OSS-Fuzz a fuzz target can be used to
[reproduce bug reports]({{ site.baseurl }}/advanced-topics/reproducing/). It is recommended to use it for
regression testing as well (see [ideal integration]({{ site.baseurl }}/advanced-topics/ideal-integration/)).
### Job type
Or **Fuzzer Build**.
This refers to a build that contains all the [fuzz targets] for a given
[project](#project), is run with a specific [fuzzing engine], in a specific
build mode (e.g. with enabled/disabled assertions), and optionally combined
with a [sanitizer].
For example, we have a "libfuzzer_asan_sqlite" job type, indicating a build of
all sqlite3 [fuzz targets] using [libFuzzer](http://libfuzzer.info) and
[ASan](http://clang.llvm.org/docs/AddressSanitizer.html).
### Project
A project is an open source software project that is integrated with OSS-Fuzz.
Each project has a single set of configuration files
(example: [expat](https://github.com/google/oss-fuzz/tree/master/projects/expat))
and may have one or more [fuzz targets]
(example: [openssl](https://github.com/openssl/openssl/blob/master/fuzz/)).
### Reproducer
Or a **testcase**.
A [test input] that causes a specific bug to reproduce.
[fuzz targets]: https://github.com/google/fuzzing/blob/master/docs/glossary.md#fuzz-target
[fuzzing engine]: https://github.com/google/fuzzing/blob/master/docs/glossary.md#fuzzing-engine
[sanitizer]: https://github.com/google/fuzzing/blob/master/docs/glossary.md#sanitizer
[test input]: https://github.com/google/fuzzing/blob/master/docs/glossary.md#test-input
### Sanitizers
Fuzzers are usually built with one or more [sanitizer](https://github.com/google/sanitizers) enabled.
You can select sanitizer configuration by specifying `$SANITIZER` build environment variable using `-e` option:
```bash
python infra/helper.py build_fuzzers --sanitizer undefined json
```
Supported sanitizers:
| `$SANITIZER` | Description
| ------------ | ----------
| `address` *(default)* | [Address Sanitizer](https://github.com/google/sanitizers/wiki/AddressSanitizer) with [Leak Sanitizer](https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer).
| `undefined` | [Undefined Behavior Sanitizer](http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html).
| `memory` | [Memory Sanitizer](https://github.com/google/sanitizers/wiki/MemorySanitizer).<br/>*NOTE: It is critical that you build __all__ the code in your program (including libraries it uses) with Memory Sanitizer. Otherwise, you will see false positive crashes due to an inability to see initializations in uninstrumented code.*
| `profile` | Used for generating code coverage reports. See [Code Coverage doc]({{ site.baseurl }}/advanced-topics/code-coverage/).
Compiler flag values for predefined configurations are specified in the [Dockerfile](https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-builder/Dockerfile).
These flags can be overridden by specifying `$SANITIZER_FLAGS` directly.
You can choose which configurations to automatically run your fuzzers with in `project.yaml` file (e.g. [sqlite3](https://github.com/google/oss-fuzz/tree/master/projects/sqlite3/project.yaml)):
```yaml
sanitizers:
- address
- undefined
```

9
docs/reference/reference.md Normal file
View File

@ -0,0 +1,9 @@
---
layout: default
title: Reference
has_children: true
nav_order: 6
permalink: /reference/
---
# Reference

44
docs/reference/useful_links.md Normal file
View File

@ -0,0 +1,44 @@
---
layout: default
title: Useful links
nav_order: 2
permalink: /reference/useful-links
parent: Reference
---
# Useful links
- TOC
{:toc}
---
## Web Interface
* The main page: [oss-fuzz.com](https://oss-fuzz.com)
## Build Status
* [This page](https://oss-fuzz-build-logs.storage.googleapis.com/index.html)
gives the latest build logs for each project.
* (Internal only) [Builds dashboard](https://builder.oss-fuzz.com).
## Blog posts
* 2016-12-01 ([1](https://opensource.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html),
[2](https://testing.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html),
[3](https://security.googleblog.com/2016/12/announcing-oss-fuzz-continuous-fuzzing.html))
* 2017-05-08 ([1](https://opensource.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html),
[2](https://testing.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html),
[3](https://security.googleblog.com/2017/05/oss-fuzz-five-months-later-and.html))
* 2018-11-06 ([1](https://security.googleblog.com/2018/11/a-new-chapter-for-oss-fuzz.html))
## Tutorials
* [libFuzzer documentation](http://libfuzzer.info)
* [libFuzzer tutorial](http://tutorial.libfuzzer.info)
* [libFuzzer workshop](https://github.com/Dor1s/libfuzzer-workshop)
* [Structure-Aware Fuzzing with libFuzzer](https://github.com/google/fuzzer-test-suite/blob/master/tutorial/structure-aware-fuzzing.md)
* [Chromium Fuzzing Page](https://chromium.googlesource.com/chromium/src/testing/libfuzzer/)
* [Chromium Efficient Fuzzing Guide](https://chromium.googlesource.com/chromium/src/testing/libfuzzer/+/HEAD/efficient_fuzzing.md)
* [ClusterFuzz documentation](https://google.github.io/clusterfuzz/)

95
docs/reproducing.md
View File

@ -1,95 +0,0 @@
# Reproducing OSS-Fuzz issues
You've been CC'ed on an OSS-Fuzz issue
([examples](https://bugs.chromium.org/p/oss-fuzz/issues/list?can=1&q=Type%3ABug%2CBug-Security)), now what?
Before attempting to fix the bug, you should be able to reliably reproduce it.
## Fuzz target bugs
Every issue has a [reproducer](glossary.md#reproducer) (aka "testcase") file attached.
Download it. If the issue is not public, you will need to login using your
[Google account](https://support.google.com/accounts/answer/176347?hl=en)
([why?](faq.md#why-do-you-require-a-google-account-for-authentication))
that the bug report CCs.
This file contains the bytes that were fed to the [Fuzz Target](http://libfuzzer.info/#fuzz-target).
If you have already [integrated](ideal_integration.md) the fuzz target with your build and test system,
all you do is run:
```bash
$ ./fuzz_target_binary <testcase_path>
```
If this is a timeout bug, add the <b><i>-timeout=25</i></b> argument.<br />
If this is an OOM bug, add the <b><i>-rss_limit_mb=2048</i></b> argument.<br />
Read more on how timeouts and OOMs are handed [here](faq.md#how-do-you-handle-timeouts-and-ooms).
Depending on the nature of the bug, the fuzz target binary needs to be built with the appropriate [sanitizer](https://github.com/google/sanitizers)
(e.g. if this is a buffer overflow, with [AddressSanitizer](http://clang.llvm.org/docs/AddressSanitizer.html)).
If you are not sure how to build the fuzzer using the project's build system,
you may also use Docker ([how?](installing_docker.md), [why?](faq.md#why-do-you-use-docker)) commands
to replicate the exact build steps used by OSS-Fuzz and then feed the reproducer input to the fuzz target.
## Building using Docker
### Pull the latest Docker images
```bash
$ python infra/helper.py pull_images
```
Docker images get regularly updated with a newer version of build tools, build
configurations, scripts, and other changes. In some cases, a particular issue
can be reproduced only with a fresh image being used.
### Build the image and the fuzzers
```bash
$ python infra/helper.py build_image $PROJECT_NAME
$ python infra/helper.py build_fuzzers --sanitizer <address/memory/undefined> $PROJECT_NAME
```
## Reproducing bugs
```bash
$ python infra/helper.py reproduce $PROJECT_NAME <fuzz_target_name> <testcase_path>
```
Find the type of sanitizer used in the report using the value in the **Sanitizer** column. It is one of the
following:
* **address** for AddressSanitizer
* **memory** for MemorySanitizer
* **undefined** for UndefinedBehaviorSanitizer
E.g. for building [libxml2](../projects/libxml2) project with UndefinedBehaviorSanitizer (undefined)
instrumentation and reproduce a crash testcase for a fuzzer named `libxml2_xml_read_memory_fuzzer`,
it will be:
```bash
$ python infra/helper.py build_image libxml2
$ python infra/helper.py build_fuzzers --sanitizer undefined libxml2
$ python infra/helper.py reproduce libxml2 libxml2_xml_read_memory_fuzzer ~/Downloads/testcase
```
## Reproduce using local source checkout
```bash
$ python infra/helper.py build_fuzzers --sanitizer <address/memory/undefined> $PROJECT_NAME <source_path>
$ python infra/helper.py reproduce $PROJECT_NAME <fuzz_target_name> <testcase_path>
```
This is essentially the previous command that additionally mounts local sources into the running container.
- *Fix issue*. Write a patch to fix the issue in your local checkout and then use the previous command to verify the fix (i.e. no crash occurred).
[Use gdb](debugging.md#debugging-fuzzers-with-gdb) if needed.
- *Submit fix*. Submit the fix in the project's repository. ClusterFuzz will automatically pick up the changes, recheck the testcase and will close the issue (in &lt; 1 day).
- *Improve fuzzing support*. Consider [improving fuzzing support](ideal_integration.md) in your project's build and test system.
## Reproducing build failures
Our infrastructure runs some sanity tests to make sure that your build was correctly configured, even if it succeeded. To reproduce these locally, run:
```bash
$ python infra/helper.py build_image $PROJECT_NAME
$ python infra/helper.py build_fuzzers --sanitizer <address/memory/undefined> --engine <libfuzzer/afl/honggfuzz> $PROJECT_NAME
$ python infra/helper.py check_build --sanitizer <address/memory/undefined> --engine <libfuzzer/afl/honggfuzz> $PROJECT_NAME <fuzz_target_name>
```
For reproducing a `coverage` build failure, follow [Code Coverage page](code_coverage.md) to build your project and generate a code coverage report.

2
infra/README.md
View File

@ -23,6 +23,6 @@ Continuous Integration infrastructure:
| `build_image` | Builds a docker image for a given project | | `build_image` | Builds a docker image for a given project |
| `build_fuzzers` | Builds fuzz targets for a given project | | `build_fuzzers` | Builds fuzz targets for a given project |
| `run_fuzzer` | Runs a fuzz target in a docker container | | `run_fuzzer` | Runs a fuzz target in a docker container |
| `coverage` | Runs fuzz target(s) in a docker container and generates a code coverage report. See [Code Coverage doc](../docs/code_coverage.md) | | `coverage` | Runs fuzz target(s) in a docker container and generates a code coverage report. See [Code Coverage doc](https://google.github.io/oss-fuzz/advanced-topics/code-coverage/) |
| `reproduce` | Runs a testcase to reproduce a crash | | `reproduce` | Runs a testcase to reproduce a crash |
| `shell` | Starts a shell inside the docker image for a project | | `shell` | Starts a shell inside the docker image for a project |

6
projects/example/my-api-repo/README.md
View File

@ -1,6 +1,6 @@
Example of [OSS-Fuzz ideal integration](../../../docs/ideal_integration.md). Example of [OSS-Fuzz ideal integration](https://google.github.io/oss-fuzz/advanced-topics/ideal-integration/).
This directory contains an example software project that has most of the traits of [ideal](../../../docs/ideal_integration.md) support for fuzzing. This directory contains an example software project that has most of the traits of [ideal](https://google.github.io/oss-fuzz/advanced-topics/ideal-integration/) support for fuzzing.
## Files in my-api-repo ## Files in my-api-repo
Imagine that these files reside in your project's repository: Imagine that these files reside in your project's repository:
@ -9,7 +9,7 @@ Imagine that these files reside in your project's repository:
* [do_stuff_unittest.cpp](do_stuff_unittest.cpp): is a unit test for `DoStuff()`. Unit tests are not necessary for fuzzing but are generally a good practice. * [do_stuff_unittest.cpp](do_stuff_unittest.cpp): is a unit test for `DoStuff()`. Unit tests are not necessary for fuzzing but are generally a good practice.
* [do_stuff_fuzzer.cpp](do_stuff_fuzzer.cpp): is a [fuzz target](http://libfuzzer.info/#fuzz-target) for `DoStuff()`. * [do_stuff_fuzzer.cpp](do_stuff_fuzzer.cpp): is a [fuzz target](http://libfuzzer.info/#fuzz-target) for `DoStuff()`.
* [do_stuff_test_data](do_stuff_test_data): corpus directory for [do_stuff_fuzzer.cpp](do_stuff_fuzzer.cpp). * [do_stuff_test_data](do_stuff_test_data): corpus directory for [do_stuff_fuzzer.cpp](do_stuff_fuzzer.cpp).
* [do_stuff_fuzzer.dict](do_stuff_fuzzer.dict): a [fuzzing dictionary file](https://github.com/google/oss-fuzz/blob/master/docs/new_project_guide.md#dictionaries) for `DoStuff()`. Optional, but may improve fuzzing in many cases. * [do_stuff_fuzzer.dict](do_stuff_fuzzer.dict): a [fuzzing dictionary file](https://google.github.io/oss-fuzz/getting-started/new-project-guide#dictionaries) for `DoStuff()`. Optional, but may improve fuzzing in many cases.
* [Makefile](Makefile): is a build file (the same can be done with other build systems): * [Makefile](Makefile): is a build file (the same can be done with other build systems):
* accepts external compiler flags via `$CC`, `$CXX`, `$CFLAGS`, `$CXXFLAGS` * accepts external compiler flags via `$CC`, `$CXX`, `$CFLAGS`, `$CXXFLAGS`
* accepts external fuzzing engine via `$LIB_FUZZING_ENGINE`, by default uses [standalone_fuzz_target_runner.cpp](standalone_fuzz_target_runner.cpp) * accepts external fuzzing engine via `$LIB_FUZZING_ENGINE`, by default uses [standalone_fuzz_target_runner.cpp](standalone_fuzz_target_runner.cpp)