oss-fuzz/docs/advanced-topics/code_coverage.md

127 lines
3.5 KiB
Markdown
Raw Normal View History

2019-08-07 14:37:16 +00:00
---
layout: default
title: Code coverage
parent: Advanced topics
nav_order: 2
permalink: /advanced-topics/code-coverage
---
# Code Coverage
You can generate code coverage report for your project using [Clang Source-based
Code Coverage].
2019-08-07 14:37:16 +00:00
- TOC
{:toc}
---
## Pull the latest Docker images
Docker images get regularly updated with a newer version of build tools, build
configurations, scripts, and other changes. It is recommended to use the most
2019-08-15 22:07:23 +00:00
recent images.
```bash
2019-08-15 22:07:23 +00:00
$ python infra/helper.py pull_images
```
## Build fuzz targets
Code Coverage report generation requires a special build configuration to be
2019-08-15 22:07:23 +00:00
used. To create a code coverage build for your project, run:
```bash
2019-08-15 22:07:23 +00:00
$ python infra/helper.py build_image $PROJECT_NAME
$ python infra/helper.py build_fuzzers --sanitizer=coverage $PROJECT_NAME
```
## Establish access to GCS
2019-08-15 22:07:23 +00:00
To get a good understanding of quality of fuzz testing established for your
project, code coverage reports should be generated by running fuzz targets
against the corpus aggregated by OSS-Fuzz. Set up `gsutil` and ensure that
you have access to the corpora using:
* Install [gsutil tool]
* Check whether you have access to the corpus for your project:
```bash
2019-08-15 22:07:23 +00:00
$ gsutil ls gs://${PROJECT_NAME}-corpus.clusterfuzz-external.appspot.com/
```
If you see an authorization error from the command above, run:
```bash
2019-08-15 22:07:23 +00:00
$ gcloud auth login
```
and try again. Once `gsutil` works, you can run the report generation.
2019-08-15 22:07:23 +00:00
## Generate code coverage reports
### Full project report
To generate code coverage report using the corpus aggregated on OSS-Fuzz, run:
```bash
2019-08-15 22:07:23 +00:00
$ python infra/helper.py coverage $PROJECT_NAME
```
If you want to generate code coverage report using the corpus you have locally,
2019-08-15 22:07:23 +00:00
copy the corpus into `build/corpus/$PROJECT_NAME/<fuzz_target_name>/` directories for
each fuzz target, then run:
```bash
2019-08-15 22:07:23 +00:00
$ python infra/helper.py coverage --no-corpus-download $PROJECT_NAME
```
### Single fuzz target
You can generate a code coverage report for a particular fuzz target with
`--fuzz-target` argument:
```bash
2019-08-15 22:07:23 +00:00
$ python infra/helper.py coverage --fuzz-target=<fuzz_target_name> $PROJECT_NAME
```
In this mode, you can specify an arbitrary corpus location for the fuzz target
via `--corpus-dir` to be used instead of the corpus downloaded from OSS-Fuzz:
```bash
2019-08-15 22:07:23 +00:00
$ python infra/helper.py coverage --fuzz-target=<fuzz_target_name> \
--corpus-dir=<my_local_corpus_dir> $PROJECT_NAME
```
### Additional arguments for `llvm-cov`
You may want to use some of the options of [llvm-cov tool], for example,
`-ignore-filename-regex=`. You can pass those to the helper script after `--`:
```bash
2019-08-15 22:07:23 +00:00
$ python infra/helper.py coverage $PROJECT_NAME -- \
-ignore-filename-regex=.*code/to/be/ignored/.* <other_extra_args>
```
To specify particular source files or directories to show in the report, list
their paths at the end of the extra arguments sequence, for example:
```bash
2019-08-15 22:07:23 +00:00
$ python infra/helper.py coverage zlib -- \
<other_extra_args> /src/zlib/inftrees.c /src/zlib_uncompress_fuzzer.cc /src/zlib/zutil.c
```
If you want OSS-Fuzz to use some extra arguments when generating code coverage
reports for your project, add the arguments into `project.yaml` file as follows:
```yaml
coverage_extra_args: -ignore-filename-regex=.*crc.* -ignore-filename-regex=.*adler.* <other_extra_args>
```
[Clang Source-based Code Coverage]: https://clang.llvm.org/docs/SourceBasedCodeCoverage.html
[gsutil tool]: https://cloud.google.com/storage/docs/gsutil_install
[llvm-cov tool]: https://llvm.org/docs/CommandGuide/llvm-cov.html