mirror of https://github.com/google/oss-fuzz.git
Add initial documentation for adding a new library.
This commit is contained in:
Oliver Chang
parent
6d46d6d0e7
commit
44f33a3e5a
|
|
@ -0,0 +1,189 @@
|
|||
# Setting up fuzzers for a new library
|
||||
|
||||
Fuzzer build configurations are placed into a top-level directory for the
|
||||
library in the [oss-fuzz repo] on GitHub. For example, fuzzers for the expat
|
||||
library go into <https://github.com/google/oss-fuzz/tree/master/expat>.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Install Docker.
|
||||
|
||||
If you're not familiar with how building libFuzzer-style fuzzers work in
|
||||
general, check out [this page](http://llvm.org/docs/LibFuzzer.html).
|
||||
|
||||
## Library directories
|
||||
|
||||
At a minimum, a library directory requires 3 files:
|
||||
|
||||
* Jenkinsfile
|
||||
* Dockerfile
|
||||
* build.sh
|
||||
|
||||
### Jenkinsfile
|
||||
|
||||
This file will be largely the same for most libraries. For expat, this is:
|
||||
|
||||
```
|
||||
def libfuzzerBuild = fileLoader.fromGit('infra/libfuzzer-pipeline.groovy',
|
||||
'https://github.com/google/oss-fuzz.git', 'master', null, '')
|
||||
|
||||
libfuzzerBuild {
|
||||
git = "git://git.code.sf.net/p/expat/code_git"
|
||||
dockerfile = "oss-fuzz/expat/Dockerfile"
|
||||
}
|
||||
```
|
||||
|
||||
Simply replace the the "git" entry with the correct git url for the library
|
||||
(only git is supported right now).
|
||||
|
||||
### Dockerfile
|
||||
|
||||
This is the definition for the Docker container that fuzzers will be built in.
|
||||
This should be very similar for most libraries as well. This file requires a
|
||||
couple of lines:
|
||||
|
||||
* `FROM ossfuzz/base-libfuzzer` to inherit settings from the base container.
|
||||
Containers are based on Ubuntu 16.04.
|
||||
* `MAINTAINER YOUR_EMAIL`
|
||||
* `RUN ....` to run custom commands. For example, if your library requires
|
||||
additional build dependencies, you should include `apt-get` commands here to
|
||||
install them.
|
||||
* `CMD /src/oss-fuzz/LIB_NAME/build.sh` to run the custom build script for your
|
||||
library (see next section).
|
||||
|
||||
expat example:
|
||||
|
||||
```
|
||||
FROM ossfuzz/base-libfuzzer
|
||||
MAINTAINER mike.aizatsky@gmail.com
|
||||
RUN apt-get install -y make autoconf automake libtool docbook2x
|
||||
|
||||
CMD /src/oss-fuzz/expat/build.sh
|
||||
```
|
||||
|
||||
### build.sh
|
||||
|
||||
This is where most of the work is done to build fuzzers for your library.
|
||||
|
||||
When this script is run, the source code for your library will be at
|
||||
`/src/LIB_NAME` (replace `LIB_NAME` with the actual name). The `oss-fuzz`
|
||||
repository will similarly be checked out to `/src/oss-fuzz`.
|
||||
|
||||
`/work/libfuzzer` contains the libFuzzer object files that need to be linked
|
||||
into all fuzzers.
|
||||
|
||||
Some useful environment variables are also set:
|
||||
|
||||
* `CC`: The C compiler.
|
||||
* `CXX` or `CCC`: The C++ compiler.
|
||||
* `CFLAGS`: Predefined C flags. This should always be passed when building
|
||||
fuzzers.
|
||||
* `CXXFLAGS`: Predefined C++ flags. This should always be passed when building
|
||||
fuzzers.
|
||||
* `LDFLAGS`: Linker flags. This should always be passed when building
|
||||
fuzzer.
|
||||
|
||||
In general, this script will need to:
|
||||
|
||||
1. Build the library using whatever build system the library is using.
|
||||
2. Build the fuzzers, linking with the library and libFuzzer. Built fuzzers
|
||||
should be placed in `/out`.
|
||||
|
||||
For expat, this looks like:
|
||||
|
||||
```bash
|
||||
#!/bin/bash -eu
|
||||
|
||||
# cd into the expat directory.
|
||||
cd /src/expat/expat
|
||||
|
||||
# build the library.
|
||||
./buildconf.sh
|
||||
./configure
|
||||
make clean all
|
||||
|
||||
# build the fuzzer, linking with libFuzzer and libexpat.a
|
||||
$CXX $CXXFLAGS -std=c++11 -Ilib/ \
|
||||
/src/oss-fuzz/expat/parse_fuzzer.cc -o /out/expat_parse_fuzzer \
|
||||
/work/libfuzzer/*.o .libs/libexpat.a $LDFLAGS
|
||||
```
|
||||
|
||||
Breaking this down:
|
||||
|
||||
1. `cd /src/expat/expat` just changes the working directory to the expat dir.
|
||||
2. `buildconfig.sh ... make clean all` calls the build system for the library
|
||||
(we need `libexpat.a`).
|
||||
3. Build the fuzzer. In this case, we're building a fuzzer that's stored in the
|
||||
oss-fuzz repo (`/src/oss-fuzz/expat/parse_fuzzer.cc`), writing it to
|
||||
`/out/expat_parse_fuzzer`, and linking in `/work/libfuzzer/*.o` (prebuilt)
|
||||
and `.libs/libexpat.a` (which we just built).
|
||||
|
||||
|
||||
### Others (e.g. fuzzer source)
|
||||
|
||||
For some libraries, the upstream repository will contain fuzzers (e.g.
|
||||
freetype2). In other cases, such as expat, we can check in fuzzer source into
|
||||
the oss-fuzz repo.
|
||||
|
||||
## Testing locally
|
||||
|
||||
```bash
|
||||
$ mkdir ~/src; cd ~/src
|
||||
$ git clone https://github.com/google/oss-fuzz
|
||||
# (also checkout upstream library)
|
||||
|
||||
$ cd oss-fuzz
|
||||
# (patch in your changes to oss-fuzz )
|
||||
|
||||
$ docker build -t ossfuzz/LIB_NAME LIB_NAME
|
||||
$ docker run -i -v ~/src/oss-fuzz:/src/oss-fuzz -v ~/src/LIB_NAME:/src/LIB_NAME -v ~/tmp/out:/out -t ossfuzz/LIB_NAME
|
||||
```
|
||||
|
||||
This should place the built fuzzers into `~/tmp/out` on your machine (`/out` in
|
||||
the container). You can then try to run these fuzzers inside the container to
|
||||
make sure that they work properly:
|
||||
|
||||
```bash
|
||||
$ docker run -i -v ~/tmp/out:/out -t ossfuzz/LIB_NAME /bin/bash
|
||||
# (you should now in bash inside the container)
|
||||
$ /out/FUZZER_NAME
|
||||
```
|
||||
|
||||
If everything works locally, then it should also work on our automated builders
|
||||
and ClusterFuzz.
|
||||
|
||||
## Checking in to oss-fuzz repository
|
||||
|
||||
Fork oss-fuzz, commit and push to the fork, and then create a pull request with
|
||||
your change!
|
||||
|
||||
### Copyright headers
|
||||
|
||||
Please include copyright headers for all files checked in to oss-fuzz:
|
||||
|
||||
```
|
||||
# Copyright 2016 Google Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#
|
||||
################################################################################
|
||||
```
|
||||
|
||||
If porting a fuzzer from Chromium, keep the Chromium license header.
|
||||
|
||||
## The end
|
||||
|
||||
Once your change is merged, the fuzzers should be automatically built and run on
|
||||
ClusterFuzz after a short while!
|
||||
|
||||
[oss-fuzz repo]: https://github.com/google/oss-fuzz
|
||||
Loading…
Reference in New Issue