mirror of https://github.com/google/oss-fuzz.git
Fixes (#236)
* Add reproduce command. * Remove unneeded run and just_run command with duplicate code as run_fuzzer. * Fix docs for reproduction.
This commit is contained in:
Abhishek Arya
GitHub
parent
3c359b58a0
commit
6c4110d4a4
|
|
@ -14,8 +14,13 @@ This file contains the bytes that were fed to the [Fuzz Target](http://libfuzzer
|
|||
If you have already [integrated](ideal_integration.md) the fuzz target with your build and test system,
|
||||
all you do is run:
|
||||
<pre>
|
||||
./fuzz_target_binary <b><i>$testcase_file_absolute_path</i></b>
|
||||
./fuzz_target_binary <testcase_path>
|
||||
</pre>
|
||||
|
||||
If this is a timeout bug, add the <b><i>-timeout=25</i></b> argument.
|
||||
If this is a OOM bug, add the <b><i>-rss_limit_mb=2048</i></b> argument.
|
||||
Read more on how timeouts and OOMs are handed [here](faq.md#how-do-you-handle-timeouts-and-ooms).
|
||||
|
||||
Depending on the nature of the bug, the fuzz target binary needs to be built with the appropriate [sanitizer](https://github.com/google/sanitizers)
|
||||
(e.g. if this is a buffer overflow, with [AddressSanitizer](http://clang.llvm.org/docs/AddressSanitizer.html)).
|
||||
|
||||
|
|
@ -26,7 +31,7 @@ to replicate the exact build steps used by OSS-Fuzz and then feed the reproducer
|
|||
- *Reproduce using latest OSS-Fuzz build:*
|
||||
|
||||
<pre>
|
||||
docker run --rm -ti -v <b><i>$testcase_file_absolute_path</i></b>:/testcase ossfuzz/<b><i>$project</i></b> reproduce <b><i>$fuzzer</i></b>
|
||||
python infra/helper.py reproduce $PROJECT_NAME <fuzzer_name> <testcase_path>
|
||||
</pre>
|
||||
|
||||
It builds the fuzzer from the most recent successful OSS-Fuzz build (usually last night's sources)
|
||||
|
|
@ -35,13 +40,13 @@ docker run --rm -ti -v <b><i>$testcase_file_absolute_path</i></b>:/testcase ossf
|
|||
E.g. for [libxml2](../projects/libxml2) project with fuzzer named `libxml2_xml_read_memory_fuzzer`, it will be:
|
||||
|
||||
<pre>
|
||||
docker run --rm -ti -v <b><i>~/Downloads/testcase</i></b>:/testcase ossfuzz/<b><i>libxml2</i></b> reproduce <b><i>libxml2_xml_read_memory_fuzzer</i></b>
|
||||
python infra/helper.py reproduce libxml2 libxml2_xml_read_memory_fuzzer ~/Downloads/testcase
|
||||
</pre>
|
||||
- *Reproduce using local source checkout:*
|
||||
|
||||
<pre>
|
||||
docker run --rm -ti -v <b><i>$local_source_checkout_dir</i></b>:/src/<b><i>$project</i></b> \
|
||||
-v <b><i>$testcase_file_absolute_path</i></b>:/testcase ossfuzz/<b><i>$project</i></b> reproduce <b><i>$fuzzer</i></b>
|
||||
python infra/helper.py build_fuzzers $PROJECT_NAME <source_path>
|
||||
python infra/helper.py reproduce $PROJECT_NAME <fuzzer_name> <testcase_path>
|
||||
</pre>
|
||||
|
||||
This is essentially the previous command that additionally mounts local sources into the running container.
|
||||
|
|
|
|||
|
|
@ -36,8 +36,7 @@ ENV LIB_FUZZING_ENGINE="/usr/lib/libFuzzingEngine.a"
|
|||
# TODO: remove after tpm2 catchup.
|
||||
ENV FUZZER_LDFLAGS ""
|
||||
|
||||
COPY coverage_report compile compile_libfuzzer srcmap reproduce run just_run \
|
||||
/usr/local/bin/
|
||||
COPY compile compile_libfuzzer coverage_report srcmap /usr/local/bin/
|
||||
|
||||
WORKDIR $SRC
|
||||
CMD ["compile"]
|
||||
|
|
@ -46,5 +45,4 @@ ADD http://lcamtuf.coredump.cx/afl/releases/afl-latest.tgz $SRC/
|
|||
RUN mkdir afl && \
|
||||
cd afl && \
|
||||
tar -xzv --strip-components=1 -f $SRC/afl-latest.tgz && \
|
||||
rm -rf $SRC/afl-latest.tgz && \
|
||||
ls -lR /src
|
||||
rm -rf $SRC/afl-latest.tgz
|
||||
|
|
|
|||
|
|
@ -1,42 +0,0 @@
|
|||
#!/bin/bash -eu
|
||||
# Copyright 2016 Google Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#
|
||||
################################################################################
|
||||
|
||||
# Fuzzer runner. Appends .options arguments and seed corpus to users args.
|
||||
# Usage: $0 <fuzzer_name> <fuzzer_args>
|
||||
|
||||
cd $OUT
|
||||
export PATH="$OUT:$PATH"
|
||||
|
||||
FUZZER=$1
|
||||
shift
|
||||
CMD_LINE="$FUZZER $@"
|
||||
|
||||
OPTIONS_FILE="${FUZZER}.options"
|
||||
if [ -f $OPTIONS_FILE ]; then
|
||||
OPTIONS_ARGS=$(grep "=" $OPTIONS_FILE | sed 's/\(\w*\)\W*=\W*\(.*\)/-\1=\2 /g' | tr '\n' ' ')
|
||||
CMD_LINE="$CMD_LINE $OPTIONS_ARGS"
|
||||
fi
|
||||
|
||||
SEED_CORPUS="${FUZZER}_seed_corpus.zip"
|
||||
if [ -f $SEED_CORPUS ]; then
|
||||
rm -rf /tmp/seed_corpus/ && mkdir /tmp/seed_corpus/
|
||||
unzip -d /tmp/seed_corpus/ $SEED_CORPUS
|
||||
CMD_LINE="$CMD_LINE /tmp/seed_corpus/"
|
||||
fi
|
||||
|
||||
echo $CMD_LINE
|
||||
bash -c "$CMD_LINE"
|
||||
|
|
@ -1,19 +0,0 @@
|
|||
#!/bin/bash -eux
|
||||
# Copyright 2016 Google Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#
|
||||
################################################################################
|
||||
|
||||
compile
|
||||
just_run $@
|
||||
|
|
@ -17,7 +17,8 @@
|
|||
FROM ossfuzz/base-image
|
||||
MAINTAINER mike.aizatsky@gmail.com
|
||||
RUN apt-get install -y zip file
|
||||
COPY llvm-symbolizer test_all test_report run_fuzzer /usr/local/bin/
|
||||
COPY llvm-symbolizer reproduce run_fuzzer test_all test_report \
|
||||
/usr/local/bin/
|
||||
|
||||
# Default environment options for various sanitizers.
|
||||
# Note that these match the settings used in ClusterFuzz and
|
||||
|
|
@ -26,4 +27,4 @@ COPY llvm-symbolizer test_all test_report run_fuzzer /usr/local/bin/
|
|||
ENV ASAN_OPTIONS="alloc_dealloc_mismatch=0:allocator_may_return_null=1:allocator_release_to_os=1:check_malloc_usable_size=0:detect_container_overflow=1:detect_odr_violation=0:detect_leaks=1:detect_stack_use_after_return=1:fast_unwind_on_fatal=0:handle_abort=1:handle_segv=1:handle_sigill=1:max_uar_stack_size_log=16:print_scariness=1:quarantine_size_mb=10:strict_memcmp=1:strict_string_check=1:strip_path_prefix=/workspace/:symbolize=1:use_sigaltstack=1"
|
||||
ENV MSAN_OPTIONS="print_stats=1:strip_path_prefix=/workspace/:symbolize=1"
|
||||
ENV UBSAN_OPTIONS="halt_on_error=1:print_stacktrace=1:print_summary=1:strip_path_prefix=/workspace/:symbolize=1"
|
||||
|
||||
ENV FUZZER_ARGS="-rss_limit_mb=2048 -timeout=25"
|
||||
|
|
|
|||
|
|
@ -24,10 +24,8 @@ if [ ! -f $TESTCASE ]; then
|
|||
exit 1
|
||||
fi
|
||||
|
||||
|
||||
compile
|
||||
export PATH=/out:$PATH
|
||||
cd /out
|
||||
|
||||
$FUZZER $@ $TESTCASE
|
||||
$FUZZER $FUZZER_ARGS $@ $TESTCASE
|
||||
|
||||
|
|
@ -22,7 +22,7 @@ cd $OUT
|
|||
|
||||
FUZZER=$1
|
||||
shift
|
||||
CMD_LINE="$FUZZER $@"
|
||||
CMD_LINE="$FUZZER $FUZZER_ARGS $@"
|
||||
|
||||
OPTIONS_FILE="${FUZZER}.options"
|
||||
if [ -f $OPTIONS_FILE ]; then
|
||||
|
|
|
|||
|
|
@ -32,7 +32,6 @@ import time
|
|||
OSSFUZZ_DIR = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
|
||||
BUILD_DIR = os.path.join(OSSFUZZ_DIR, 'build')
|
||||
|
||||
|
||||
GLOBAL_ARGS = None
|
||||
|
||||
def main():
|
||||
|
|
@ -46,7 +45,7 @@ def main():
|
|||
help='do not specify --pull while building an image')
|
||||
parser.add_argument(
|
||||
'command',
|
||||
help='One of: generate, build_image, build_fuzzers, run_fuzzer, coverage, shell',
|
||||
help='One of: generate, build_image, build_fuzzers, run_fuzzer, coverage, reproduce, shell',
|
||||
nargs=argparse.REMAINDER)
|
||||
global GLOBAL_ARGS
|
||||
GLOBAL_ARGS = args = parser.parse_args()
|
||||
|
|
@ -65,6 +64,8 @@ def main():
|
|||
return run_fuzzer(args.command[1:])
|
||||
elif args.command[0] == 'coverage':
|
||||
return coverage(args.command[1:])
|
||||
elif args.command[0] == 'reproduce':
|
||||
return reproduce(args.command[1:])
|
||||
elif args.command[0] == 'shell':
|
||||
return shell(args.command[1:])
|
||||
else:
|
||||
|
|
@ -100,6 +101,11 @@ def _check_fuzzer_exists(project_name, fuzzer_name):
|
|||
return True
|
||||
|
||||
|
||||
def _get_absolute_path(path):
|
||||
"""Returns absolute path with user expansion."""
|
||||
return os.path.abspath(os.path.expanduser(path))
|
||||
|
||||
|
||||
def _get_command_string(command):
|
||||
"""Returns a shell escaped command string."""
|
||||
return ' '.join(pipes.quote(part) for part in command)
|
||||
|
|
@ -120,7 +126,7 @@ def _build_image(image_name):
|
|||
build_args = []
|
||||
if not GLOBAL_ARGS.nopull:
|
||||
build_args += ['--pull']
|
||||
build_args += ['-t', 'ossfuzz/' + image_name, dockerfile_dir ]
|
||||
build_args += ['-t', 'ossfuzz/%s' % image_name, dockerfile_dir ]
|
||||
|
||||
command = [ 'docker', 'build' ] + build_args
|
||||
print('Running:', _get_command_string(command))
|
||||
|
|
@ -151,6 +157,8 @@ def build_fuzzers(build_args):
|
|||
parser = argparse.ArgumentParser('helper.py build_fuzzers')
|
||||
parser.add_argument('-e', action='append', help="set environment variable")
|
||||
parser.add_argument('project_name')
|
||||
parser.add_argument('source_path', help='path of local source',
|
||||
nargs='?')
|
||||
args = parser.parse_args(build_args)
|
||||
project_name = args.project_name
|
||||
|
||||
|
|
@ -161,12 +169,20 @@ def build_fuzzers(build_args):
|
|||
if args.e:
|
||||
env += args.e
|
||||
|
||||
command = (['docker', 'run', '--rm', '-i', '--cap-add', 'SYS_PTRACE'] +
|
||||
sum([['-e', v] for v in env], []) +
|
||||
['-v', '%s:/out' % os.path.join(BUILD_DIR, 'out', project_name),
|
||||
'-v', '%s:/work' % os.path.join(BUILD_DIR, 'work', project_name),
|
||||
'-t', 'ossfuzz/' + project_name
|
||||
])
|
||||
command = (
|
||||
['docker', 'run', '--rm', '-i', '--cap-add', 'SYS_PTRACE'] +
|
||||
sum([['-e', v] for v in env], [])
|
||||
)
|
||||
if args.source_path:
|
||||
command += [
|
||||
'-v',
|
||||
'%s:/src/%s' % (_get_absolute_path(args.source_path), args.project_name)
|
||||
]
|
||||
command += [
|
||||
'-v', '%s:/out' % os.path.join(BUILD_DIR, 'out', project_name),
|
||||
'-v', '%s:/work' % os.path.join(BUILD_DIR, 'work', project_name),
|
||||
'-t', 'ossfuzz/%s' % project_name
|
||||
]
|
||||
|
||||
print('Running:', _get_command_string(command))
|
||||
|
||||
|
|
@ -209,6 +225,7 @@ def run_fuzzer(run_args):
|
|||
pipe = subprocess.Popen(command)
|
||||
pipe.communicate()
|
||||
|
||||
|
||||
def coverage(run_args):
|
||||
"""Runs a fuzzer in the container."""
|
||||
parser = argparse.ArgumentParser('helper.py coverage')
|
||||
|
|
@ -236,7 +253,7 @@ def coverage(run_args):
|
|||
'-v', '%s:/out' % os.path.join(BUILD_DIR, 'out', args.project_name),
|
||||
'-v', '%s:/cov' % temp_dir,
|
||||
'-w', '/cov',
|
||||
'-e', 'ASAN_OPTIONS=coverage=1,detect_leaks=0',
|
||||
'-e', 'ASAN_OPTIONS=coverage=1',
|
||||
'-t', 'ossfuzz/base-runner',
|
||||
'/out/%s' % args.fuzzer_name,
|
||||
'-max_total_time=%s' % args.run_time
|
||||
|
|
@ -264,6 +281,38 @@ def coverage(run_args):
|
|||
pipe.communicate()
|
||||
|
||||
|
||||
def reproduce(run_args):
|
||||
"""Reproduces a testcase in the container."""
|
||||
parser = argparse.ArgumentParser('helper.py reproduce')
|
||||
parser.add_argument('project_name', help='name of the project')
|
||||
parser.add_argument('fuzzer_name', help='name of the fuzzer')
|
||||
parser.add_argument('testcase_path', help='path of local testcase')
|
||||
|
||||
args = parser.parse_args(run_args)
|
||||
|
||||
if not _check_project_exists(args.project_name):
|
||||
return 1
|
||||
|
||||
if not _check_fuzzer_exists(args.project_name, args.fuzzer_name):
|
||||
return 1
|
||||
|
||||
if not _build_image('base-runner'):
|
||||
return 1
|
||||
|
||||
command = [
|
||||
'docker', 'run', '--rm', '-i', '--cap-add', 'SYS_PTRACE',
|
||||
'-v', '%s:/out' % os.path.join(BUILD_DIR, 'out', args.project_name),
|
||||
'-v', '%s:/testcase' % _get_absolute_path(args.testcase_path),
|
||||
'-t', 'ossfuzz/base-runner',
|
||||
'reproduce',
|
||||
'/out/%s' % args.fuzzer_name,
|
||||
]
|
||||
|
||||
print('Running:', _get_command_string(command))
|
||||
pipe = subprocess.Popen(command)
|
||||
pipe.communicate()
|
||||
|
||||
|
||||
def generate(generate_args):
|
||||
"""Generate empty project files."""
|
||||
parser = argparse.ArgumentParser('helper.py generate')
|
||||
|
|
@ -311,7 +360,7 @@ def shell(shell_args):
|
|||
'docker', 'run', '--rm', '-i', '--cap-add', 'SYS_PTRACE',
|
||||
'-v', '%s:/out' % os.path.join(BUILD_DIR, 'out', args.project_name),
|
||||
'-v', '%s:/work' % os.path.join(BUILD_DIR, 'work', args.project_name),
|
||||
'-t', 'ossfuzz/' + args.project_name,
|
||||
'-t', 'ossfuzz/%s' % args.project_name,
|
||||
'/bin/bash'
|
||||
]
|
||||
print('Running:', _get_command_string(command))
|
||||
|
|
|
|||
Loading…
Reference in New Issue