Commit Graph

245 Commits

Author SHA1 Message Date
Mike Aizatsky 11b0c45f91 [infra] added memory sanitizer config 2016-12-13 12:47:20 -08:00
Abhishek Arya 4f3b0123f6 Fix merge conflict. 2016-12-13 11:10:15 -08:00
Abhishek Arya 10d8f2f7b2 Use same *SAN_OPTIONS as ClusterFuzz. 2016-12-13 11:07:48 -08:00
Mike Aizatsky 7bd5ae5115 [infra] coverage flags can be overriden on target basis. #84 2016-12-13 10:37:21 -08:00
Mike Aizatsky d03d1679f9 debugging on 2016-12-12 22:44:37 -08:00
Mike Aizatsky e30377104f debug output 2016-12-12 22:09:22 -08:00
Mike Aizatsky 4d0d878609 [infra] failure should be indicated by a separate tag. #168 2016-12-12 21:40:28 -08:00
Mike Aizatsky 7f162d0fed [infra] pipefail is no longer needed 2016-12-12 16:33:03 -08:00
Mike Aizatsky 598c8ba618 [infra] separate test_report script for jenkins integration
Separate test_report script runs tests and generates junit-style
reports. The script doesn't fail when fuzzer fails, but marks it as
failed instead.

Fixes #168
2016-12-12 16:30:51 -08:00
Oliver Chang f2e3c0e50d Include file in base-runner. (#160) 2016-12-09 12:32:40 -08:00
Oliver Chang 6dde379193 Fix fuzzer executable detection in test_all. (#156) 2016-12-08 12:41:14 -08:00
Mike Aizatsky 0e7acedab3 Update test_all 2016-12-07 14:45:25 -08:00
Mike Aizatsky 764a8b99a7 [infra] removed libfuzzer.a. Fixes #139 2016-12-07 13:46:19 -08:00
Mike Aizatsky e92e678188 [infra] specifying testcase for junit report 2016-12-07 12:19:06 -08:00
Mike Aizatsky 8e68455932 Update Jenkinsfile 2016-12-07 12:07:45 -08:00
Mike Aizatsky 3ab381b653 Update compile_libfuzzer 2016-12-07 11:49:34 -08:00
Mike Aizatsky 3dd707ed58 [infra] using -lFuzzingEngine instead of -lfuzzer 2016-12-07 11:41:08 -08:00
Mike Aizatsky ddaa976438 removed trailing whitespaces 2016-12-07 11:35:53 -08:00
Mike Aizatsky 2d7e3adcaf [infra] env var rename 2016-12-06 22:21:06 -08:00
Mike Aizatsky feff4b8a66 [infra] FUZZING_ENGINE to choose compile script (#128) 2016-12-06 22:18:54 -08:00
Mike Aizatsky 8156f1d7a1 [infra] (nit) optimizing away couple layers 2016-12-06 21:53:28 -08:00
Mike Aizatsky bddc4d7fee [infra] using all.sh instead of jenkins script 2016-12-06 21:50:51 -08:00
Mike Aizatsky ba0304c6a9 [infra] providing libFuzzingEngine.a (#139) 2016-12-06 15:54:53 -08:00
Mike Aizatsky 93476cdefe Update README.md 2016-12-05 09:44:57 -08:00
Mike Aizatsky 85cdb1cc07 Update README.md 2016-12-02 11:02:18 -08:00
Mike Aizatsky d48041d880 Update README.md 2016-12-02 11:02:02 -08:00
Mike Aizatsky 18125f52db Update README.md 2016-12-02 10:59:38 -08:00
Mike Aizatsky f478dca0e8 [infra] SANITIZER environment variable (#103)
The variable picks one of the predefined flags configuration.
2016-12-02 10:58:51 -08:00
inferno-chromium 832d367b37 Update README.md 2016-11-30 08:26:02 -08:00
Mike Aizatsky e16ee1785b [infra] using project instead of target 2016-11-29 11:23:30 -08:00
Mike Aizatsky a0611d84d1 Update README.md 2016-11-22 12:03:13 -08:00
Mike Aizatsky 6b5163a38e Update README.md 2016-11-22 12:02:26 -08:00
Mike Aizatsky c0fc3a4ea3 [infra] moving symbolizer to base runner 2016-11-21 12:35:58 -08:00
Mike Aizatsky d92be61e1c [infra] adding zip package, it is needed for corpuses 2016-11-18 20:52:41 -08:00
Mike Aizatsky a27eca33a3 [infra] adding zip to base-runner: run_fuzzer needs it 2016-11-18 18:34:18 -08:00
Mike Aizatsky 0b58d66e89 [infra] moving test command into base-runner, using it on jenkins 2016-11-18 16:46:26 -08:00
Mike Aizatsky 456e952486 silencing pushd 2016-11-18 16:46:04 -08:00
Mike Aizatsky ceb465a0e8 Create README.md 2016-11-18 14:59:10 -08:00
Mike Aizatsky 01364bf2b7 [infra] extracting base-runner image 2016-11-18 14:53:09 -08:00
Mike Aizatsky 26630f32cb [infra] .a merging is no longer necessary b/c it now happens during llvm build
60479ea31b

Thanks @EricWF

Issue #73
2016-11-18 12:45:02 -08:00
Mike Aizatsky 711ffb3247 [infra] silencing ar 2016-11-18 12:23:23 -08:00
Mike Aizatsky 487e9f4ed6 [infra] removing FUZZER_LDFLAGS (fixes #73)
Workarounds libc++abi issue by folding libc++abi.a into libc++.a.
Will keep FUZZER_LDFLAGS empty for a while until tpm2 is fixed.
2016-11-18 12:16:35 -08:00
Mike Aizatsky a1d663b95e [fuzzer] adding ldd printout 2016-11-18 11:58:32 -08:00
Mike Aizatsky a0a5872b44 removed debug output 2016-11-18 11:44:37 -08:00
Mike Aizatsky 3599908dbf [infra] use $src, $out and $work in build scripts instead of /src, /out, /work (#88)
This will make it possible to run scripts outside of docker container.
2016-11-18 11:16:38 -08:00
Mike Aizatsky 5dfb810427 [infra] setting current directory to src checkout (#87) 2016-11-17 09:49:34 -08:00
Mike Aizatsky 9e8151b5cf [infra] less output 2016-11-15 12:09:06 -08:00
Mike Aizatsky 95605c91d9 [infra] adding -x to compile scripts to see all commands that are executed 2016-11-15 11:35:25 -08:00
Mike Aizatsky e29ef5876d rolling back trace pc guards
fuzzers fail with:

=================================================================
==18057==ERROR: AddressSanitizer: global-buffer-overflow on address 0x000000b8a144 at pc 0x0000007ae0ca bp 0x7fff2b91a4d0 sp 0x7fff2b91a4c8
WRITE of size 4 at 0x000000b8a144 thread T0
    #0 0x7ae0c9 in fuzzer::TracePC::HandleInit(unsigned int*, unsigned int*) /src/libfuzzer/FuzzerTracePC.cpp:49:8
    #1 0x7bcab9 in __sanitizer_cov_trace_pc_guard_init /src/libfuzzer/FuzzerTracePC.cpp:286:15
    #2 0x5156bf in sancov.module_ctor (/out/curl_fuzzer+0x5156bf)
    #3 0x88c1cc in __libc_csu_init (/out/curl_fuzzer+0x88c1cc)
    #4 0x7f4ab7aed7be in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x207be)
    #5 0x41fd78 in _start (/out/curl_fuzzer+0x41fd78)
2016-11-14 12:21:02 -08:00
Mike Aizatsky c771c31753 switching to trace-pc-guard 2016-11-14 09:49:30 -08:00
Mike Aizatsky 528d2e6900 Update README.md 2016-11-11 15:25:37 -08:00
Mike Aizatsky 43a5d34649 do not install unneded packages, they come from base-clang 2016-11-04 15:48:58 -07:00
Mike Aizatsky 5cf48acc50 installing packages needed to build binaries 2016-11-04 15:31:22 -07:00
Max Moroz 7d834ef339 [infra] Add zip installation to libfuzzer-runner image.
To fix: /usr/local/bin/run_fuzzer: line 37: unzip: command not found.
2016-11-04 15:13:02 +01:00
Mike Aizatsky 48778facde folding coverage into base-libfuzzer since target images is where sources are 2016-11-03 20:13:29 -07:00
Mike Aizatsky 43489960b0 Update run_fuzzer 2016-11-02 17:30:46 -07:00
Mike Aizatsky 19a558f8b8 Update just_run 2016-11-02 17:30:33 -07:00
Mike Aizatsky da2ddedf02 setting path in just_run 2016-11-02 16:33:21 -07:00
Mike Aizatsky 7df69b51d1 test uses .options and seed corpus 2016-11-02 16:24:50 -07:00
Mike Aizatsky 217223fd72 use seed corpus while running a fuzzer. Fixes #19 2016-11-02 16:22:37 -07:00
Mike Aizatsky 8e2aa92c9c helper.py run should use .options file
It is very unfortunate, but we have to copy the file. I'll think
how it is possible to restructure our images to avoid duplication.

Issue #19
2016-11-02 16:07:48 -07:00
Mike Aizatsky e1a2d3c14c [infra] .options support for run command (#61)
Initial work for #19.
2016-11-02 16:02:17 -07:00
Mike Aizatsky fddbae61b7 pipefail fuzzer run 2016-11-02 14:40:01 -07:00
Mike Aizatsky efbcadbe12 putting libfuzzer.a into /usr/lib/ 2016-11-01 13:19:39 -07:00
Mike Aizatsky 28dec3521c [infra] adding zip to the image, since corpus files should be zips 2016-10-28 15:24:34 -07:00
Mike Aizatsky 916e155609 improving junit report a bit 2016-10-27 14:14:59 -07:00
Mike Aizatsky 39479734da cleanup 2016-10-27 10:08:01 -07:00
Mike Aizatsky 3b64ebbc9e using basename for report 2016-10-26 15:19:35 -07:00
Mike Aizatsky d4086332cd generating dummy test report 2016-10-26 14:22:28 -07:00
Mike Aizatsky 459b0c1b40 Update compile 2016-10-26 13:53:05 -07:00
Mike Aizatsky 43de5caa37 [infra] reproduce command (#53)
runs specified fuzzer with `/testcase` as input.

example:

```
docker run -v <local_file>:/testcase -t ossfuzz/libxml2 reproduce libxml2_xml_read_memory_fuzzer
```
2016-10-26 08:28:38 -07:00
Mike Aizatsky a95b41b0b3 replacing library with target 2016-10-25 16:36:29 -07:00
Mike Aizatsky 6e41f2f824 [infra] hg sourcemap 2016-10-20 14:31:20 -07:00
Mike Aizatsky 22afda8b9e [infra] fail test if there are zero fuzzers 2016-10-20 14:14:15 -07:00
Mike Aizatsky 6ccbe7f757 [infra] revisions->srcmap cleanup 2016-10-20 13:15:28 -07:00
Mike Aizatsky 7a1d8d645a [infra] renaming revisions.json into srcmap.json 2016-10-20 00:45:57 -07:00
Mike Aizatsky 20de8979b1 [infra] test command 2016-10-20 00:44:35 -07:00
Mike Aizatsky 59f3805868 [infra] make revisions.json world readable 2016-10-20 00:28:16 -07:00
Mike Aizatsky 9d54ccaeee [infra] getting back to root user 2016-10-20 00:24:22 -07:00
Mike Aizatsky 4a1014636d [infra] chown 2016-10-19 18:07:08 -07:00
Mike Aizatsky 0334615e51 [infra] determining source code revisions 2016-10-19 15:07:24 -07:00
Mike Aizatsky f61b870b1c libtool is needed to build libfuzzer.a 2016-10-19 11:42:57 -07:00
Mike Aizatsky faecacb464 adding subversion 2016-10-19 11:34:58 -07:00
Mike Aizatsky ce3c1a60c8 calling script directly 2016-10-19 10:03:42 -07:00
Mike Aizatsky ea957bd4ee [infra] checking out library sources in the image (#40)
fixes #33
2016-10-18 15:37:23 -07:00
Mike Aizatsky 089add6e8b [infra] replacing libfuzzer/*.o by -lfuzzer
Fixes issue #32
2016-10-17 14:37:19 -07:00
Vitaly Buka bc1aedf2f9 Add docker image for coverage 2016-10-14 15:28:32 -07:00
Mike Aizatsky 941e47d6e1 [infra] renaming LDFLAGS into FUZZER_LDFLAGS. Fixes #31 2016-10-13 14:20:19 -07:00
Mike Aizatsky 744ae17922 Update README.md 2016-10-13 14:02:19 -07:00
Mike Aizatsky d1e59b5bbd Update Jenkinsfile 2016-10-12 16:26:40 -07:00
Mike Aizatsky dae2012980 [infra] do not checkout oss-fuzz (#23)
Promising oss-fuzz in /src/oss-fuzz creates lots of confusion about where files come from.
Let's make everything explicit.

Fixes #20
2016-10-12 16:25:06 -07:00
Mike Aizatsky 58dc27a57e Update run 2016-10-10 15:49:54 -07:00
Mike Aizatsky ea2bf52e10 Update README.md 2016-10-10 14:21:59 -07:00
Mike Aizatsky 28e7afba10 [infra] run script 2016-10-10 14:16:48 -07:00
Mike Aizatsky 106efbe664 Update Jenkinsfile 2016-10-10 14:12:25 -07:00
Mike Aizatsky 85dad5d52d [infra] checkout command: automatic code checkout and compiling (#11) 2016-10-10 13:21:45 -07:00
Mike Aizatsky 286e3d8e0a [infra] removed image flattening: env variables do not surviver 2016-10-07 12:27:58 -07:00
Mike Aizatsky de6c41a671 [infra] building clang with a single RUN command.
Each RUN command creates an overlay layer. This results in keeeping all
intermediate files.

By consolidating all work in a single script, the size of base-clang
image is decreased from 3G to 1G.
2016-10-07 12:06:24 -07:00
Mike Aizatsky c08356aa08 Update README.md 2016-10-07 11:35:01 -07:00
Mike Aizatsky 84584a535e Create README.md 2016-10-07 11:34:42 -07:00
Mike Aizatsky 389fb0e4ab disable image flattening for a while 2016-10-06 16:53:03 -07:00
Mike Aizatsky 1f6691451c Update all.sh 2016-10-06 15:23:38 -07:00
Mike Aizatsky 9bca6e5bbc [infra] flattening base-clang image
The image has lots of layers with lots of deleted files (sources and intermediates).
Flatten the image to reduce its size.
2016-10-06 14:57:40 -07:00
Mike Aizatsky e2ca7bcbe0 [infra] fixing build script location (/src/build.sh)
Fixing the location of build.sh gets rid of ENTRYPOINT. Which opens a possibility to implement different commands like:

````bash
docker run -ti expat compile
docker run -ti expat run <fuzzer_name> <input_data>
````

and keep the knowledge of build script location.

This is a breaking change.
2016-10-06 13:45:12 -07:00
Mike Aizatsky e4c18c4797 Update Jenkinsfile 2016-10-05 19:05:24 -07:00
Mike Aizatsky 537a5ed7c4 Updated README.md 2016-10-05 15:07:52 -07:00
Mike Aizatsky 55dafa1543 [infra] added llvm-symbolizer to libfuzzer-runner 2016-10-05 14:26:40 -07:00
Mike Aizatsky c7194d1b58 [infra] base-images build script 2016-10-05 14:18:14 -07:00
Mike Aizatsky d204145f6b [infra] added gdb to libfuzzer-runner 2016-10-03 13:33:35 -07:00
Oliver Chang 8e4f09aaa6 Don't detect leaks in libfuzzer build images. 2016-09-29 13:18:39 -07:00
Mike Aizatsky 6fb6912b25 specifying default asan options 2016-09-27 15:57:51 -07:00
Mike Aizatsky 7f070f9f65 [infra] set workdir to /out 2016-09-14 12:14:11 -07:00
Mike Aizatsky 4d88838907 [infra] printing libfuzzer work location 2016-09-13 13:15:55 -07:00
Mike Aizatsky da4ce50ee9 [infra] do not build msan
it conflicts with static-only libc++.
2016-08-16 13:19:33 -07:00
Mike Aizatsky b580229493 [infra] build static libc++ only 2016-08-16 12:56:42 -07:00
Mike Aizatsky c9f6190beb Create README.md 2016-08-16 12:53:06 -07:00
Mike Aizatsky b9103917c5 [infra] build only x86 2016-08-15 15:24:03 -07:00
Mike Aizatsky 5c1713225e [infra] building msan libc++ variant 2016-08-15 15:13:30 -07:00
Mike Aizatsky f36e9b176d [infra] disabling msan build for a while. 2016-08-11 17:02:20 -07:00
Mike Aizatsky af93794f39 [infra] using whole-archive options 2016-08-11 15:42:21 -07:00
Mike Aizatsky 69110e1f33 [infra] no need to call /env 2016-08-07 19:53:25 -07:00
Mike Aizatsky a4b1f6c2a5 [infra] moved env variables into Dockerfiles 2016-08-06 11:11:39 -07:00
Mike Aizatsky 3b9b8fe433 [infra] updated linker flags. 2016-08-06 10:29:42 -07:00
Mike Aizatsky 255a2c63b4 [infra] using LDFLAGS with static library linkage 2016-08-05 16:29:38 -07:00
Mike Aizatsky c687a7ec43 [infra] runner image 2016-08-05 14:35:26 -07:00
Mike Aizatsky f1b094b16a [infra] removing workspace directory 2016-08-01 14:15:43 -07:00
Mike Aizatsky a8394ce0b6 adding make to Dockerfiles 2016-07-28 13:46:35 -07:00
Mike Aizatsky 0dd9848893 [infra] docker images nits 2016-07-27 16:07:07 -07:00
Mike Aizatsky 7ef8ddd092 [infra] docker images nits 2016-07-27 16:04:01 -07:00
Mike Aizatsky e0b3abddf8 [infra] checking out oss-fuzz into base-libfuzzer 2016-07-25 22:38:54 -07:00
Mike Aizatsky fbae9ad868 [infra] adding lc++abi 2016-07-25 16:02:56 -07:00
Mike Aizatsky 33a03d3a3f [infra] more stages 2016-07-25 13:27:11 -07:00
Mike Aizatsky 3dcb9040c1 [infra] do not push docker images 2016-07-25 13:17:37 -07:00
Mike Aizatsky 679945be2d [infra] env script cleanup 2016-07-22 14:05:53 -07:00
Mike Aizatsky 50a790b896 [infra] removing build artifacts after installation 2016-07-22 13:24:49 -07:00
Mike Aizatsky 3fa412a17c [infra] more dockerfiles tweaks 2016-07-21 14:10:04 -07:00
Mike Aizatsky 06a204d5cd [infra] docker files cleanup 2016-07-21 13:51:01 -07:00
Mike Aizatsky 99633cda6a [infra] docker image name update 2016-07-21 12:43:53 -07:00
Mike Aizatsky afa7a89f2c [infra] fixed stage syntax 2016-07-21 10:14:13 -07:00
Mike Aizatsky 95bddce1ae [infra] renaming base-fuzzer into base-libfuzzer image. 2016-07-20 17:16:57 -07:00
Mike Aizatsky c23a04daf7 [infra] logging into docker 2016-07-20 17:09:09 -07:00
Mike Aizatsky b9e00ed4ba [infra] updating base-mages Jenkinsfile 2016-07-20 15:51:36 -07:00
Mike Aizatsky 03d9ccfa97 base-images Jenkinsfile tweaks 2016-07-20 15:11:12 -07:00
Mike Aizatsky 6de229ba12 disabling docker cleanup for a while 2016-07-20 15:09:52 -07:00
Mike Aizatsky 1d0da0ad89 images->base-images folder rename to match jenkins job 2016-07-20 15:07:26 -07:00