Rooba
/
oss-fuzz
mirror of https://github.com/google/oss-fuzz.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,171 Commits 189 Branches 0 Tags 121 MiB
Commit Graph

1271 Commits

Author SHA1 Message Date
Abhishek Arya 0dc7c358d8
Remove unneeded ADD layers in base-builder. (#5028) 
See https://github.com/google/oss-fuzz/issues/5012#issue-789981845
 2021-01-20 22:18:03 -08:00
jonathanmetzman 492a4f2a1d
Run all tests when infra-tests is run from presubmit (#5026) 
The previous approach of only running tests in changed directories is broken.
Tests can fail even when files outside of their directory are modified.
Also blocklist failing tests (see https://github.com/google/oss-fuzz/issues/5025) for why build tests are blocklisted.
 2021-01-20 18:17:33 -08:00
Jonathan Metzman c5397ce3df fix nits 2021-01-20 18:00:33 -08:00
Jonathan Metzman e4195808a8 Explain why blacklisting base-sanitizer-libs-builder 2021-01-20 17:58:06 -08:00
jonathanmetzman f913f61129
[CIFuzz][coverage] Fix bug where unaffected fuzzers not removed + refactor 
Previously region count was used instead of region covered.
This means that unaffected fuzzers only worked when a file wasn't linked 
into a fuzzer build (i.e. it was mostly broken).
Add tests to ensure this doesn't happen again.

Fixes: #5013

Also refactor.
1. Create a coverage module and move coverage functionality there.
2. Remove some overly-defensive programming. We probably aren't going to be given an invalid repo dir in coverage module.
3. Convert integration test to unittest.
4. Add helpers for: normalizing paths, getting coverage per file, determining if file is covered (which was done incorrectly before), and getting fuzzer stats dir url to make code easier to understand.
5. Add a class for getting coverage info.
6. Create an affected_fuzz_targets module and move functionality dealing with affected_fuzz_targets there.
7. Add is_fuzz_target_affected helper and log more.
8. Refer to fuzz targets as fuzz targets instead of fuzzers.
9. Move `url_join` to `utils`.
10. Move `GSUTIL_BASE_URL` to `utils`.
11. Add a util function for converting gs:// URLs into https:// urls.
12. Add a util function for removing prefixes (instead of using `.replace` which operates on the whole string not just the prefix).
13. Use more common style of mocking in unittests.
14. Delete unnecessary tests like '' when an invalid fuzzer is already tested.
15. Make constants capitalized in tests.
16. Better variable naming and consistency also reuse variables in tests.
17. Leave TODOs around code that looks suspicious.
18. Cleanup pylint directives.
19. Use single instead of double quotes.
 2021-01-20 16:00:35 -08:00
Jonathan Metzman 0dbdeae5d2 fmt/lnt 2021-01-20 15:55:25 -08:00
Jonathan Metzman adebf8ece3 implement blocklist 2021-01-20 15:52:20 -08:00
Jonathan Metzman 9ce539763f fix 2021-01-20 15:18:18 -08:00
Jonathan Metzman 1193e419a3 Disable linting on dataflow_tracer 2021-01-20 13:40:50 -08:00
Jonathan Metzman 4184718c61 undo changes 2021-01-20 13:34:26 -08:00
Jonathan Metzman 05dc963d03 add license 2021-01-20 13:31:16 -08:00
Jonathan Metzman b3e30e5170 Fix lint function and add pylint directive 2021-01-20 13:29:47 -08:00
Jonathan Metzman a5ad91c7f5 Fix docstring 2021-01-20 13:25:56 -08:00
Jonathan Metzman 2b26c5446f Lint everything (and test everything too? 2021-01-20 13:25:56 -08:00
jonathanmetzman b998058ef3
Move entrypoints for CIFuzz to cifuzz folder. (#5020) 
Move entrypoints for CIFuzz to cifuzz.
This allows us to reduce some complexity by getting rid of
an unnecessary copy in docker and a hack to making importing work.
 2021-01-20 13:23:55 -08:00
Abhishek Arya 9d7f4f3100
Bump up llvm version to fix compile failure regression (old gcc) 2021-01-20 13:06:56 -08:00
Jonathan Metzman 5a48e9109a Merge branch 'master' of github.com:google/oss-fuzz into cifuzz-cov 2021-01-20 13:05:35 -08:00
Jonathan Metzman 63925e0e0d match behavior of removeprefix 2021-01-20 12:59:11 -08:00
Jonathan Metzman 64aeebf94f json.load -> json.loads 2021-01-20 12:53:47 -08:00
Jonathan Metzman 3a9668edd0 remove newline 2021-01-20 12:52:44 -08:00
Andrew Lytvynov 687187f07e
compile_go_fuzzer: pass build tags to `go list` (#5008) 
`go list` will fail if all files in the fuzzed package use a build tag
restriction (like the common `gofuzz` tag).

Also, pass the `gofuzz` tag in the teleport build to plumb it through.
 2021-01-20 11:44:15 -08:00
Jonathan Metzman d7049a82f2 fmt/lnt 2021-01-20 11:40:55 -08:00
Abhishek Arya 26dd144e52
Build modified projects as well when infra is modified. (#5019) 2021-01-20 11:37:37 -08:00
Jonathan Metzman cf0c702ee8 fix 2021-01-20 11:33:16 -08:00
Jonathan Metzman a5a1504477 add tests back 2021-01-20 11:24:25 -08:00
Jonathan Metzman aa815fc33b Fix unittests and make sure functionality stays same 2021-01-20 11:19:15 -08:00
Jonathan Metzman 0c26e0e2c8 fmt 2021-01-20 10:31:18 -08:00
Jonathan Metzman 2800e4eefa Add test 2021-01-20 10:31:03 -08:00
Jonathan Metzman 1647e41bef fmt 2021-01-20 10:18:39 -08:00
Jonathan Metzman 453253c427 fmt/lnt 2021-01-20 10:16:02 -08:00
Jonathan Metzman c136dd660b Fix bug where region count was used to determine if file was covered 
instead of region coverage.
Add a test for this.
Add a remove_prefix util function.
 2021-01-20 10:13:42 -08:00
Abhishek Arya c3087b7018
Install rust in base-builder with minimal profile. (#5015) 
Remove docs and other unneeded stuff.
https://blog.rust-lang.org/2019/10/15/Rustup-1.20.0.html
 2021-01-20 08:59:05 -08:00
Jonathan Metzman fe7b56af48 fmt/lnt 2021-01-20 08:48:29 -08:00
Jonathan Metzman 95f1089d7a fmt 2021-01-20 08:48:02 -08:00
Jonathan Metzman a00890cb73 more tests 2021-01-20 08:46:38 -08:00
Jonathan Metzman dafddab431 move more tests 2021-01-20 08:22:24 -08:00
Jonathan Metzman 8cda07fb9f move over test 2021-01-20 07:59:32 -08:00
Jonathan Metzman 913962ba54 Add coverage_test module and fmt/lnt 2021-01-20 07:38:27 -08:00
Jonathan Metzman 9935321c0d fmt/lnt 2021-01-20 07:26:02 -08:00
Jonathan Metzman ddb0add036 fuzzers->fuzz targets and finish affected_fuzz_targets module 2021-01-20 07:25:08 -08:00
Jonathan Metzman 0be9a235b8 merge 2021-01-20 07:02:39 -08:00
Jonathan Metzman 50109305cb New module for affected fuzz targets 2021-01-20 06:56:13 -08:00
Jonathan Metzman cb262dfbcb fmt/lnt 2021-01-20 06:51:30 -08:00
Jonathan Metzman 2ce7db4d47 Refactor coverage and move to own module 2021-01-20 06:47:48 -08:00
jonathanmetzman b9094ea003
[CIFuzz][NFC] Refactor affected fuzzers and improve logging (#5006) 
1. Add a get_affected_fuzzers function.
2. Add comments explaining rationales.
3. Fix logging so that each affected fuzzer is only logged once.
4. Don't put each file affecting each fuzzer on its own line, this makes logs unreadable.
5. Improve logging wording.
 2021-01-20 06:41:39 -08:00
Abhishek Arya 24d1b4e01e
Remove unneeded honggfuzz examples from base-builder. (#5011) 
Save 177Mb.

root@eca2ea09a598:/src/honggfuzz/examples# du -h
16K	./glibc
8.0K	./libxml2
2.4M	./openssl/corpus_privkey
5.1M	./openssl/corpus_x509
18M	./openssl/corpus_client
27M	./openssl/corpus_server
52M	./openssl
16K	./externalfuzzers
12K	./libpng
25M	./bind/corpus
25M	./bind
29M	./linux_kernel_ip/corpus
30M	./linux_kernel_ip
16K	./libjpeg
20K	./terminal-emulators
33M	./apache-httpd/corpus_http1
39M	./apache-httpd/corpus_http2
72M	./apache-httpd
12K	./file
16K	./badcode/targets
8.0K	./badcode/inputfiles
36K	./badcode
177M	.
 2021-01-20 05:51:08 -08:00
Stefan Bucur 54232af56f
Add the Bazel dependencies to the base builder image. (#4976) 
* Add the Bazel dependencies to the base builder image.

* Removing Python 2 from the base image.

* Fix the Bazelisk fetching script.
 2021-01-20 05:33:01 -08:00
Oliver Chang 6e41513dbb
build_specified_commit: Remove an exception. (#5009) 
Return None rather than exceptioning out when a suitable base-builder
cannot be found to allow more graceful error handling.
 2021-01-20 16:52:49 +11:00
Catena cyber 433ddc2337
Sets language env variable when running shell (#5007) 
* Sets language env variable when running shell

* Always sets FUZZING_LANGUAGE
 2021-01-19 14:20:39 -08:00
Abhishek Arya a252dc2d3a
Use python3 in checkout_build_install_llvm.sh, revert python move patches. (#4974) 
* Revert "Update Dockerfile"

This reverts commit 5eae19fb57.

* Revert "Move python 3 install to base-clang, needed for clang build. (#4967)"

This reverts commit 018314eefa.

* Use python3 in checkout_build_install_llvm.sh
 2021-01-14 15:11:59 -08:00