* Get list of changed files from branch head, instead of master.
Fixes https://github.com/google/oss-fuzz/issues/5022
* Add debug with subprocess.call.
* Try again debugginig.
* Try again
* Fix works!
The previous approach of only running tests in changed directories is broken.
Tests can fail even when files outside of their directory are modified.
Also blocklist failing tests (see https://github.com/google/oss-fuzz/issues/5025) for why build tests are blocklisted.
Previously region count was used instead of region covered.
This means that unaffected fuzzers only worked when a file wasn't linked
into a fuzzer build (i.e. it was mostly broken).
Add tests to ensure this doesn't happen again.
Fixes: #5013
Also refactor.
1. Create a coverage module and move coverage functionality there.
2. Remove some overly-defensive programming. We probably aren't going to be given an invalid repo dir in coverage module.
3. Convert integration test to unittest.
4. Add helpers for: normalizing paths, getting coverage per file, determining if file is covered (which was done incorrectly before), and getting fuzzer stats dir url to make code easier to understand.
5. Add a class for getting coverage info.
6. Create an affected_fuzz_targets module and move functionality dealing with affected_fuzz_targets there.
7. Add is_fuzz_target_affected helper and log more.
8. Refer to fuzz targets as fuzz targets instead of fuzzers.
9. Move `url_join` to `utils`.
10. Move `GSUTIL_BASE_URL` to `utils`.
11. Add a util function for converting gs:// URLs into https:// urls.
12. Add a util function for removing prefixes (instead of using `.replace` which operates on the whole string not just the prefix).
13. Use more common style of mocking in unittests.
14. Delete unnecessary tests like '' when an invalid fuzzer is already tested.
15. Make constants capitalized in tests.
16. Better variable naming and consistency also reuse variables in tests.
17. Leave TODOs around code that looks suspicious.
18. Cleanup pylint directives.
19. Use single instead of double quotes.
Move entrypoints for CIFuzz to cifuzz.
This allows us to reduce some complexity by getting rid of
an unnecessary copy in docker and a hack to making importing work.
`go list` will fail if all files in the fuzzed package use a build tag
restriction (like the common `gofuzz` tag).
Also, pass the `gofuzz` tag in the teleport build to plumb it through.
1. Add a get_affected_fuzzers function.
2. Add comments explaining rationales.
3. Fix logging so that each affected fuzzer is only logged once.
4. Don't put each file affecting each fuzzer on its own line, this makes logs unreadable.
5. Improve logging wording.
* Revert "Update Dockerfile"
This reverts commit 5eae19fb57.
* Revert "Move python 3 install to base-clang, needed for clang build. (#4967)"
This reverts commit 018314eefa.
* Use python3 in checkout_build_install_llvm.sh
```
-- Looking for os_signpost_interval_begin - not found
CMake Error at /usr/local/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:218 (message):
Could NOT find Python3 (missing: Python3_EXECUTABLE Interpreter)
Call Stack (most recent call first):
/usr/local/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:582 (_FPHSA_FAILURE_MESSAGE)
/usr/local/share/cmake-3.19/Modules/FindPython/Support.cmake:3148 (find_package_handle_standard_args)
/usr/local/share/cmake-3.19/Modules/FindPython3.cmake:398 (include)
CMakeLists.txt:711 (find_package)
-- Configuring incomplete, errors occurred!
See also "/work/llvm-stage1/CMakeFiles/CMakeOutput.log".
See also "/work/llvm-stage1/CMakeFiles/CMakeError.log".
The command '/bin/sh -c /root/checkout_build_install_llvm.sh' returned a non-zero code: 1
```
1. Fix problem where permissions were being changed to root by non-root test (test was doing this by invoking test_all.py within docker).
2. Mark tests as integration tests so that cifuzz_test.py can be run in a reasonable amount of time.
3. Prevent some unittests from polluting source repo.
4. Add .venv to .gitignore
5. Rename test_test_all.py to the correctly formatted name "test_all_test.py"
When we fail a build, we attempt to identify the first OSS-Fuzz commit
prior to the upstream commit date.
If this does not exist, we bailed out. This commit changes it such that
we at least try on the oldest integration commit.
Rewrite test_all in python.
Bash is quite annoying to write and test.
One issue with bash is it is even worse than Python for parallelism (which may be causing #4707).
Rewrite test_all in python and optimize base-runner/Dockerfile for fast development.
Also, combine some docker layers.
* Update infra/base-images/all.sh
Add build of base-sanitizer-libs-builder and msan-libs-builder to this
shell script.
* msan: Don't warn on un-instrumented standard libs
These libraries do not need to be built with instrumentation, because
MemorySanitizer includes interceptors for them.
* Fix indentation
* Add missing docstrings
* Fix unused variable
* Fix invalid names
* Install python-apt on CI
* Revert "Install python-apt on CI"
This reverts commit d3da49cf90.
* Install and use python-apt in system directory
* Revert "Install and use python-apt in system directory"
This reverts commit e0ede101fb.
* Build python-apt from source
* Check out correct version of python-apt
* Fix octal literals
* More indentation fixes
* Add more missing docstrings
* Change variable names of opened files
* Remove unused import
* Ignore lints about package.Package API
* Fix or ignore remaining invalid names
* Fix apparent typo in compiler_wrapper_test.py
-z should precede a keyword, not a long option
* Fix use of xrange
* Style fixes, compiler_wrapper
* Fix apparent error in compiler_wrapper_test.py
Similar to the previous error, the test case would pass "-z
--no-undefined" to the linker. "-z" only has an effect when it is
followed by a keyword, otherwise ld ignores it and prints a warning
message. In this test case, "-z" and "--no-undefined" were passed in two
separate "-Wl," compiler arguments, but they reflect a common issue.
* Add missing license header
* Rename more functions
* Better name for global variable
* Rename methods of Package
* Rename functions in msan_builder.py
* Fix invalid variable names
* Fix useless-object-inheritance
* pylint: Fixes for Package and its subclasses
* Remove unused imports
* Indentation fixes
* Fix too-may-locals error in msan_build.py
* Add missing docstrings
* Good fuzz target for golang coverage with modules
* Place target in right directory for go-json-iterator
So that coverage gets access to the right package
* Golang coverage summary for each fuzz target
* Document usage of compile_go_fuzzer
* update the documentation change
Co-authored-by: Max Moroz <mmoroz@chromium.org>
This will help us catch breaking changes to the base-images.
Unfortunately caching seems to fail here when I expect it to help.
For example, base-builder doesn't build from cache when I do it locally.
This means that every other image I try to build doesn't use the cache.
That means that base-clang would take forever to rebuild.
So to compromise, I don't rebuild base-clang here.
This means that this PR won't catch breaking changes to base-image or base-clang that break in base-builder.
But it will catch breaking changes to base-image that break in base-runner and it will catch breaking changes to base-runner and base-builder.
[CI] Build a canary project on infra/ changes.
Build a specific project, sckms, that does msan, ubsan, asan, i386
builds quickly, when infra/ code is changed. This can let us know
when infra/ changes break proper functioning of OSS-Fuzz.
For this to work more thoroughly we also need to rebuild images.
* Initial support for ujson (as a Python fuzzing sample).
* Add python to supported languages.
* Fix TODO.
* Update presubmit to ignore python in projects.
* Format
* Remove explicit addition of -fsanitize=fuzzer-no-link
* Only fuzz using libFuzzer.
Co-authored-by: Martin Barbella <mbarbella@google.com>
Fix some coding and style issues.
1. Treat "testcase" as one word. This is consistent with ClusterFuzz.
2. Treat "stacktrace" as one word. This is consistent with ClusterFuzz.
3. Fix spelling of comments and words in code.
4. Use descriptive style docstrings and remove unnecessary clutter
(e.g. "Test the functionality of foo_function function in bar module"
becomes "Tests foo_function."
5. Use JSON for a list needed in testing instead of pickle.
[CIFuzz] Don't assume fuzzer output is ascii
This is wrong because the output can be arbitrary. Instead change
code that deals with the output to deal with bytes. The testcase,
which is derived from the output can be decoded as UTF-8 since it
will be a unix path.
Skia has seen some unhandled connection reset errors.
When we see these errors when downloading old OSS-Fuzz build, retry downloading it, since the error is likely transient.
This is needed for CF to determine whether or not to allocate the
project on a higher end bot to run the blackbox fuzzer.
Previously this was keyed on the "none" entry under fuzzing_engines,
which wasn't very descriptive. This change also lets us do both blackbox
and greybox fuzzing in the same project, which will be done for quickjs.
Coverage builds need to be special cased since they aren't specified
in sanitizers. Instead they are done for all C/C++ projects that
use libFuzzer.
Move all of this special casing to `should_build_coverage` and call that from
`should_build` so we have one place where we decide this.
Add tests as well.
Fixes: https://github.com/google/oss-fuzz/issues/4371
* [infra] Fix reproduce command invoking run_fuzzer for a single testcase (#4338).
* review feedback
* do not introduce new variable, rely on SKIP_SEED_CORPUS only
The issue with 55d9a81 that caused it to be reverted by 3370f8f
was probably due to the build trigger on GCB being stale. I've
updated it so this commit should stick.
Make CIFuzz building accept an env var `MANUAL_SRC_PATH` that points to a manually prepared checkout of the project-under-tests's source code. This allows projects like Skia which are not on OSS-Fuzz and/or need to make changes to the repo after checking out the right commit to use CIFuzz.
Note that for now we aren't supporting this in GitHub, so projects that need to modify the source after checkout can't use it until
we do support it.
Also, use the local copy of OSS-Fuzz when building cifuzz-base instead of cloning it from GitHub. This makes local debugging/development much easier since it allows one to use CIFuzz with local changes.
- Remove gcb folder. Move the files that we still need into the
build/functions dir.
- Remove badge_images. They are now stored on GCS.
- Remove jenins configs.
* Initial UI Change to display historical logs
* Minor changes
* More formatting changes
* Adding license header
* Changes, take a look Oliver
* Changes for handling empty build history/ no last successful build
* Reverting the logs folder path from /logs/log to /log
* Merged to latest master and other chnages to template
* Changing backend to reflect build history
* Fixing order of query
* Removed last build status
* More changes
* Adding unit tests and fixes found by unit testing
* Fixing lint errors
* Added more unit tests and fixed typos and formatting
* Fixing comment caps
* Minor formatting issue
Co-authored-by: Oliver Chang <oliverchang@users.noreply.github.com>
* [rnp] Add afl engine
* Add fuzz_dump and fuzz_keyring to the exception list
These binaries - from https://github.com/rnpgp/rnp/tree/master/src/fuzzing -
are very tiny and just call into the library with a single API,
and are not staticly compiled - so they have very few UBSAN calls.
- Remove base image project argument from deploy.sh. Deploy all
functions to the main oss-fuzz image.
- Reduce max instances of functions to 1 to avoid rate limiting issues.
- Fix missing ndb context initialization in request_build.py
- Fix incorrect tags in BuildHistory entities. "-" was doubled.
- Fix base build deployment with incorrect schedule variable.
- Add scripts for requesting builds locally.
Abhishek Arya