1. Fix problem where permissions were being changed to root by non-root test (test was doing this by invoking test_all.py within docker).
2. Mark tests as integration tests so that cifuzz_test.py can be run in a reasonable amount of time.
3. Prevent some unittests from polluting source repo.
4. Add .venv to .gitignore
5. Rename test_test_all.py to the correctly formatted name "test_all_test.py"
When we fail a build, we attempt to identify the first OSS-Fuzz commit
prior to the upstream commit date.
If this does not exist, we bailed out. This commit changes it such that
we at least try on the oldest integration commit.
Rewrite test_all in python.
Bash is quite annoying to write and test.
One issue with bash is it is even worse than Python for parallelism (which may be causing #4707).
Rewrite test_all in python and optimize base-runner/Dockerfile for fast development.
Also, combine some docker layers.
* Update infra/base-images/all.sh
Add build of base-sanitizer-libs-builder and msan-libs-builder to this
shell script.
* msan: Don't warn on un-instrumented standard libs
These libraries do not need to be built with instrumentation, because
MemorySanitizer includes interceptors for them.
* Fix indentation
* Add missing docstrings
* Fix unused variable
* Fix invalid names
* Install python-apt on CI
* Revert "Install python-apt on CI"
This reverts commit d3da49cf90.
* Install and use python-apt in system directory
* Revert "Install and use python-apt in system directory"
This reverts commit e0ede101fb.
* Build python-apt from source
* Check out correct version of python-apt
* Fix octal literals
* More indentation fixes
* Add more missing docstrings
* Change variable names of opened files
* Remove unused import
* Ignore lints about package.Package API
* Fix or ignore remaining invalid names
* Fix apparent typo in compiler_wrapper_test.py
-z should precede a keyword, not a long option
* Fix use of xrange
* Style fixes, compiler_wrapper
* Fix apparent error in compiler_wrapper_test.py
Similar to the previous error, the test case would pass "-z
--no-undefined" to the linker. "-z" only has an effect when it is
followed by a keyword, otherwise ld ignores it and prints a warning
message. In this test case, "-z" and "--no-undefined" were passed in two
separate "-Wl," compiler arguments, but they reflect a common issue.
* Add missing license header
* Rename more functions
* Better name for global variable
* Rename methods of Package
* Rename functions in msan_builder.py
* Fix invalid variable names
* Fix useless-object-inheritance
* pylint: Fixes for Package and its subclasses
* Remove unused imports
* Indentation fixes
* Fix too-may-locals error in msan_build.py
* Add missing docstrings
* Good fuzz target for golang coverage with modules
* Place target in right directory for go-json-iterator
So that coverage gets access to the right package
* Golang coverage summary for each fuzz target
* Document usage of compile_go_fuzzer
* update the documentation change
Co-authored-by: Max Moroz <mmoroz@chromium.org>
This will help us catch breaking changes to the base-images.
Unfortunately caching seems to fail here when I expect it to help.
For example, base-builder doesn't build from cache when I do it locally.
This means that every other image I try to build doesn't use the cache.
That means that base-clang would take forever to rebuild.
So to compromise, I don't rebuild base-clang here.
This means that this PR won't catch breaking changes to base-image or base-clang that break in base-builder.
But it will catch breaking changes to base-image that break in base-runner and it will catch breaking changes to base-runner and base-builder.
[CI] Build a canary project on infra/ changes.
Build a specific project, sckms, that does msan, ubsan, asan, i386
builds quickly, when infra/ code is changed. This can let us know
when infra/ changes break proper functioning of OSS-Fuzz.
For this to work more thoroughly we also need to rebuild images.
Abhishek Arya