All latest builds seem to be broken with:
```
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
INFO: Loaded 112 hooks from com.code_intelligence.jazzer.runtime.TraceCmpHooks
INFO: Loaded 4 hooks from com.code_intelligence.jazzer.runtime.TraceDivHooks
INFO: Loaded 2 hooks from com.code_intelligence.jazzer.runtime.TraceIndirHooks
INFO: Loaded 4 hooks from com.code_intelligence.jazzer.runtime.NativeLibHooks
INFO: Loaded 8 hooks from com.code_intelligence.jazzer.sanitizers.Deserialization
INFO: Loaded 5 hooks from com.code_intelligence.jazzer.sanitizers.ExpressionLanguageInjection
INFO: Loaded 70 hooks from com.code_intelligence.jazzer.sanitizers.LdapInjection
INFO: Loaded 46 hooks from com.code_intelligence.jazzer.sanitizers.NamingContextLookup
INFO: Loaded 1 hooks from com.code_intelligence.jazzer.sanitizers.OsCommandInjection
INFO: Loaded 52 hooks from com.code_intelligence.jazzer.sanitizers.ReflectiveCall
INFO: Loaded 8 hooks from com.code_intelligence.jazzer.sanitizers.RegexInjection
Exception in thread "main" java.lang.ExceptionInInitializerError
at com.code_intelligence.jazzer.sanitizers.RegexRoadblocks.<clinit>(RegexRoadblocks.java:72)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:398)
at com.code_intelligence.jazzer.instrumentor.Hooks$Companion$HooksLoader.loadHooks(Hooks.kt:66)
at com.code_intelligence.jazzer.instrumentor.Hooks$Companion$HooksLoader.load(Hooks.kt:48)
at com.code_intelligence.jazzer.instrumentor.Hooks$Companion.loadHooks(Hooks.kt:42)
at com.code_intelligence.jazzer.agent.Agent.premain(Agent.kt:137)
at com.code_intelligence.jazzer.driver.Driver.start(Driver.java:97)
Caused by: java.lang.NumberFormatException: For input string: "./jazzer_driver"
at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:66)
at java.base/java.lang.Long.parseLong(Long.java:680)
at java.base/java.lang.Integer.parseUnsignedInt(Integer.java:835)
at java.base/java.lang.Integer.parseUnsignedInt(Integer.java:929)
at com.code_intelligence.jazzer.api.Jazzer.getLibFuzzerSeed(Jazzer.java:635)
at com.code_intelligence.jazzer.api.Jazzer.<clinit>(Jazzer.java:38)
... 8 more
```
native go fuzzing: Restrict fuzzer lookup to *.go
Projects containing words starting with Fuzz in non-go
files break compile_native_go_fuzzer with the error:
sed: -e expression #1, char 15: unknown option to 's'
Signed-off-by: Paulo Gomes <pjbgf@linux.com>
Signed-off-by: Paulo Gomes <pjbgf@linux.com>
* infra: have timeout per fuzz target for coverage
As is done for other languages
* ngolo-fuzzing: remove temporary workaround
now that https://github.com/golang/go/issues/53190 is closed
* ngolo-fuzzing: use built go toolchain in its directory
without copying it to /root/.go/
in order to get coverage for std lib in the end
* infra: ability to get coverage for additional golang package
And uses it with ngolo-fuzzing :
ngolo-fuzzing fuzz targets live in a different repository than
the code being fuzzed, and we we want to get the coverage, for
both the fuzz target and the package being fuzzed
* fixup bash unbound
* fixup ngolo-fuzzing only match at beginning for std package
* stricter check for every additional go package
* infra: fix per-target coverage html generation
When generating coverage reports for single targets we should not
include all fuzz targets (in the $objects var), but only the object
(binary file) for the given target.
Ref: https://github.com/ossf/fuzz-introspector/issues/340
* coverage: remove potential race condition
to make it easier to catch issues like
https://github.com/google/honggfuzz/issues/465.
With -V when unreproducible crashes are triggered honggfuzz shows something like
```
[2022-06-10T15:21:12+0000][W][2373157] subproc_checkTimeLimit():532 pid=2374490 took too much time (limit 30 s). Killing it with SIGKILL
Sz:280 Tm:30,021,052us (i/b/h/e/p/c) New:0/0/0/0/0/3, Cur:0/0/0/0/0/591
Launching verifier for HASH: 1b859677dd (iteration: 1 out of 5)
[2022-06-10T15:21:13+0000][E][2373157] fuzz_runVerifier():280 Verifier stack mismatch: (original) 1b859677dd != (new) 0
```
It should make it easier to figure out why exactly
`./infra/helper.py check_build project fuzz-target` fails
by turning
```
INFO: performing bad build checks for /tmp/not-out/tmpa4lph9dr/fuzz-bus-message
ERROR:root:Check build failed.
```
into something like
```
INFO: performing bad build checks for /tmp/not-out/tmpa4lph9dr/fuzz-bus-message
BAD BUILD: /tmp/not-out/tmpa4lph9dr/fuzz-bus-message seems to have either startup crash or exit:
/tmp/not-out/tmpa4lph9dr/fuzz-bus-message -rss_limit_mb=2560 -timeout=25 -seed=1337 -runs=4 < /dev/null
/tmp/not-out/tmpa4lph9dr/fuzz-bus-message: error while loading shared libraries: libcap.so.2: cannot open shared object file: No such file or directory
ERROR:root:Check build failed.
```
* infra: use only valid targets for python coverage
Some python coverage reports gets clobbered because non-fuzz targets are
included in FUZZ_TARGETS. The consequence of this is that some reports
will have erroneous data. This commit ensures only valid targets are
used when running coverage, which consequently fixes the reports.
* Move executable logic into python fuzzer builder instead
The current implementation will append rather than overwrite
coverage_wrapper.py which holds the coverage stub. The effect of the
appending is that the coverage_wrapper.py will include X amount of
coverage stubs when a project has X amount of fuzzers. We just need a
single coverage stub at the top of each fuzzer. This ensure we only add
a single coverage stub.
The current run_fuzzer will fail when multiple libfuzzer options are
specificed in a .options file. Currently, if there are multiple options
then none of the options will be used, but rather an issue about a
"binary operator expected" will occur. One implication of this is that
check_build fails to use it and may, therefore, report erroneously. This
fixes it by enabling the use of zero, one and many libfuzzer options.
* infra: fix python coverage
The python coverage has not started working on oss-fuzz.com yet. This is
an effort to fix this.
* add logs and stats files
* log stderr too
* infra: enable argument passing to compile_python_fuzzer
Enable passing arguments to pyinstaller. This is used by, e.g. Django.
Ref:
0cb820e5af (commitcomment-72063587)
* only unzip real files
* Roll AFL++
* Adapt to us-central1
* fix
* test faster
* All projects
* all
* All
* fix
* start small
* bigger
* print dbg
* handle sanitizers
* handle sanitizers
* all means all
* handle flake
* Fix
* fix
Main changes in fuzz-introspector:
- reduce logging in fuzz-introspector as some oss-fuzz build logs are
huge.
- insert links in the html reports to the newly added doc.
This adds ci.py a script that can be used to test changes to the base-images.
The script does the following:
1. Builds and pushes a test image for each base image. This will be used in step 3.
2. Finds which projects are buildable based on user specifications of: projects, build type (e.g. sanitizer/fuzzing engine) and which of these succeeded in production.
3. Submits builds for those projects.
4. Polls GCB to check which builds succeeded and which failed.
5. Returns 0 if all builds succeeded.
Currently this script is only useful to run locally.
Related: https://github.com/google/oss-fuzz/issues/7145
1. Reduce image size by 1.6GB by deleting unneeded directory containing
swift tools that were already copied to /usr.
2. Use apt-get instead of apt to avoid warnings.
Both the JaCoCo .exec file and the XML report are very useful for
automated analysis of coverage data and IDE integration, so serve them
just like the HTML report.
The patched Go versions adds a new runtime function to register
8bit counters when initializing the fuzz target. This is needed to
register those counters with libfuzzer and get real edge coverage.
This change is meant to be a temporary change until the Go patches are
accepted upstream. Then, we will create a pull request for
go114-fuzz-build
- Point `compile` to the right `fuzz-introspector` location (since it's no longer checked out in $SRC).
- Rename build tag to "introspector" to be more consistent with other
tags.
- Fix bad merge in deploy.sh script.
- Add introspector setup to project sync.
- Enable more logging for project sync cron.
* Clean up build infra issues.
0. Don't clone fuzz-introspector to /src since it breaks builds.
1. Install packages in python install script properly. Previously
pip install atheris>=2.0.6 was interpreted as "redirect the output
from pip install atheris" to the file "=2.0.6".
3. Clean up some miscellanious issues.
* Include removal
jonathanmetzman