Instead of recompiling libFuzzer each time we do a libFuzzer
build of a project, always use Clang's builtin version of libFuzzer.
Do this by copying the builtin libFuzzer to /usr/local/lib/FuzzingEngine.a.
This means that the projects that aren't using -fsanitize=fuzzer now also
use the builtin libFuzzer. And we no longer need to compile a sanitized
libFuzzer for them.
This change improves fuzzing performance and developer experience.
1. It improves developer experience by saving time spent compiling libFuzzer
when recompiling fuzzers.
The time saved is about 25 seconds on my machine.
This will make iterating on fuzzer integration much easier.
2. It improves fuzzer performance. The builtin libFuzzer isn't sanitized so it is faster.
In some cases (see [here](https://bugs.chromium.org/p/chromium/issues/detail?id=934639))
sanitized libFuzzers can waste 37% of the time running non-performant implementations
of code that the builtin-libFuzzer can do almost instantaneously (assembly vs C code).
The consequences of improving developer experience and
fuzzer performance aren't so easy to measure (though
we will look for perf consequences on ClusterFuzz).
But some of the consequences of saving time compiling libFuzzer
are easy to figure out and quite important. They are:
1. Saving $14646 a year on build costs. Based on the following:
build time saved (on GCB): ~38 seconds
libFuzzer builds per day: 990
builds per year: >365
price per build-minute (32 core instance, https://cloud.google.com/build/pricing): 0.064
38/60*.064*990*365 = 14,646
2. Speeding up infra-tests.
Many of the integration tests build fuzzers and so building libFuzzer
was a considerable bottleneck.
On my many-core machine the savings were good and noticeable
(and are probably larger on the less performant CI machines).
| | With compiling libfuzzer | Without compiling libfuzzer |
| ---------------------- | ------------------------------- | ----------------------------------- |
| Parallel tests | 45 | 34 |
| Sequential tests | 276 | 190 |
3. Speeding up CIFuzz.
CIFuzz needs to be fast but it spends about 40 seconds compiling libFuzzer.
In a run where no bugs are discovered which is intended to take about 20 minutes
compiling libFuzzer takes about 3% of the time (40/(20*60)*100).
Now we don't need to waste that time.
See https://github.com/google/oss-fuzz/issues/5180, which this partially fixes.
This bug fixes https://github.com/google/oss-fuzz/issues/2312 and https://github.com/google/oss-fuzz/issues/4677.
* [infra] Compile fuzzing engine without SANITIZER_FLAGS unless MSan is used.
* Use an additional FUZZING_ENGINE_SANITIZER_FLAGS variable and simplify the script
* Use $FUZZING_ENGINE_SANITIZER_FLAGS in compile_libfuzzer.
jonathanmetzman