Commit Graph

18 Commits

Author SHA1 Message Date
Oliver Chang e8646d25ec
Fix bisection when the original checkout is for a single branch. (#5478)
Repos cloned with `--branch BRANCH` will only track that branch, even
when we unshallow. If we provide a git SHA from another branch, it will
not be recognized.

To fix, this, we update the remote tracking config and fetch them.

For google/osv#88.
2021-03-23 14:17:37 +11:00
jonathanmetzman d7e85a20b0
[CIFuzz] Fix diffing + Refactor (#5032)
* [CIFuzz] Fix diffing + Refactor

Make diffing work in two scenarios where it previously failed:
1. Commit fuzzing. In this case, we diff $COMMIT against
$COMMIT^1 because the intent here is to fuzz the commit.
2. Fuzzing PRs that aren't to master. In this case, we previously
were diffing against origin/master. Instead, diff against the local
version of the base repo. This also has the nice effect of handling
PRs that havent pulled from master recently enough.

Also do refactoring.
1. Move code that is different for differenct CI systems to continuous_integration.py
2. Change how configuration in build_fuzzers is handled.
Previously configuration was gotten in build_fuzzers_entrypoint
and passed as individual params. This made code ugly and hard to
read. Instead, move code dealing with config to it's own module
config_utils. This module implements a config class which can
be used to create objects that are passed around to code that needs
it. Making the code much easier to read.
TODO: Move run_fuzzers code to new config system.
2021-01-28 12:10:57 -08:00
Oliver Chang 0a7813d978
repo_manager: Allow unshallow to return non-zero. (#4963)
Unshallow can fail in a submodule, even if the unshallow succeeded for
the main repo.
2021-01-14 14:16:20 +11:00
jonathanmetzman b0b99d5ccd
Cifuzz external build (#4656)
* Support building fuzzers for projects outside of OSS-Fuzz
* Use retry wrapper
* Fix some tests.
2020-12-07 10:50:11 -08:00
Oliver Chang 662260da6e
bisector: try older commits if old_commit has the same result as new_commit (#4096) 2020-07-09 20:04:13 +10:00
Oliver Chang 47068dff0a
Use base-builder close to commit date. (#3715) 2020-04-28 11:31:02 +10:00
Oliver Chang ed292c0b40
Bisector: Be a bit smarter about picking which OSS-Fuzz commit to build with. (#3665)
When the build fails against HEAD OSS-Fuzz, we find the date of the
commit for the project, and use the latest revision of OSS-Fuzz before
it to rebuild the project builder container.

Subsequent runs will use the last built container, and if that fails
that will again find the closest revision of OSS-Fuzz.

Also factor BaseRepoManager out of RepoManager to provide a generic repo
manager class for dealing with existing checkouts (which don't need a
clone).
2020-04-21 10:11:29 +10:00
Oliver Chang 3d6a6f546b
bisector: Don't clean when checking out commits. (#3652) 2020-04-15 16:17:13 +10:00
Oliver Chang 0399965d7e
Allow bisecting with no old commit. (#3641)
Fixes #3598.
2020-04-15 08:59:33 +10:00
Oliver Chang aa045b9b37
Bisector fixes. (#3601)
- Copy /src from host instead of checking out repo on host. This fixes
  issues with dependencies in the main repo.
- Add some more logging.
2020-04-14 11:38:23 +10:00
Leo Neat f798890b66
[Infra] Add git diff to repo manager class (#3427)
This adds the functionality of git diff --name-only to the repo manager module. This functionality will be used for the affected fuzzer feature of CIFuzz. Associated tests have also been added.
2020-02-28 08:30:42 -08:00
Leo Neat 52e640af34
[CIFuzz] Fix .strip bug. (#3349) 2020-02-06 09:33:54 -08:00
Leo Neat 53cb4f7935
[Infra] Update execute function in utils.py (#3319) 2020-02-04 11:51:18 -08:00
Leo Neat 1a87da68c8
[cifuzz] - commit and pull request hook functionality (#3310) 2020-01-31 15:31:18 -08:00
Leo Neat f2588ea5a8
Moving execute to utils.py (#3313) 2020-01-31 10:19:12 -08:00
Leo Neat 8ffc6db00c
[Infra] CIFuzz pipeline complete. (#3281)
* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Testing action build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working build

* Working fuzzers with out error surface

* Working fuzzers with out error surface

* Working fuzzers with out error surface

* Printing std err

* Adding fuzzer timeout

* Adding fuzzer timeout

* Changing fuzzer timeout to fuzz time

* Formatting and refactoring

* Spelling in fuzz_target.py

* Spelling in fuzz_target.py

* Spelling in fuzz_target.py

* Upload artifact fix

* Upload artifact fix

* Upload artifact fix

* Upload artifact fix

* Upload artifact fix

* Upload artifact fix

* Upload artifact fix

* Refactoring error codes.

* reverting helper.py

* reverting helper.py

* reverting helper.py

* chaning method to static

* moving cifuzz file

* Jonathan changes

* Oliver and Jonathan comments

* Oliver and Jonathan comments

* Oliver and Jonathan comments

* Utils unit tests

* Test formatting and documentation

* Build fuzzer test added

* Changed repo manager errors

* Unit and integration tests complete

* Jonathan comments pt.1

* Jonathan comments pt.1

* Jonathan comments pt.1

* adding cifuzz_test

* Build fuzzer test completed

* Run fuzzers test finished.

* Removed SRC dependency

* Jonathan comments pt.2

* Max comments pt.1

* Max comments pt.2

* removing log specified out stream

* Max comments pt.3

* Adding OSS_FUZZ_HOME env var

* Jonathan comments pt.3

* Formatting

* Olivers comments

* Jonathan comments
2020-01-29 11:03:43 -08:00
Leo Neat 9532d4781a [infra] Fixing bisection to work with more OSS-Fuzz projects (#3152)
* Max comments

* Added bisection tests

* Added test cases

* Docker run command added to build_specific commit

* Infer main repo uses docker image rather than docker file

* Added mores tests for repo infer

* With verbosity

* Formatting

* Bisection fixed

* Formatting updates

* Oliver small comments.

* Script infer main repo

* Detect repo and test module finished

* Detect main repo from script

* Regex searching

* Regex searching

* Max comments + bisector test script

* Oliver comments

* Max's comments

* String concat update

* Jonathan comments
2020-01-06 12:17:26 -08:00
Leo Neat 2c05fb229f [infra] Add support for Bisecting Git Commits Introducing Crashes(#3119) 2019-12-16 16:48:49 -08:00