Commit Graph

101 Commits

Author SHA1 Message Date
mpl 74bb79e63b buildbot: broke into a master and a builder bot
This change is the first major step towards more improvements to the
buildbot. It cleans up various details, but most notably:
-it now uses a task as the basic data structure, to improve readability
and maintainability
-it breaks the bot into a master bot and a builder bot.
-it allows a master to receive reports from remote bots, and the
status page now aggregates all those reports.
-it allows a builder bot to send its reports to several master bots.
-the master bot uses whatever go version is installed to build the
builder bot, or the downloaded go tip if none is found.

The master bot is in charge of
1) monitoring the changes to the Camlistore and Go repos,
2) recompiling builder bot, which allows to add some new tests to the
test suite without having to interrupt the master bot.
3) starting a builder bot instance
4) waiting for the builder bot report and/or polling the builder bot
for its progress
5) answering the requests to display the various status pages.

http://camlistore.org/issue/185

Change-Id: I46a0b8fabbebf76b0c3ed04fd2ee73362d565cf7
2013-10-18 19:32:24 +02:00
Bill Thiede 0712432e1f buildbot: use ringBuffer to store last 1MB of logs
Change-Id: I689ca82c64b9c7f89913551c5138a78718377c0d
2013-10-14 17:20:36 -07:00
Brad Fitzpatrick 652719fd3f Merge "buildbot: Add more debugging." 2013-09-22 10:33:49 +00:00
Bill Thiede 570ea2a82d buildbot: Add more debugging.
This adds the error returned from exec to the log message, and
provides '(empty)' placeholders when no stdout or stderr is present
from the called binary.

This has the effect of changing the silent errors I was seeing on my
buildbot from:

"""
Error:
	:
"""

To:

"""
Error:

	exec: "devcam": executable file not found in $PATH

Stdout:
(empty)

Stderr:
(empty)
"""

Also provides a /stderr handler linked from the status page that
allows all the logs to be viewed.  This is handy when the logs from an
individual test don't provide enough info to debug a build problem.

Change-Id: I4b3ef5a5db6aac1ddf41c41bc2fff310f19a82a1
2013-09-21 12:04:39 -07:00
Brad Fitzpatrick cfc9ce026b Add 0.5 to misc/release-history-tags
Change-Id: Id5756cd120ebd2548ac314f34e5f0095280fd71f
2013-09-21 19:57:49 +01:00
mpl 0c1dc0d6e4 devcam test: generate keyblob with devcam put init
The integration tests run by devcam test require a
GPG keyblob in the default location (for camput), and
we do not track in git such a keyblob.
This change allows devcam test to create the keyblob
with devcam put init before running the tests.

Also, devcam server is now using CAMLI_KEYID instead
of hardcoding the key id in the dev server config file.

Change-Id: I030b88c9a9321016fedb62424e0e707621206922
2013-09-20 20:03:21 +02:00
mpl ba8f88c21a devcam test, replaces make presubmit
http://camlistore.org/issue/223

Change-Id: Iaaea9926a98d7d3335fd6ee52daa225e51ec4703
2013-09-19 17:00:59 +02:00
mpl 3a26c433db devcam: 'put' and 'get' commands, replace dev-camput and dev-camget
Also added -port flag, and untied the port to the
password in devcam server, always 'pass3179' now.

http://camlistore.org/issue/212

Change-Id: Iadf693951763d47907be17c53d4807a4706eb150
2013-09-01 10:25:09 -07:00
Brad Fitzpatrick f2dd7c1328 Add 0.4 to history
Change-Id: Idaf22fd681106349c693fc49a7bc5499212eb17c
2013-08-26 18:06:54 -05:00
mpl b3b16f73c2 buildbot: use make.go
Change-Id: I76544d7e0601be9a5bc2b9378a29db6c97894d76
2013-08-21 19:33:55 +02:00
Bill Thiede 502aff1fd5 camweb,buildbot: Update camlistore git repo URLs.
Also add redirect for /code/ request with a commit hash to new code host.

Change-Id: I68a8746dacb6f0a4fc6719e3f5ed23ee0f37778b
2013-08-18 08:56:16 -07:00
Brad Fitzpatrick b6a8dc3351 More git links updated
Change-Id: I2e527d73c2907dbc87e84a876b0f0ba7b2cf18b9
2013-08-14 10:47:56 -07:00
Brad Fitzpatrick 36f93b8240 Updating website for new git & Gerrit location.
Change-Id: Ifaceae9e40e2028c7c643020ea82abcece2e3554
2013-08-14 10:35:33 -07:00
mpl 66bc4f35d6 Merge "dev-server: remove. replaced by devcam server." 2013-08-03 16:34:35 +00:00
mpl f4ef8c0375 dev-server: remove. replaced by devcam server.
Fix related docs, config files, buildbot...

Change-Id: I638d3856f42953603b13204fb56782974d0db9ee
2013-08-03 01:12:38 +02:00
Brad Fitzpatrick 979444de66 dev-camput, dev-camget: options to make demos better
Change-Id: Ie109cc156a081b72b83b9851630addf515c1ceec
2013-07-29 10:15:59 -07:00
Brad Fitzpatrick 30682116cf Add 0.3 release.
Change-Id: Iabd93c3f92f18c737edb818b66a449804527bfa3
2013-07-28 18:47:01 -07:00
mpl 5645570ca9 dev-server: use updatelibrary.go to update closure lib
This change also removes misc/get_closure.pl and updates
the Makefile accordingly.

http://camlistore.org/issue/140

Change-Id: I508271cee24eb20d8391cb3d84e1201ce3ef567e
2013-07-23 16:19:46 +02:00
mpl 93e9ad812d buildbot: properly answer if-modified-since requests
http://camlistore.org/issue/119

Change-Id: If5b486bf95748267e33f354e0a8fed9c79f220c0
2013-07-17 19:13:07 +02:00
Brad Fitzpatrick 466dfcdc59 buildbot: add doc comment
Change-Id: Iaa713039f58ea607db179b99638971f8e194e8b0
2013-07-07 15:32:11 -07:00
Brad Fitzpatrick f7707d75c1 Remove misc/sqlite directory. Old experiment.
Change-Id: I207c96dc77221fddeebb8ecdf1bc61c15269ef16
2013-07-05 09:55:11 -07:00
Brad Fitzpatrick 59f5b41900 Use make.go from dev-* scripts. Updates Issue 140.
Change-Id: I0acea921e311b8228829dbad37ee502034c7dfe7
2013-06-29 13:52:01 -07:00
Brad Fitzpatrick c86b1bafdf Add 0.2 release tag.
Change-Id: I42eb5ae160814865e6f8139e50978a1a96178007
2013-06-22 07:52:25 -07:00
mpl 2430cf7a20 camlibot: run without --staticres --offline since they are gone
Change-Id: I14d696e2f07967719fa82e4782d8a4ed139b411e
2013-06-19 16:12:52 +02:00
mpl 7efc0d131e camlibot: run in staticres mode until closureRoot works
Change-Id: I24252453e1935d9e7308512c705f8ca6e3fda01b
2013-06-14 00:16:35 +02:00
mpl 40b4485b9e buildbot: actually print output when server fails to start
Makes it way easier for debugging.

Change-Id: Ic71c78134664cb512d1ae2b14eb994724e367ef9
2013-06-14 00:08:42 +02:00
mpl 25776d90bb buildbot: quit hitting /ui/new/
Change-Id: I2e5c443ee90e299516cab6067ced8fbcde4fe257
2013-06-11 21:34:14 +02:00
Brad Fitzpatrick 6b7e37f4a8 Add misc/release-history-tags
Change-Id: Ieb53dc2ebaa6b2619416fdc9a45b265a9fc90b2b
2013-06-11 17:18:28 +02:00
Josh Bleecher Snyder 224e25d8cc Stop using $USER in misc/review
For folks whose $USER doesn't match their
Gerrit username, editing misc/review is not a
great solution, since the changes cannot be
committed (nf notwithstanding).

Instead, let ssh config files do their thing;
these are not part of the repo, and thus can
be set once and merrily forgotten.

Change-Id: If03fe05e5779553ed1f98310e8f22b0635ada7bb
2013-06-04 17:30:23 -07:00
mpl d9649744b9 buildbot: more idiomatic chan timeout, and chan reuse.
Change-Id: I3b4a1e69c6b2c3d3889dbb62067d53cebf4eb8aa
2013-06-04 15:16:02 +02:00
mpl 5b73925c2a pre-commit hook: skip deleted files
http://camlistore.org/issues/74

Change-Id: Ie1cdeca8d22d0a41ced2a3a3d9bd7bc4b6c89951
2013-04-25 19:25:32 +02:00
mpl ea3e16adf9 Merge "a few camlibot changes" 2013-03-07 21:00:55 +00:00
mpl 6e5ee35085 precommit: check trailing space in java sources too
Change-Id: I85820763b6c1075f5c8ff5d793b082bd11181931
2013-02-25 19:28:06 +01:00
Jani Monoses 48838c6651 Use git instead of svn for fetching closure.
Change-Id: I3fc4af0f17d59aa951bdd59176fcbce5072226cf
2013-02-13 11:38:16 +02:00
mpl 7cb62ad6f5 a few camlibot changes
1) Use "make forcefull". Please note that this will fail
with the default Go1 version installed by ubuntu packages.
2) Use --offline with dev-server to avoid pulling the
closure stuff.
3) Waterfall display (new stuff on top, older at bottom).
4) Added a camput -vivify run, with the subsequent camgets
to check that the file has indeed been pushed.

Change-Id: Icd164932bd61d54ddbece12ef420524643e07f1b
2013-02-07 16:16:15 +01:00
Brad Fitzpatrick e398532b4a buildinfo: include dirty + flag in version
Change-Id: Idc4763de41fdfa055fed5c06a36db9e9539f60bc
2013-02-03 10:14:21 -08:00
mpl fc8ecb3b29 closure: changed the library setup, added the compiler
1) The code to get the closure library has been moved from
dev-server to misc/get_closure.pl
2) The svn checkout version for the closure library was bumped
to the latest: 2459
3) The closure library checkout now goes into tmp/closure-library/,
to keep tmp/ clean for other things we might want to add there
(like the compiler). So the path to the actual libraries is now
tmp/closure-library/closure/
4) The code to get the closure compiler build was added in
misc/get_closure.pl. The compiler goes to
tmp/closure-compiler/compiler.jar
5) The version of the closure compiler is the latest build as
of this time: 20121212

Change-Id: I7e03ad7d6fe440430c5727cf56a956efed360d43
2013-01-25 17:13:19 +01:00
mpl e28418946c camlibot: use make full
Change-Id: If4a8e3ea7add16137263d25c7d9540d19e0116bd
2013-01-22 01:55:45 +01:00
Brad Fitzpatrick fb24b240cc dev-server: more verbose, build camdbinit lazily (speeds start-up)
Change-Id: I0a5dc255efd2cfcde7ff735a79ceeeb4b9965b08
2013-01-20 12:33:35 -08:00
mpl b8a8e86012 buildbot: now hits on http://localhost:3179/ui/new/ since home.html gone
Change-Id: I58d2f5b7fc06fc326e08d95daa69a1e6e5bd2753
2012-12-24 02:14:06 +01:00
mpl 2d094d4ea5 camlibot: print stdout as well on failure page
Change-Id: Iabe2734fb6013c62a19ead5a2232975a445dac32
2012-12-23 23:56:58 +01:00
mpl f59fe1d535 Merge "buildbot: now with history and prettier output" 2012-12-20 13:35:20 +00:00
mpl f45e6f5508 buildbot: now with history and prettier output
Change-Id: I863a5899024a74930bfa534f2ce79bcd083326fe
2012-12-19 15:49:56 +01:00
Brad Fitzpatrick c966fe8c12 Remove old cgo tests
Change-Id: Iebdfadd483e858b9127342fc35f21ea7da69da80
2012-12-16 20:43:48 -08:00
mpl 489068e277 buildbot
Change-Id: I6ae90b2ea75af6d0ca45905a5f2316cc94953b9e
2012-12-07 22:19:44 +01:00
mpl c2b746a086 pre-commit hook: warn about broken third party imports
Change-Id: Ib1315af2833a757093b36127b62d64abbd055963
2012-10-24 13:09:06 +02:00
Brad Fitzpatrick 3c80cc17f8 Move old broken dev-* scripts away.
Change-Id: Ide5dfe2dffc5d1af8f1c88be0a897c97932eb14c
2012-10-19 13:38:55 -07:00
mpl 4a184c84d5 pre-commit git hook:
-gofmt
-trailing space check

Change-Id: Icd63483b1873b21cd5ff6f00fff4b191095942ad
2012-10-16 18:18:46 +02:00
Brad Fitzpatrick e0d10d941f hack 2012-08-05 10:49:20 +10:00
Brad Fitzpatrick 1b63c44b4e Update all os.ENOENT references to ErrNotExist. Mostly docs/dead code.
Change-Id: I2bb4d5a9d9c610e8dce2309335d06046dbb1e049
2012-05-13 12:06:21 -07:00