Commit Graph

5288 Commits

Author SHA1 Message Date
mpl e3f16b8ead pkg/deploy/gce: add web wizard to deploy on Google Cloud
Issue #531

Change-Id: I47900eb7b82b7775ec1d26b1d6796e5701ee672d
2015-01-19 16:24:37 +01:00
Brad Fitzpatrick 3026631521 serverinit: move incompatible option checks further down, into the right places
I was getting bogus errors because I replicate to s3, but the
s3+blobpacked error should only come up if s3 is primary.

Also, document the database key used for blobpack's index when mysql
or another db is in use.

Change-Id: I21a1815715f70edc3f98abf7209744703aaf6d3b
2015-01-18 19:04:31 -08:00
Brad Fitzpatrick 81bca97523 localdisk: auto-create packed directory on start-up
Change-Id: I2e3453929c9f55b0fe22847699118876e4d2640e
2015-01-18 19:02:42 -08:00
Brad Fitzpatrick 9bea14733d importer: support for disabling importers at start-up
Change-Id: Ia2fc81f496ed020669a1c6ede489197d4a52ae89
2015-01-18 19:02:02 -08:00
Brad Fitzpatrick 4155ac6a3c serverinit: high-level config support for using blobpacked
Users need to set:

     "packRelated": true,

... in their server config. This will probably become the default
value in the future.

This is currently mutually exclusive with diskpacked, which isn't good
at deleting things, and blobpacked loves to delete things (from the
loose blobs).

Updates #532

Change-Id: I8f4ea9406859b2705f26e9d1103d3acf9d1a8411
2015-01-18 18:08:18 -08:00
Brad Fitzpatrick 098ec2d70c server/camlistored: link in blobpacked
Update #532

Change-Id: I66513fda7cde57c8c3a0aa7e5213826400b659f0
2015-01-18 18:03:44 -08:00
Brad Fitzpatrick e0624177cd search: add blobref sorting, implementing sorting of results when source is unsorted
Part of upcoming blobpacked migration tool.

Change-Id: Ifdc4723d4d1468eb4d2798fadb256b409cfe0555
2015-01-17 20:31:41 -08:00
Brad Fitzpatrick a99dc14885 cmd/camtool: add flag to send a raw SearchQuery; permits sending a sort field
Part of upcoming blobpacked migration tool.

Change-Id: If0e91aa90aab78936976e8b0d6b3bfe2e5a4c27a
2015-01-17 20:29:05 -08:00
Brad Fitzpatrick 768b244f92 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2015-01-17 13:01:05 -08:00
Brad Fitzpatrick 5f041a3b6a Update github.com/bradfitz/http2 to f0f7876b264 in third_party.
Includes:

commit f0f7876b264515bf0c86c37d78ab558d939b82b0
Author: Brad Fitzpatrick <brad@danga.com>
Date:   Sat Jan 17 12:25:12 2015 -0800

    Don't enforce SNI for now.

    It was causing problems in development.

commit 36f7934bd38f22905a9eba016885fe04856ca4d1
Author: Brad Fitzpatrick <brad@danga.com>
Date:   Sat Jan 17 12:24:34 2015 -0800

    Be more verbose when rejecting connections; including which bogus ciphersuite.

commit 76847809286eae2e2f066cd3f88c8bc960a08dbd
Author: Brad Fitzpatrick <brad@danga.com>
Date:   Sat Jan 17 12:23:24 2015 -0800

    In curl integration test, include output along with non-zero exit status in failure message.

    Change-Id: I5bc392586ba54fc83bc5e54b7b4c27ef25530e4c

commit af958b3ad4d6dd0ffe9599ea2351653ed45c2fe5
Author: Brad Fitzpatrick <brad@danga.com>
Date:   Sat Jan 17 12:21:49 2015 -0800

    Update curl in integration test Dockerfile to 7.40.0

    And pin the nghttp2 version, and to something more modern.

    Change-Id: Ie631b519b9ec9278b521598353730d1cfd4f8976

Change-Id: I2ec8bc049e27e63dce07cabe3529f96123181e40
2015-01-17 13:00:15 -08:00
Tamás Gulácsi a6329fa8b8 pkg/app: use netutil.HostPort for ListenAddress()
As TODO said, so we gain IPv6 support.

Change-Id: Ia40824481e47a15dfdb376be624dde20c92e3af5
2015-01-17 18:57:06 +01:00
Brad Fitzpatrick ab9cd80301 httputil: use a random serial number in self-signed certs, not 0
Change-Id: Ifdba7c3efb9bd08e09da883eecdf3ba36dfa3f12
2015-01-16 16:46:15 -08:00
mpl 00b18d99d4 misc/docker: rm spdy proxy since we have http2
issue #534

Change-Id: Ic05b60aa1c220b9df76718edc2ca222cddc427dc
2015-01-14 15:46:18 +01:00
mpl 3f0874eb88 Merge "third_party/github.com/bradfitz/http2: update to 7882a790a7e1a06d865147c791cbf1ac8dc197c5" 2015-01-13 14:05:26 +00:00
mpl a27306d74a third_party/github.com/bradfitz/http2: update to 7882a790a7e1a06d865147c791cbf1ac8dc197c5
issue #533

Change-Id: Id3e32e60565502c77c63a13821cd1084198d1617
2015-01-13 15:02:48 +01:00
mpl cb492fdcc7 Merge "pkg/deploy/gce: revert to hardcoded google endpoints for oauth2" 2015-01-13 00:53:02 +00:00
mpl 1a286f919f pkg/deploy/gce: revert to hardcoded google endpoints for oauth2
google.Endpoint is the only thing we were using from
third_party/golang.org/x/oauth2/google
And pkg google would in turn bring in lots of dependencies (that I had
forgotten in 22f45f14a5).
Replacing Endpoint with the hardcoded values hence allows us to not
vendor in all those dependencies.

Also fix make.go to build camdeploy as well.

Change-Id: I6ba51991187a11705c97c9546848303540f99aa1
2015-01-13 01:28:09 +01:00
Mathieu Lonjaret a6981ff97b Merge "client: log when GPG signatures mismatch" 2015-01-12 21:10:16 +00:00
Salmān Aljammāz 8bee1495ab client: log when GPG signatures mismatch
Mismatching GPG keys seems to be a very common issue for newcomers.

Change-Id: If83eee5d0ba5f0c85a4d75a31ab15a5469c2965d
2015-01-12 14:06:22 +03:00
mpl 22f45f14a5 Add new oauth2, use it in deploy/gce
third_party/golang.org/x/net/context at 67f25490433376b5f7da086c0b9283fcdeca3a7b
third_party/golang.org/x/oauth2 at 5361962df4497a7fc777f5c3bd1b5c7ac36c604c

I only updated camdeploy, and did not remove code.google.com/p/goauth2
yet, because I do not want to get distracted by potential breakages, so
we can first get the gce deploy wizard in.

Change-Id: Ifa7987a044c741d3f00f49f47b31e3e07bdf973d
2015-01-09 22:02:39 +01:00
Aaron Boodman bc5fd6ea9d Remove no longer used permanode.js
Change-Id: I02a723ac81957e9f5438ab27399974554e7c39c7
2015-01-04 22:52:32 -08:00
Aaron Boodman ea2c8fa990 Add a list of references to a blob to the blob aspect.
Change-Id: I2fe96bc8f04f8137773aee393eb934e06517575d
2015-01-04 21:59:10 -08:00
Aaron Boodman e40504a9b2 Improve heuristic for determining when to show search aspect.
We now show it when either there are existing children, or when
the permanode doesn't have a more specific type.

Change-Id: Ic90829a60bf5f6213afd53476473eed4ffee6ca8
2015-01-02 23:10:45 -08:00
Aaron Boodman 437b8f99e1 Factor out cam.permanodeUtils.getCamliNodeType
Change-Id: I205c1551fd316d9a844cd3fe4041d6e2cf0845b9
2015-01-02 23:10:43 -08:00
Aaron Boodman 7d688b9d85 Revert "Add camliNodeType:'set'"
This reverts commit 2d67328243.

I decided to do this only with heuristics, since we have to maintain
backward compat with data before this camliNodeType existed.

Change-Id: Ieba5dd1f8a4c8f5aedc1477c4fab1ffebcc2878c
2015-01-02 23:08:16 -08:00
Tamás Gulácsi 7f995f3048 storagetest: test SubFetch with invalid offsets
Fix this newly introduced test case where it fails:
memory, localdisk and diskpacked blobservers.

Change-Id: Idb4b4173c3a03b64e0eb73d5fcfd9994e3e23bcc
2015-01-01 19:32:51 +01:00
Brad Fitzpatrick 3648550b9f blobserver/blobpacked: add missing SubFetch bounds checks
Missing from c137f9030

Change-Id: Ib6467c242228a727876189a358d1cf6a1204daa4
2014-12-31 14:16:50 -08:00
Tamás Gulácsi c137f9030e blobpacked: implement SubFetch
Change-Id: Ibd990752b85f082f866c295b2b5d2b9b3788d684
2014-12-31 22:03:14 +01:00
Brad Fitzpatrick dcf1da7fcd blobserver/storagetest: revert TestStreamer back to only requiring a streamer
Fixes the build.

Change-Id: I76141148bc0564e9ef82ef09fc15c4485b43333c
2014-12-30 22:47:05 -08:00
Brad Fitzpatrick a050f78a93 blobserver: add TODO about simplifying the BlobEnumerator interface
Change-Id: I9d48337543fd22772b2019ee3d6246603cf8ed36
2014-12-30 22:37:09 -08:00
Brad Fitzpatrick fffa94b45e pkg/blobserver: new test to find zip packer boundaries, exercise edge cases
Change-Id: Ida2fc5210a6a94433d3c3cfeb6d6d57dc5e826bc
2014-12-30 22:32:43 -08:00
Brad Fitzpatrick 0ecf7fe5b4 blobserver/blobpacked: include error in log message when packing fails
Change-Id: I793101c7244dd1797a0a92afb2d9db27f5f782a3
2014-12-30 22:28:00 -08:00
Brad Fitzpatrick cf59d7f630 test: add option to NewLogger to have a list of phrases to mute
Change-Id: Id8ff8ac7fd685369c818e0b4e36b6ed48e62e6a3
2014-12-30 22:27:31 -08:00
Brad Fitzpatrick aafb86cb20 blobserver/blobpacked: add an error type for failure to open a packed zip
Change-Id: I08f51b0170acbd154992fa85912f611fc0effa78
2014-12-30 21:05:13 -08:00
Brad Fitzpatrick 6801df337a blobserver/blobpacked: add meta.isPacked method
Change-Id: I30ad05476f8a988ed3a20679a9c701b6be3d3a2f
2014-12-30 21:02:54 -08:00
Mario Russo a645be4554 Merge "ui: redirect to set after creation" 2014-12-29 17:10:44 +00:00
Brad Fitzpatrick 72e5063d53 Merge "cammount: report errors from mount process" 2014-12-28 20:01:48 +00:00
Aaron Boodman 2d67328243 Add camliNodeType:'set'
Change-Id: Ia174d747dbe386f0150c4470c4db170d9cc1d362
2014-12-28 11:38:32 -08:00
Mario Russo 936bd8a336 ui: redirect to set after creation
Updates #553

Change-Id: I1b2f8e2f07cab08f30d2b30356ba02834dcdb95b
2014-12-28 09:52:25 -07:00
Eric Drechsel fd66007592 cammount: report errors from mount process
Change-Id: I0d3ba508a5df48caac2c25c3b92517b5f3228251
2014-12-26 16:52:24 -08:00
Salmān Aljammāz 64220e06ff fs: set OpenDirectIO in stats files to avoid having them cached in kernel
bazil.org/fuse switched from having OpenDirectIO on to having it
off by default. This has the side effect of allowing OS X's kernel
to cache the filesystem's files in its UBC, giving us incorrect
stats from cammount's .camli_fs_stats directory. This change sets
it explicitly to get the correct behaviour again.

Updates #556

Change-Id: Ic6400a550e4874086cf5fb725d423fcfde47fd61
2014-12-26 18:09:32 +00:00
Brad Fitzpatrick b72628c68a blobserver/blobpacked: add test of stream working with 2 zips
Update #532

Change-Id: Ida179a39c4f20fcba2909b0e5eee22a531cf89e4
2014-12-25 19:26:04 -08:00
Brad Fitzpatrick 1b2240f51c blobserver/blobpacked: finish StreamBlobs support
Updates #532

Change-Id: Ifcd3d5a2274942355bed5302861dfa27ddf9dcc5
2014-12-25 19:21:24 -08:00
Brad Fitzpatrick 1d8ffdd2db blobserver/storagetest: more paranoia and debug info on failure in TestStreamer
Change-Id: Ie98503038ef568d5d3e71860978bff846d77da4f
2014-12-25 19:19:57 -08:00
Brad Fitzpatrick b3d751594c blobserver/storagetest: remove debug log
Change-Id: I3d6389a3444284ad325ef74a6918bd4c824ffb95
2014-12-25 16:25:17 -08:00
Brad Fitzpatrick 6e3fc37ae5 blobserver/storagetest: make TestStreamer stricter, compares against Enumerator
Also then fix up diskpacked to work under these stricter rules.

Noticed this while working on blobpacked's tests and noticed them
surprisingly pass at one point. Now they no longer pass (as they
shouldn't yet, since parts are still TODO).

Updates #532

Change-Id: Ie05d19823453594486caf921cabbd149c43df221
2014-12-25 16:15:46 -08:00
Salmān Aljammāz 04bba70cc1 camput: s/name/title in usage string
fixes #544

Change-Id: Ie39e1fb1e26b80853f9d02d15187f221bf5dbaa6
2014-12-25 19:20:21 +00:00
Brad Fitzpatrick 63a0e5d9fe Merge "server: restart camlistored from the /status handler" 2014-12-24 20:10:59 +00:00
Aaron Boodman fbc116392b Increase size and padding of blob item checkmarks
Change-Id: I36bd82cd7cb0e1d57428734eff05133158b10a04
2014-12-24 10:49:47 -08:00
Aaron Boodman 773b4465b9 Merge "Save selection state across navigations." 2014-12-24 18:39:14 +00:00