Commit Graph

82 Commits

Author SHA1 Message Date
mpl 955cd34404 dockertest: skip test when DB in container not ready
http://camlistore.org/issue/474

Change-Id: Id65b3c35a2f38466c4d7370bad4c5f73a8cb22db
2014-07-17 18:09:02 +02:00
mpl 1fd526b999 test/transport.go: move to httputil, so we can use it without linking testing
Change-Id: I889d167a78ef0fba5ef2e9e3e884f594e151e940
2014-07-14 18:33:10 +02:00
Brian Gitonga Marete 53ab352fa7 pkg/schema, cmd/camget, cmd/camput: Support sockets.
This patch:

1. Introduces a JSON schema for unix domain sockets (socket files).
2. Adds support for the new schema to pkg/schema.
3. Adds support for uploading and restoring sockets to camput and
camget respectively.

Depends on CL 2706 (https://camlistore-review.googlesource.com/2706/)

Change-Id: Ib0d6e9f9ebd348f55efee5a08cd774f3969d0760
2014-05-21 18:51:14 +03:00
Brian Gitonga Marete e9dd1af195 cmd/camget, cmd/camput, pkg/schema: Support FIFOs.
This patch:

1. Introduces a JSON schema for FIFOs (named pipes).
2. Adds support for uploading and restoring FIFOs to camput and camget
respectively.

It includes tests for the basic schema support in pkg/schema and
integration tests for camput and camget.

This CL does not add FIFO support to the HTTP server and fs (cammount
support) code. Those will be addressed in a subsequent patch.

Change-Id: Iab926073c36e653d25e693a559178bea9bf19b9a
2014-05-21 14:52:23 +03:00
Brad Fitzpatrick c9c737ed05 test: add forgotten file from 94ab7a8
Change-Id: Ied44257d300d55b6b65daedf8f8ebd1a407d53a9
2014-05-14 12:57:23 -07:00
Brad Fitzpatrick 20eca7aad0 search: new DescribeRule describe support
Step towards getting rid of the the integer describe depth API.

Now, after the integer describe depth, all the DescribeRules will also
run to expand things.

Change-Id: I5cfe7e6058be51328e529a2299e13a6a2ba5f869
2014-04-19 12:58:55 -07:00
Daniel Erat 991675db09 test: Fix FakeIndex breakage from 82429eee.
Change-Id: I403fe39617b3d61212b6a0e6aeaffa350c441221
2014-04-16 09:35:41 -07:00
Brad Fitzpatrick 9fb9b4fca9 Merge "netutil,dockertest: some timeout changes" 2014-04-10 02:09:29 +00:00
mpl a9db189cd7 docker tests: remove container when done
http://camlistore.org/issue/421

Change-Id: Iefde3a8235db0103e8fcfac51f5dfcce5df73ef6
2014-04-09 23:26:14 +02:00
mpl 2ed5cfe4ea netutil,dockertest: some timeout changes
1) replaced fixed sleep with exponential back-off tries when creating
test db on postgres.

2) disabled journaling for mongo, so no need to wait for it for a long
time anymore.
(http://docs.mongodb.org/manual/tutorial/manage-journaling/#enable-journaling)

Change-Id: I049eefecb0c19b492735a1c0a2cad27ef997c874
2014-04-08 23:39:05 +02:00
mpl e34c4bc828 client: replace CAMLI_CLIENT_IDENTITY with CAMLI_KEYID
And remove all mentions of CAMLI_CLIENT_IDENTITY because there isn't any
need for the both of them.

http://camlistore.org/issue/412

Change-Id: I0d6b8cff0f23738231de98171a23550d82f05fe9
2014-03-31 18:56:11 +02:00
mpl b0c4f85ee0 sorted/postgres: test with docker
Also, NewKeyValue now automatically creates the required tables.

http://camlistore.org/issue/263

Change-Id: I02dd7660ee08c2eaeec17131622d6f581fdd6c55
2014-03-28 19:34:09 +01:00
Brad Fitzpatrick 768cb6e0c2 Merge "sorted,dockertest: added MySQL (dockerified) test" 2014-03-27 00:34:30 +00:00
mpl 08f9c821f5 sorted,dockertest: added MySQL (dockerified) test
Two important related changes:

1) sorted/mysql now takes into account the host given in the config
2) the required tables are now automatically created by NewKeyValue

http://camlistore.org/issue/263

Change-Id: I0043f36edb0630d6484148508d3a1e08c8e88a94
2014-03-26 00:39:38 +01:00
Brad Fitzpatrick bf5740afb2 Add BaseURL method to handler Loader.
Change-Id: Ibc61cd19798aa1b0858cbe40b10544953db67b36
2014-03-23 17:13:40 -07:00
Govert Versluis 96637ffcd3 pkg/blobserver/mongo: Add support for blob storage in MongoDB
Related changes:
Split docker-related test-helper functions from pkg/sorted/mongo.
These helper functions are now also used in the pkg/blobserver/mongo
tests too.
Also fixed a typo in a comment in pkg/blob/fetcher.go and a missing variable in
debug output in pkg/blobserver/storagetest/storagetest.go

Addresses http://camlistore.org/issue/127

Change-Id: I8b6f57f9ced066d6f83788fdcc87be6619c65c3c

Conflicts:
	pkg/blob/fetcher.go
2014-03-22 12:36:59 -07:00
Brad Fitzpatrick bfe51f3ad5 gofmt
Change-Id: Ie6be315d02a439cbfdd870a0b0b7f16cba12a168
2014-03-20 12:29:45 -07:00
Brad Fitzpatrick bf28dd4488 More status handler HTML+JSON, more sync status.
Change-Id: I0381853191d5b871af649d102b976e592def791f
2014-03-16 20:14:57 -07:00
Brad Fitzpatrick bf94a73859 Get rid of SeekFetcher vs StreamingFetcher distinction and complexity.
StreamingFetcher is now just Fetcher, and its FetchStreaming is now
just Fetch.

SeekFetcher is gone. Blobs are max 16 MB anyway, so we can slurp to
memory when needed. The main thing that cared about SeekFetcher
was the GET handler, ServeBlobref, because http.ServeContent needed
one for range requests. That's rewritten in an earlier commit, using
the FakeSeeker from another earlier commit.

Lot of code got simpler as a result.

Change-Id: Ib819413e48a8f9b8d97f596d0fbf771dab211f11
2014-03-14 12:29:13 -07:00
Brian Gitonga Marete 68074bbadd pkg/schema: Fix camput panic on non-utf8 filenames or symlink targets.
Also implement the relevant integration tests for camput and camget.

Depends on CL 2216.

Change-Id: I3f541efd4e583674e9817473292fa3de8a3cdabc
2014-03-10 21:01:51 -07:00
Brian Gitonga Marete 0c4b67d3bf cmd/camget: Support restoring symbolic links when writing to filesystem.
This patch adds support for restoring symbolic links when camget is
called with the -o option.

This fixes an issue which caused `camget -o' to write an incomplete
snapshot to the target directory when it encountered a symlink.

Note that no attempt is made to restore the symlink metadata since: a)
A symlink's permissions do not matter and are fixed to 1777 on some
systems b) Go's os.Chtimes() always acts on the symlink target c)
os.Chown() is not currently called for any file in the current code.

Tested on Linux.

Change-Id: I7099592dcd5cc513c2685671be6f6b165f8f8053
2014-03-08 05:14:59 +03:00
mpl b4c684dd07 tests: ping camlistored before camput
Change-Id: Icf66bd5b990db07e9516b8ab8945cbf681caf25e
2014-02-24 19:57:44 +01:00
mpl 3f0974ef13 tests: make sure camlistored is running before cammount
Context: http://camlistore.org/issue/385

Change-Id: I793298745096fc1696f01226698eb6682b24806d
2014-02-24 18:29:08 +01:00
Brian Gitonga Marete ccc81242a1 pkg/tests: Warn about deprecation of asserts.* functions in pkg description.
Change-Id: I39e158a92a6f4fa6f3f93551985d9d20ad40c98f
2014-02-19 10:42:29 +03:00
Tamás Gulácsi 97520583b8 Use 'uint32' instead of 'int64' for blob sizes everywhere.
Not just in blob.SizedRef, but in blobserver.Fetch and
blobserver.FetchStreaming, too.
Blobs have a max size of 10-32 MB anyway, and the index.Corpus is now using
uint32 to save memory.

Change-Id: I1172445c2f9463fdaee55bfe0f1218d44be4aa53
2014-02-08 17:58:12 +01:00
Brad Fitzpatrick 50c329d4a2 Add integration test for websocket query
Change-Id: I37c48d5749c7bf31aa118b2bee0861aeb07c5ff3
2014-02-07 15:02:22 -08:00
Brad Fitzpatrick c137cfc32d test: make Fetcher use a RWMutex so fetch doesn't deadlock during enumerate
Change-Id: Iac1f18725231a298c2c636875647c3e6f3f081a4
2014-02-07 14:28:56 -08:00
Brad Fitzpatrick 78c50e8151 test: add World.NewPermanode
Change-Id: I737ae2f69b567eaebf13a3f7c786e06b791e2f7c
2014-02-04 18:49:15 -08:00
Brad Fitzpatrick 1b7ac2da01 test: add TODO to simplify 2014-01-21 14:18:11 -08:00
Brad Fitzpatrick 9abba638a0 Add new environment variables for setting client identity for use with test.World.
Testing clients should now use CAMLI_CLIENT_IDENTITY,
CAMLI_DISABLE_CLIENT_CONFIG_FILE (both new) and existing
CAMLI_SECRET_RING, set using the new test.World accessors.

Change-Id: I9fc0571b008f033674466bbc98129ae1919a0640
2014-01-20 13:54:42 -08:00
Brad Fitzpatrick a384ff188d Get rid of CAMLI_DEV_KEYBLOBS / keyblobs / etc.
Fixes camlistore.org/issue/277

Change-Id: I7f380f9d18785f600fa0c442d5a19cd118782788
2014-01-20 13:47:08 -08:00
Brad Fitzpatrick 31125295a4 test: fix enumerate bug in in-memory storage when double-uploaded
Change-Id: I7e6abb1c9dfc61f39fa9d315857bf7aeb13b6b59
2014-01-20 13:07:20 -08:00
Brad Fitzpatrick aebc8f00c6 cond: add test using storagetest
Change-Id: Iffd987de42d82e1c19ff2c984ff3d2f01824e34c
2014-01-20 10:37:23 -08:00
Brad Fitzpatrick 8e92ad3c71 cond: clean-up and start of tests
Change-Id: Ie7397bde5650f997c9d6abcc1040838b55fccedf
2014-01-19 15:06:55 -08:00
Brad Fitzpatrick f015c227f0 serverconfig: allow marking handlers as internal, so they're not accessible over HTTP
Change-Id: I37190e90fbc2560c07f742270c827f6025a2fb6f
2014-01-19 14:50:34 -08:00
Brad Fitzpatrick 4d9e797ae1 test: fix probing test server. do HTTP request. TCP port will always be listening (by parent)
Change-Id: I3d2bac6419f5e5e0362f549a5aca7b7ccbf967e6
2014-01-19 14:35:05 -08:00
Brad Fitzpatrick e29c43bfde replica: test minWritesForSuccess
Change-Id: I5c845e99b1c440c35bfb7a5a7865176fd72ae031
2014-01-18 22:53:03 -08:00
Brad Fitzpatrick cfeeda0fa9 replica: change receive strategy, fixing a TODO, and start of tests
Change-Id: I74c1206ce4bb34ddffe2601cf95fce393116c303
2014-01-18 21:16:24 -08:00
Brad Fitzpatrick b84c2457ae Add integration test for Android camput environment.
Fixes camlistore.org/issue/323

Change-Id: Iefa5b073aa6d6b00899e7e17afb3bcf2f18ad6b3
2014-01-06 20:01:07 -08:00
Brad Fitzpatrick 5ef53c8bbd Fix build with Go tip. Will file upstream bug.
Change-Id: I1081ff8498caf809c6b4e22a284a59b95032c3ce
2013-12-20 18:19:51 -08:00
Brad Fitzpatrick 4c8d79e170 test: let TLog take a testing.B too. add tests.
Change-Id: I25c47960ef63be85994a133da85072005ca47a85
2013-12-03 21:12:06 -08:00
Brad Fitzpatrick db46a44758 test: add TLog function to set log output to a *testing.T
Change-Id: I68d945634ce7780acabf120fe27989083fb02a04
2013-12-03 18:23:42 -08:00
Brad Fitzpatrick bf4d2721ba search: efficient permanode queries, sorted, and with a limit.
Change-Id: I7f8edfe3636ff3f1344ebb41e27d23073c630ff5
2013-12-02 20:01:37 -08:00
Brad Fitzpatrick b82b8efe4c Start of new context package and *context.Context type.
Will eventually be plumbed through lots of APIs, especially those requiring or benefiting from
cancelation notification and/or those needing access to the HTTP context (e.g. App Engine).

Change-Id: I591496725d620126e09d49eb07cade7707c7fc64
2013-12-02 13:20:51 -08:00
Brad Fitzpatrick f544114844 Fix diskpacked regression, add tests, start of storagetest.
Regressed from rev cb6f423e. Eventually pkg storagetest should test all methods of blobserver.Storage
for all storage target types.

Change-Id: I2c1c93b76fd9280a3eb429b1d71c64a693ed1ace
2013-11-30 13:06:04 -08:00
Brad Fitzpatrick 8c46afe490 Shrink camtypes.BlobMeta a bit. Saves 4 bytes of memory per blob.
For me,

Before:
Corpus stats: 148.859 MiB mem: 799465 blobs (89.487 GiB) (282303 schema (6478 permanode, 150962 file (112224 image), ...)

After:
Corpus stats: 146.559 MiB mem: 799465 blobs (89.487 GiB) (282303 schema (6478 permanode, 150962 file (112224 image), ...)

Change-Id: Id421dfc9d72c2a24c228418c01f9a3f05900c941
2013-11-29 13:41:39 -08:00
Brad Fitzpatrick a918fac281 search: let GetRecentPermanodes take a before param.
For infinite scroll.  Will be done better later, but this works for now.

Change-Id: I88dc7793b643226a4a5e3a583c5de5d4c7fa4819
2013-11-25 20:35:59 -08:00
Brad Fitzpatrick 26f9b630f8 Add func test.BrokenTest and RUN_BROKEN_TESTS to doc/environment-vars.txt
Change-Id: I43dc7b10ac6175ab8ab5eae4cfdd98bc08515ff5
2013-11-24 15:00:16 -08:00
Brad Fitzpatrick 531b4dc32b search: remove MIME from DescribedBlob; fix Handler tests.
Change-Id: I2d92155e3446325520d91379ac24becef49ca56f
2013-11-18 16:36:08 -08:00
Brad Fitzpatrick 0b07171c0c index: change GetOwnerClaims interface for efficiency, rename to AppendClaims
Change-Id: Icec235961ed8a09256df87c5fe1455cbad81ae3f
2013-11-17 14:54:30 -08:00