Commit Graph

193 Commits

Author SHA1 Message Date
Salmān Aljammāz 04bba70cc1 camput: s/name/title in usage string
fixes #544

Change-Id: Ie39e1fb1e26b80853f9d02d15187f221bf5dbaa6
2014-12-25 19:20:21 +00:00
mpl 250664d8eb camput: use Errorf to print critical errors even when non-verbose
http://camlistore.org/issue/524

Change-Id: Ied24471288ea459785291281afa7316a257b6884
2014-11-03 22:15:36 +01:00
Brad Fitzpatrick 8f2bdd3a82 Few little fixes to 51ef6152f
Change-Id: Id319d1fa4ad725f1ce9178fd0fbd7f787880a6fa
2014-09-06 19:19:36 -07:00
Aaron Boodman 51ef6152fd Add -search argument to camput share.
Bug: https://code.google.com/p/camlistore/issues/detail?id=512

Change-Id: Ic0dd559c98589107fcdeade8784699cf6b2f1761
2014-09-06 18:50:21 -07:00
mpl 0fa50c4355 client&camput: use cmdmain.Stderr for logging
And use -verbose in tests, for easier debugging.

Change-Id: Iac5e1a8e9c987f82fc221ce73c7e3ca10a86a924
2014-08-12 22:41:55 +02:00
Brad Fitzpatrick cc956829f3 Rename android.go files to androidx.go
Go 1.4 (Go tip currently) defines a new GOOS=android, and a GOOS.go file is ignored on
other operating systems. But these android.go aren't actually meant to be only run on
GOOS=android, since we just build Linux binaries on Android for now.
2014-07-08 11:18:40 -07:00
Brian Gitonga Marete 53ab352fa7 pkg/schema, cmd/camget, cmd/camput: Support sockets.
This patch:

1. Introduces a JSON schema for unix domain sockets (socket files).
2. Adds support for the new schema to pkg/schema.
3. Adds support for uploading and restoring sockets to camput and
camget respectively.

Depends on CL 2706 (https://camlistore-review.googlesource.com/2706/)

Change-Id: Ib0d6e9f9ebd348f55efee5a08cd774f3969d0760
2014-05-21 18:51:14 +03:00
Brian Gitonga Marete e9dd1af195 cmd/camget, cmd/camput, pkg/schema: Support FIFOs.
This patch:

1. Introduces a JSON schema for FIFOs (named pipes).
2. Adds support for uploading and restoring FIFOs to camput and camget
respectively.

It includes tests for the basic schema support in pkg/schema and
integration tests for camput and camget.

This CL does not add FIFO support to the HTTP server and fs (cammount
support) code. Those will be addressed in a subsequent patch.

Change-Id: Iab926073c36e653d25e693a559178bea9bf19b9a
2014-05-21 14:52:23 +03:00
mpl 2b091db606 camput init: do not call DefaultSecretRingFile in init()
Because we fail in tests when calling methods that rely on default
paths. (See buildinfo.TestingLinked)

Change-Id: I4d3a99dd2ea83a8351a993353e5e7c29d3cb17a7
2014-03-31 18:01:04 +02:00
mpl cdc4ed5ae2 camput init: ring/key related fixes
1) Removed exec call to gpg, because it automatically looks in .gnupg/,
which we don't use anymore as a default.
2) Now taking into account global --secret-keyring flag.
This flag is now in osutil.
3) New or modified funcs in osutil
4) Made sure --gpgkey works too.
5) Cleaned up error messages and hints.

Context: http://camlistore.org/issue/364
         http://camlistore.org/issue/368

Change-Id: I2e51032ed0597da656db100d72f5588b37308e1a
2014-03-28 19:55:18 +01:00
Brad Fitzpatrick f1a4aae0da camput: add --blobdir=discard debug option
Does a dry-run and just shows the final blobref, without writing data anywhere.

Change-Id: Ia234b642b5b9029ee4fd9c61bc7edcdcfa5ca858
2014-03-04 22:47:58 -08:00
Brad Fitzpatrick e3491e5515 camput: fix tests 2014-03-03 12:53:40 -08:00
Brad Fitzpatrick 26cbd36e0b camput, camtool: update docs
Change-Id: If54df42be3b95ec9f10782454bc247449e1a74e6
2014-02-27 21:23:01 -08:00
Brad Fitzpatrick 593e0f0f81 cmdmain, camput: PreExit should always run, not just in verbose mode
Change-Id: I6001a3474071a00f0ab56b031ac8c5504e5e9da5
2014-02-27 20:47:15 -08:00
Brad Fitzpatrick bf35e5f011 camput: fix camput blob from stdin
Change-Id: Ic9997c408e1241e16dd4470b3f733ef3fb4f79c9
2014-02-25 14:34:26 -08:00
Brad Fitzpatrick 897591a444 camput: hermetic client config for test
Change-Id: I3c77b500da2d445851b0d8383a11e7f0942b2ff2
2014-02-23 10:59:41 -08:00
Brad Fitzpatrick ccacf76451 camput: re-enable deadlock test, now that it's fixed
And tweak limits a bit lower.

Change-Id: I7a38ffce37a997aa6ce9943e13605e76476f5646
2014-02-21 15:38:21 -08:00
Tamás Gulácsi 89fc2bb83e camput: limit upload concurrency (number of goroutines)
Use separate workers for directory uploads.
Limit upload concurrency to 3+2+3 (normal+directories+stat)
goroutines on Android, and 10+5+10 otherwise.

Thanks to the predictably failing test (1333551)
which we can test against, we can eliminate the need for
unbounded number of upload goroutines (see ee4550b).

Change-Id: Idac4c94b7d7ec4fc6c6a77fed111e0ece0843545
2014-02-21 21:31:13 +01:00
Brad Fitzpatrick 85e2e5e301 Fix crash with "go test -v -cpu=1,1 ./cmd/camput/"
Don't double-register the help flag. Only affects tests run twice, like above.

Change-Id: Ia235f2e2aa4a70e3b5a5c6fe21d8add90069664f
2014-02-21 10:59:34 -08:00
Brad Fitzpatrick 1333551de1 camput: predictably-failing disabled deadlock test
Change-Id: Ibf0877ac1b4e32ad32fe9592c487915130f9f488
2014-02-20 12:08:03 -08:00
Brad Fitzpatrick dfd94e505d camput: start of a deadlock test
Currently disabled until it's working reliably. See comments in code.

Much of this is from Tamás Gulácsi in https://camlistore-review.googlesource.com/#/c/2128/

Change-Id: I2fe2abfdee98584c698a2ac52202f13dda9beef2
2014-02-19 00:07:28 -08:00
Brad Fitzpatrick 77a34e060c camput: add a var for uploadWorkers
Change-Id: I529336ece16a35e25c33960589f706ebf8b2c5ec
2014-02-18 23:59:44 -08:00
Brad Fitzpatrick b023265043 camput: configurable timeout on env
Change-Id: Iffd0b404e48e4fd5235c22bdd00ad22c564fb0a0
2014-02-18 23:58:09 -08:00
Brad Fitzpatrick 685b44cd43 camput: don't show HTTP req stats when not using HTTP
Change-Id: I4d0c0dc44de84fe62f3ee2385bcd3875eb27be97
2014-02-18 23:56:22 -08:00
Brad Fitzpatrick 53ec1b496c camput: move debug flag registration to its own func, for use by tests later.
Also, don't call TransportForConfig on client when we're in blobdir mode.
Avoids a warning.

Change-Id: I5d618b7ab7d701b4c9d9086c8adedef7b447f62b
2014-02-18 21:24:52 -08:00
Brad Fitzpatrick 1509f266be camput: set os.Args from env.Run.args before test's cmdmain.Main runs
Not required now, but will be for future tests.

Change-Id: I50c469d4f4fa589494036f306c25fe60914fb902
2014-02-18 18:47:11 -08:00
Tamás Gulácsi 97520583b8 Use 'uint32' instead of 'int64' for blob sizes everywhere.
Not just in blob.SizedRef, but in blobserver.Fetch and
blobserver.FetchStreaming, too.
Blobs have a max size of 10-32 MB anyway, and the index.Corpus is now using
uint32 to save memory.

Change-Id: I1172445c2f9463fdaee55bfe0f1218d44be4aa53
2014-02-08 17:58:12 +01:00
Brad Fitzpatrick 5458321511 campit: add -blobdir flag for serverless mode.
Fixes camlistore.org/issue/362

Change-Id: If4c75b649cc4cc62f3e7a0ab07c61b088560c3e4
2014-02-07 11:02:07 -08:00
Brad Fitzpatrick 48373fc8a7 camput: clean stale stuf in cachedir on start
Change-Id: Ie7b8d7da49703e5c430b8195202791445223cf25
2014-01-31 12:42:59 +01:00
Brad Fitzpatrick e2dc7a9256 camput: make uploading a directory tree less verbose. maybe too much.
Change-Id: Ia33588f4adc5af76bdca9ea8d26cec2340a94b30
2014-01-31 11:50:08 +01:00
Brad Fitzpatrick d77426ed6e camput: remove some android TODOs, now done
Change-Id: I206808f59bbf5a8f8dae35586b61180e0d404374
2014-01-30 17:29:41 +01:00
Brad Fitzpatrick 3f27e4f708 Add android camput pre-exit hook.
Change-Id: I30f9771e5562ece6a35d44cc8ab57c305a2899cb
2014-01-30 17:29:28 +01:00
Brad Fitzpatrick a70b554023 pkg/client, camput: don't stat each chunk before uploading
Trust the have cache for skipping uploads.

Reduces total number of HTTP requests.

Also, set Transport's MaxIdleConnsPerHost to be the same as
our HTTP concurrency level, to not keep re-creating TCP
connections.
2014-01-27 16:07:15 -08:00
Brad Fitzpatrick a384ff188d Get rid of CAMLI_DEV_KEYBLOBS / keyblobs / etc.
Fixes camlistore.org/issue/277

Change-Id: I7f380f9d18785f600fa0c442d5a19cd118782788
2014-01-20 13:47:08 -08:00
Brad Fitzpatrick 2086406552 Client fixes and cleanups:
* Fix TODO about moving camput file's havecache up a layer into
  camput itself. So now it's used for all commands (camput permanode,
  camput attr, etc). Reduces HTTP requests.

* Fix camlistore.org/issue/343 -- don't do stats before uploads
  in cases where it's useless. Adds a field to UploadHandle.

* Fix camlistore.org/issue/344: upload the public key whenever
  uploading a signed blob. Usually this is a no-op because of the
  have cache.

* Use zero (or <=0 rather), not -1, to mean unknown on UploadHandle
  Size.

* More docs on public stuff.
2014-01-15 18:32:02 -08:00
Mateus Braga 3d4ccbe0f4 Remove unused NewSetAttributeClaim in RunCommand
Change-Id: I092020f675296e729e3dfe15d97a42682dd13ab9
2014-01-08 11:11:57 -02:00
mpl 10d863e323 Merge "pkg/client: multi servers config" 2014-01-06 07:24:52 +00:00
mpl bf8c463d0a pkg/client: multi servers config
http://camlistore.org/issue/309

Change-Id: I22bbbf6a808a772272f153b2535d693bd986d13a
2014-01-05 23:19:43 -08:00
Aaron Racine 07a42aaa40 camput: Skip base directories where IsIgnoredFile() returns true.
camput currently panics if a base directory is ignored.  Fixes issue 314.

Change-Id: I9466b4576b1134994e5e53e1067e0435ddba0fdf
2014-01-02 18:41:54 -08:00
Brad Fitzpatrick 33c42a2326 Remove message about gnupg-agent.
Change-Id: Ibe239b079e52b520e87dec194c6166aabcc02f63
2013-12-29 14:46:04 -08:00
mpl 6e7eacea4a Merge "pkg/client: some cleanup, doc on the website" 2013-12-23 22:31:45 +00:00
mpl 1e8d46ee45 pkg/client: some cleanup, doc on the website
Changed some keys to be consistent with the server config, and removed
some others.

We're now using jsonconfig to get the keys and validate the config.

Removed some unused funcs/methods, moved some global vars as members of a client.

Also, having two possible defaults for the secret keyring
(.config/camlistore/identity-secring.gpg and .gnupg/secring.gpg)
was annoying and confusing imho, so I removed the latter.

http://camlistore.org/issue/259

Change-Id: Icf79fa8822b62b5d91843698861dd19cb553607a
2013-12-23 20:23:21 +01:00
Brad Fitzpatrick 450409d0a8 camput: rename -name flag to -title for consistency with other commands
Fixes camlistore.org/issue/285

Change-Id: Ide3fba2c40fc8e9f0ff187c183f2cc0acd97f68e
2013-12-22 09:29:32 -08:00
jakubbrzeski 33e925ffa1 camput: allow ignoring files NOT only if --filenodes used
Change-Id: I7856c5f6602e6b439ebcebbcb83eddb1d1865e88
2013-12-16 16:52:59 +01:00
mpl 62cbf8550c camput init: use osutil.KeyBlobsDir()
Change-Id: I1b9f29c24361ffab708af4bdaff8c870ff8730e6
2013-12-11 00:47:50 +01:00
Brad Fitzpatrick 47828b6900 camput: add --delete_after_upload flag.
But untested in tree mode. I think it's incomplete still.

Change-Id: Ia0c692e53a099b12a2b66735c228ea6f78d9c940
2013-12-10 15:05:41 +04:00
mpl d51647ae33 camput: delete command, to send a delete claim
http://camlistore.org/issue/191

Change-Id: I053ab468420cc173e6122909bf3a0b16e4d43224
2013-11-29 00:19:42 +01:00
Brad Fitzpatrick dcf5f6abe7 client: clean up. finish moving all Android stuff to pkg/client/android
Change-Id: Ia3d19a323d9c734a01ec9a12509b115191afcfdd
2013-10-26 11:08:34 -07:00
Brad Fitzpatrick 2040d2e442 Add new pkg/client/android, to clean up Android API gunk from pkg/client's godoc.
This is just a start. AndroidStatsTransport and AndroidStatusReceiver and most (all?)
of pkg/client/android.go should migrate too.

Change-Id: Id4140730bbdd218fdf7ba7cedc7870ce5164e06c
2013-10-22 07:37:15 -07:00
Brad Fitzpatrick cf15b2be90 client: use the schema.Signer API, remove Client.SignBlob method
Change-Id: I177c78f4f7f4ca54afd021d4ada866a35eea06eb
2013-10-22 07:26:44 -07:00