Commit Graph

119 Commits

Author SHA1 Message Date
mpl 893b5b2f22 camput: sqlite backend for have cache and stat cache
http://camlistore.org/issue/73

Change-Id: Id6fb95f1980139f2b525321a7e1b553cb11fe727
2013-05-01 18:42:48 +02:00
Brad Fitzpatrick 2ebd1aa8e4 Merge "camput: reenable -secret-keyring, use flag.StringVar" 2013-05-01 14:34:12 +00:00
mpl db9fdb7e6e camput: reenable -secret-keyring, use flag.StringVar
http://camlistore.org/issue/76

Change-Id: I37229a1082e5d955cadcc63521ea1b13dd63a33c
2013-05-01 16:48:19 +02:00
Brad Fitzpatrick 6be77f09e4 Merge "Allow HTTPS with a self-signed certificate." 2013-05-01 02:30:23 +00:00
mpl 9b145ad1d7 camput: refactor; break up uploadNodeRegularFile
Change-Id: Ifde8f1a249eb4831757a09ea0ac881b049a90525
2013-04-24 17:51:04 +02:00
mpl ac431439f3 Allow HTTPS with a self-signed certificate.
A new client configuration parameter, "trustedCerts" (list of strings)
is introduced. A certificate fingerprint is defined as the 10 digits
prefix of the sha1 of the whole certificate (in ASN1. DER form).
trustedCerts should contain the list of fingerprints of the
certificates we trust. If not empty, the server's certificate
is checked against that list, instead of using the full x509 validation
on it.

-added a dial function and tls configuration, which check if we're using
SSL, and if we're in "trustedCerts" mode.
pkg/client/client.go
pkg/client/config.go

-moved android specific hacks from camput to the client layer, so that
the dial and tls config could reuse/access them. Also allows future
reuse for other commands, such as camget.
pkg/client/android.go

-adapted camput to the above changes:
cmd/camput/android.go
cmd/camput/camput.go
cmd/camput/files.go

-server prints a hint when it generates the self-signed:
pkg/misc/misc.go
server/camlistored/camlistored.go

-camliactivity:
clients/android/res/xml/preferences.xml
clients/android/src/org/camlistore/Preferences.java
clients/android/src/org/camlistore/SettingsActivity.java
clients/android/src/org/camlistore/UploadService.java
clients/android/src/org/camlistore/UploadThread.java

http://camlistore.org/issue/131

Change-Id: I6be20161549a69aafc8eb7b9e96e9351dc1c5b09
2013-04-23 17:10:37 +02:00
mpl 0b873649cb camput: change "name" option to "title".
Title is the attribute used in the ui, and actually recorded
by the indexer, so having "name" in camput was not consistent.

Change-Id: I6f63e54e3b19a83031f3c56c717b27336b175284
2013-04-22 17:13:50 +02:00
mpl e6a5882511 website doc: cmds: golang style + install command
Change-Id: I4f38dfb87e066ef81881bc9a2262c52109e172c9
2013-03-15 23:18:09 +01:00
mpl 698e8605e6 website: /pkg/ and /cmd/ work.
-added some doc.go for each command to make /cmd/*
work nicely.
-hacked a bit of style, mostly borrowed from golang too.

Change-Id: I011d70bca2c0151a29dfd53c78a896821730d5fb
2013-03-15 16:40:44 +01:00
mpl b5e75e9207 cmdmain: use ExitWithFailure, exit on error instead of returning it
Change-Id: I553bed110394cb8ab48b331ba74d551b840ada5c
2013-02-27 22:04:50 +01:00
mpl 81cb48980e camtool: added dbinit.
Change-Id: I26c58023a5275612dfd2bf85adfd59e8c08baab5
2013-02-27 16:07:02 +01:00
mpl 3a21c89712 camput: fix test. Add descriptions for better help.
Change-Id: I9016fca1dcf36ca5908947e84a5251bf587a83a7
2013-02-26 21:08:24 +01:00
mpl 989092a39b camput: do not parse flags before AddFlags
Change-Id: I514a89999e0e2e71b97d586bd0baa74ff0330472
2013-02-22 23:27:39 +01:00
mpl 59d0d6cb7e Create camtool cmd, with sync subcommand.
Code in common with camput was factorized in pkg/cmdmain/
Partly addresses
http://code.google.com/p/camlistore/issues/detail?id=117

Change-Id: Iabebea7ea78e56bdb8a6eccee9456c52bfe9cceb
2013-02-20 23:47:17 +01:00
mpl 981a4c9fbd camput: even with -vivify, do not reupload "big" file when server has it
This change takes advantage of Uploader.fileMapFromDuplicate to avoid
reuploading "large" files when the server already has them and the
client has lost its cache (and hence does not know about it).

Fixes: http://code.google.com/p/camlistore/issues/detail?id=88

Change-Id: I9fa819cef37e1d930a5393c9bdce5f8bb71da9c6
2013-02-12 19:02:05 +01:00
mpl f7d048d4ee Merge "rm TODO" 2013-02-03 22:50:42 +00:00
Marc-Antoine Ruel 73870dc3c2 Enable the use of localhost as a proxy for debugging purposes.
This is useful to do blackbox performance analysis at the http layer.

Change-Id: Idf26f17e1fd0cb9421b3def9c846c4bffecaa433
2013-02-03 12:26:29 -08:00
Brad Fitzpatrick a8e3cfde54 update TODO
Change-Id: I5075de51cc8fa64658c1d74038a37c3f1a92383c
2013-02-03 10:26:18 -08:00
Brad Fitzpatrick 96ea9af952 camput: add debug mechanism CAMLI_NO_FILE_DUP_SEARCH to disable fileMapFromDuplicate
Change-Id: I1772ed69cccd4245e660887e257da2ab30bd59e4
2013-02-02 21:34:05 -08:00
Brad Fitzpatrick ddcec4e92c Add --version flags to tools.
Change-Id: I1c86c3de09fb59bc20539f7d1efbd4d4710bf494
2013-02-02 10:59:39 -08:00
mpl 9ef7e5d2b9 rm TODO
Verified: if a claimDate is set, the client will create
a signRequest that has a signature time set to that claimDate.
Then when the opengpg signing is done, it will use that time
instead of time.Now().

Change-Id: I658d52dadfcea7a9b6f9291c4d8020ac0efa83b5
2013-02-01 16:09:58 +01:00
mpl 0b2532d96a camput: s/androidStatusRecevier/androidStatusReceiver/
Change-Id: I9cefc638b821948fad911c3e51e4a07c4da655b0
2013-02-01 15:52:39 +01:00
Brad Fitzpatrick e8ef81c626 android/camput: more stats
Change-Id: I668b8ee0b7c32d33075e421445af2973be1f576a
2013-01-30 17:22:53 -08:00
Brad Fitzpatrick dfb45e6784 camput/android: more stat tracking. camput side only so far.
Change-Id: Ifbafda6f5fae01db44cfd557f7b3e81433f85e70
2013-01-30 10:56:10 -08:00
Brad Fitzpatrick 0c2aa8058a android: TODO notes for camput
Change-Id: I751756a6d69bda73ecc5e6b3bd52e7b8b523cc1c
2013-01-29 17:12:32 -08:00
Brad Fitzpatrick 5945b3a7dc campt: in android mode, also notify on skipped chunk uploads
Change-Id: I25e6991c3d8a0703f00091154258e1d5073a28a2
2013-01-29 16:39:18 -08:00
Brad Fitzpatrick 14f038c667 android: working but kinda-sucky progress bars. need camput to broadcast better stats.
Change-Id: I28e6a16eba835116d8e8e205fe71073e0357b9c7
2013-01-28 21:52:23 -08:00
Brad Fitzpatrick 502026bd89 android: UploadThread re-work. use pipe-to-camput-stdin model, and be more Go-like.
Change-Id: I0f598dae28e47036bf713bfde488a7af0884e932
2013-01-28 19:06:37 -08:00
Brad Fitzpatrick 34d1588a6f camput: add android child process input/input mode
Change-Id: I54c80b958c3610b5f9e9b229f68f626553241d47
2013-01-28 17:17:46 -08:00
Brad Fitzpatrick f1dc25c9c5 camput: move initCache to its own function
Change-Id: I1d2029652fe4215a71b89bce178e6df8c40826b9
2013-01-28 07:14:11 -08:00
Brad Fitzpatrick f5a588527b android: untested: more upload / status work
Change-Id: I15dbf861b0bd8684a4e61a09f910315b874db1ed
2013-01-27 17:12:08 -08:00
Brad Fitzpatrick dc49611c53 android: clean-up and start of protocol from camput to android host
Change-Id: Icda33c1d54bdcd0a57f6ac62c09fd3e9cf9cd6f7
2013-01-27 15:27:17 -08:00
Brad Fitzpatrick 2ef8612595 camput: android dialer cleanup
Change-Id: I7f3749cc81029b5a24a3d8696660c79aefdffa84
2013-01-26 17:12:07 -08:00
Brad Fitzpatrick 51e88c7434 camput: run in an Android linux non-cgo environment.
Teaches it about Android DNS resolution and Android SSL root certs.

Change-Id: I22b53d4f836f58cd3dfc8c97e993138a8a5fe318
2013-01-26 16:50:24 -08:00
Brad Fitzpatrick c7bbccf435 config: Rename blobServer config key and flag to just server. use auth from environment when an explicit server is given
Change-Id: I07cfe3df3b644e143fcb92456e56d238bbbdd52e
2013-01-25 16:23:31 -08:00
Brad Fitzpatrick 1bf01d7315 schema: lot of Blob/Builder work, and kill schema.Map.
Next up: unexport schema.Superset.

Change-Id: Ia4bb34790abba75bdb4d4101df7a4f15875f4237
2013-01-21 20:56:12 -08:00
Brad Fitzpatrick 02e3c08947 schema: start of eliminating/unexporting schema.Superset and schema.Map
For now, unfortunately, there are even more types. The old ones will hopefully die soon.

Change-Id: I9d0bb1743c94a0de429175ef0b05ad6a964e07c0
2013-01-21 17:31:08 -08:00
Brad Fitzpatrick 7ceaaa0012 blobref: simplify the FromHash func. Make type implicit.
Change-Id: I2e01c3663bdb1151c11dfc9a1d59c7081940ffac
2013-01-20 13:36:27 -08:00
mpl ff382ceef5 camput: concurrent tagging
Change-Id: Ice7a77d030087648187c1c8ad37888a1a9f406ca
2013-01-17 23:20:40 +01:00
mpl 77d044f4b7 schema: add FileTime to guess the file
creation time from potential file metadata.
camput: add the -exiftime option that allows
the use of the above for -filenodes and -vivify.

Fixes: http://code.google.com/p/camlistore/issues/detail?id=77

Change-Id: I05d1af3d875bb984e47e06775fd149e5ed0ed7b1
2013-01-10 17:43:05 +01:00
mpl 39218e3cd5 Merge "Changed auth to take into account not only the credentials, but the requested operation/action too." 2013-01-04 21:09:37 +00:00
mpl 12213c058e Changed auth to take into account not only the credentials,
but the requested operation/action too.

This allows to restrict vivify credentials to only upload
(as well as get and stat, because they're needed) to the
blobserver.

Change-Id: Idaed60d1f0d679cb9795ba9a11f094f964774335
2013-01-04 22:17:12 +01:00
Brad Fitzpatrick fe13c7c712 move camput's statsTransport to httputil.StatsTransport, use it in camget too.
Change-Id: Iab48e32ccb8a99b8ec6b058061cff9a5b99e8712
2013-01-02 20:32:13 -08:00
Brad Fitzpatrick 6b7d73d757 camput: ignore atimes
Change-Id: Ia55d6fef6a8e3aa95b6265c958853da7b170e3c4
2013-01-01 21:05:02 -08:00
Brad Fitzpatrick 6b466d0faa camput, client: push havecache down deeper, make it track blob sizes.
also, clean up some camput HTTP stuff.

Change-Id: I2762e5a1b83139132b5f660d306ed45584a7799a
2013-01-01 20:23:44 -08:00
Brad Fitzpatrick f8df0845ea camput: Don't use statcache when using vivify
Change-Id: I2c450afd36efdcd2d0fbd16023435936fbc43397
2013-01-01 13:29:15 -08:00
Brad Fitzpatrick 57a60aa944 camput: flatcache TODO.
Change-Id: I4e2368f8e22d5aa71bee8f5319f59444f3c0e638
2012-12-31 13:53:44 -08:00
Brad Fitzpatrick 519c46a388 camput: add FileInfo.Sys fields to stat fingerprint
Change-Id: Idc6af6e945c922aacbf2b51dbed70f4f8e20ab2a
2012-12-31 13:52:17 -08:00
Brad Fitzpatrick 93e8a6c5c2 camput: remove debug log
Change-Id: I1c335ea4c291fe59b56b3b12a0f50a1fbcbc822d
2012-12-31 13:46:49 -08:00
Brad Fitzpatrick 9bee164972 camput: enabling client caching by default
make havecache/statcache named on disk by remote server's generation,
hide debug options in camput unless CAMLI_DEBUG=1 is set,
push havecache down into pkg/client.

Change-Id: Ic152397abc11b4ff3931468ffe93cf1eba62b438
2012-12-31 13:45:13 -08:00