257fd29afa
camget: allow insecure TLS with shared URLs
...
Change-Id: I22f610009edfecea932019c35310ae62b0613cb8
2013-07-16 15:56:32 +02:00
e43e703599
dev-cammount, cammount: add xterm flag, to bring up a debug xterm in that dir.
...
remove child process mode. not necessary.
Change-Id: I4f6e75efe127a2b9e7867dbfac5c42fcc874e3a6
2013-07-15 21:29:00 +10:00
167138fd65
cammount: quieter. and only exit when 'q' is pressed.
...
Change-Id: I2c8719490f977d14e526cd34672ce784589dd460
2013-07-11 19:21:59 +10:00
02d38b07ca
camput, client: move lot of camput's code into pkg/client
...
so cammount can use it.
Change-Id: I1bd513e6dfe6faab785c856abbd0d1337e978dd9
2013-07-11 16:46:23 +10:00
ae8aec21e4
dev-cammount / cammount: sane hack/debug cycle. reliable start-up & shutdown / unmount.
...
especially on OS X.
Change-Id: Ibb60b82be72049dc44984e8aff7cb54560800364
2013-07-10 21:27:54 +10:00
1a89d81bf1
cammount: instruct user to install FUSE
...
Change-Id: Ia3e30bbd9713dad542e3d2f946c27892e6bb35ea
2013-07-10 19:37:31 +10:00
c2669a6dfc
issue 23: Fix google storage blobserver as sync dest
...
Change-Id: I33d88dc5028ebb76d4c17d2a00d8f8346114f876
2013-07-08 11:33:10 -05:00
9468e5ba70
More docs. Every package is documented now.
...
misc.CountingReader moves into readerutil.
pkg/atomics is folded into pkg/types.
pkg/test/testdep is folded into pkg/test, with better name/docs.
Old cruft from pkg/webserver is deleted.
Change-Id: I3f72d8b29804254ef944995fb085837c878f79f5
2013-07-07 21:12:30 -07:00
76013089a4
Move blobref.ParseMulti to its sole caller, with error checking.
...
Change-Id: I6351591497879ea5a113bc8e8414b751b3d1c812
2013-07-07 11:46:21 -07:00
fd8af44431
camput: stat cache: use a different key in -filenodes mode
...
http://camlistore.org/issue/157
Change-Id: If58116be6073e0bd160d57bb8f778e498a8763bb
2013-06-28 19:52:57 +02:00
f7732db604
cammount: rm TODO
...
Change-Id: Id97cd0400c14ddbbfc7590055ba14d6bc1d1c554
2013-06-26 15:36:19 +02:00
bb7487ea0b
cammount: setup transport (trusted certs support)
...
Change-Id: I2970986637094d5bf581545bbedab601c62c1641
2013-06-26 15:19:14 +02:00
90ecd785ba
Merge "client: factorize the transport setup, and use it in camget"
2013-06-26 01:58:07 +00:00
a5d72541c3
Enhance PostgreSQL support:
...
* Fix dbinit command in dev server
* Make sslmode configurable in dbinit
* Don't fail on index schema setup when plpsql language already exists
Change-Id: Ic70461963f6266172d21cda8e6f071e2442d7651
2013-06-25 16:48:38 -07:00
65f3079498
Replace github.com/bmizerany/pq with github.com/lib/pq postgres driver.
...
Change-Id: If62fd5b489696171c8be6b42a988e1c7d0a634d0
2013-06-25 16:13:22 -07:00
b24ac50965
client: factorize the transport setup, and use it in camget
...
Fixes http://camlistore.org/issue/153
Change-Id: If393b34bbfcf4a6282357c9c57ebb85bae8741a3
2013-06-25 23:37:41 +02:00
ded8f8d0d7
Be helpful about how to setup SQLite, especially on a Mac.
...
Change-Id: I87dbcd077383f0148a8c50f4120e6606139fcacd
2013-06-22 17:51:09 -07:00
0de2881a16
client: help (wrt to the gpg key) initialize the config
...
Change-Id: I983e4396abacbc4d8fc354863cffeece65dd5b90
2013-06-22 01:16:11 +02:00
2e632793ca
encrypt: start of index recovery/compaction from meta blobs on start-up
...
Change-Id: I8aae66194ee67b1d63704ff98868e02f8b943e17
2013-06-17 21:56:25 -07:00
d1fd483548
camput: remove Go 1.0 workaround.
...
Now that we rely on Go 1.1, we no longer need to workaround https://code.google.com/p/go/issues/detail?id=4589
Change-Id: Ib6631679e7f9bcea82ef96df0f50d9ecace31aa3
2013-06-16 20:56:58 -07:00
a32b4a10be
Merge "sqlite: Write-Ahead Logging, for improved concurrency"
2013-06-12 15:00:30 +00:00
36c1ecebd9
Let camtool sync's --dest and --thirdleg flags take local disk paths, instead of servers.
...
Change-Id: I8daddf06a7201384edff2cb22ecfd5872fc84928
2013-06-09 11:36:26 +02:00
893b5b2f22
camput: sqlite backend for have cache and stat cache
...
http://camlistore.org/issue/73
Change-Id: Id6fb95f1980139f2b525321a7e1b553cb11fe727
2013-05-01 18:42:48 +02:00
a4aba6671a
sqlite: Write-Ahead Logging, for improved concurrency
...
http://camlistore.org/issue/114
Change-Id: I93c3d48f06a6cb0703bd9ce3f89b03fc59826a73
2013-05-01 18:04:03 +02:00
2ebd1aa8e4
Merge "camput: reenable -secret-keyring, use flag.StringVar"
2013-05-01 14:34:12 +00:00
db9fdb7e6e
camput: reenable -secret-keyring, use flag.StringVar
...
http://camlistore.org/issue/76
Change-Id: I37229a1082e5d955cadcc63521ea1b13dd63a33c
2013-05-01 16:48:19 +02:00
6be77f09e4
Merge "Allow HTTPS with a self-signed certificate."
2013-05-01 02:30:23 +00:00
9b145ad1d7
camput: refactor; break up uploadNodeRegularFile
...
Change-Id: Ifde8f1a249eb4831757a09ea0ac881b049a90525
2013-04-24 17:51:04 +02:00
ac431439f3
Allow HTTPS with a self-signed certificate.
...
A new client configuration parameter, "trustedCerts" (list of strings)
is introduced. A certificate fingerprint is defined as the 10 digits
prefix of the sha1 of the whole certificate (in ASN1. DER form).
trustedCerts should contain the list of fingerprints of the
certificates we trust. If not empty, the server's certificate
is checked against that list, instead of using the full x509 validation
on it.
-added a dial function and tls configuration, which check if we're using
SSL, and if we're in "trustedCerts" mode.
pkg/client/client.go
pkg/client/config.go
-moved android specific hacks from camput to the client layer, so that
the dial and tls config could reuse/access them. Also allows future
reuse for other commands, such as camget.
pkg/client/android.go
-adapted camput to the above changes:
cmd/camput/android.go
cmd/camput/camput.go
cmd/camput/files.go
-server prints a hint when it generates the self-signed:
pkg/misc/misc.go
server/camlistored/camlistored.go
-camliactivity:
clients/android/res/xml/preferences.xml
clients/android/src/org/camlistore/Preferences.java
clients/android/src/org/camlistore/SettingsActivity.java
clients/android/src/org/camlistore/UploadService.java
clients/android/src/org/camlistore/UploadThread.java
http://camlistore.org/issue/131
Change-Id: I6be20161549a69aafc8eb7b9e96e9351dc1c5b09
2013-04-23 17:10:37 +02:00
0b873649cb
camput: change "name" option to "title".
...
Title is the attribute used in the ui, and actually recorded
by the indexer, so having "name" in camput was not consistent.
Change-Id: I6f63e54e3b19a83031f3c56c717b27336b175284
2013-04-22 17:13:50 +02:00
1148a51b09
camtool dbinit: sqlite support
...
Change-Id: I649a28e52df8d2689daa784dd2b1c06770e4b857
2013-03-20 18:10:57 +01:00
f1c5a12085
Merge "website doc: cmds: golang style + install command"
2013-03-15 22:05:26 +00:00
e6a5882511
website doc: cmds: golang style + install command
...
Change-Id: I4f38dfb87e066ef81881bc9a2262c52109e172c9
2013-03-15 23:18:09 +01:00
cca2ce6cd8
cmd: clean up obsolete camsync dir
...
Change-Id: Idfb82c175a5dd0b73ef277b289aeac29319b753d
2013-03-15 19:36:00 +01:00
698e8605e6
website: /pkg/ and /cmd/ work.
...
-added some doc.go for each command to make /cmd/*
work nicely.
-hacked a bit of style, mostly borrowed from golang too.
Change-Id: I011d70bca2c0151a29dfd53c78a896821730d5fb
2013-03-15 16:40:44 +01:00
eb81f69fdd
implement camtool sync -thirdleg flag
...
Fixes issue 92.
Change-Id: Ib6d5620a98d197d84f6f2954dbcbc14eaaa8e228
2013-03-11 21:09:25 +11:00
1838fe8fa6
cammount: fix for bug I introduced in 2384f368f6
...
Change-Id: I81e85e65496113166e9716fc553475b4bf23b1b6
2013-03-03 21:50:06 +01:00
2384f368f6
Merge "cammount: enable share URL as arg (akin to -shared for camget)"
2013-02-27 22:03:41 +00:00
7b65f21858
cammount: enable share URL as arg (akin to -shared for camget)
...
https://code.google.com/p/camlistore/issues/detail?id=106
Change-Id: Ib45783c0c89c9fb1d0aa7a3f7d892de9f7e03de7
2013-02-27 23:14:40 +01:00
b5e75e9207
cmdmain: use ExitWithFailure, exit on error instead of returning it
...
Change-Id: I553bed110394cb8ab48b331ba74d551b840ada5c
2013-02-27 22:04:50 +01:00
81cb48980e
camtool: added dbinit.
...
Change-Id: I26c58023a5275612dfd2bf85adfd59e8c08baab5
2013-02-27 16:07:02 +01:00
3a21c89712
camput: fix test. Add descriptions for better help.
...
Change-Id: I9016fca1dcf36ca5908947e84a5251bf587a83a7
2013-02-26 21:08:24 +01:00
d1a909ea5e
camtool: added gsinit, was cmd/camgsinit. Better help too.
...
Change-Id: Icefe51759db8b89b51f408a66a3d10f2148d7137
2013-02-26 14:45:49 +01:00
4223b02d52
Merge "camdebug moved as camtool debug"
2013-02-25 14:47:48 +00:00
d2afc53a61
camdebug moved as camtool debug
...
Change-Id: I54a4ec816b4fdaca8b2476f304e13b23abbe217b
2013-02-23 00:53:03 +01:00
989092a39b
camput: do not parse flags before AddFlags
...
Change-Id: I514a89999e0e2e71b97d586bd0baa74ff0330472
2013-02-22 23:27:39 +01:00
317c87c189
Merge "Create camtool cmd, with sync subcommand."
2013-02-20 22:47:33 +00:00
59d0d6cb7e
Create camtool cmd, with sync subcommand.
...
Code in common with camput was factorized in pkg/cmdmain/
Partly addresses
http://code.google.com/p/camlistore/issues/detail?id=117
Change-Id: Iabebea7ea78e56bdb8a6eccee9456c52bfe9cceb
2013-02-20 23:47:17 +01:00
0379ff1d43
camdebug: add exif mode; camlistore.org/issue/118
...
Change-Id: Ifb58580fda9ca35e2f709b65f7a8c6f7b7fb3061
2013-02-18 19:42:42 -08:00
59566cd0ab
camsync: some verbiage
...
Change-Id: Id386173d5710021e13dc6a3f9c4ab56dc05edc32
2013-02-14 01:24:20 +01:00
981a4c9fbd
camput: even with -vivify, do not reupload "big" file when server has it
...
This change takes advantage of Uploader.fileMapFromDuplicate to avoid
reuploading "large" files when the server already has them and the
client has lost its cache (and hence does not know about it).
Fixes: http://code.google.com/p/camlistore/issues/detail?id=88
Change-Id: I9fa819cef37e1d930a5393c9bdce5f8bb71da9c6
2013-02-12 19:02:05 +01:00
1b06c3a8c1
NewCamliFileSystem needs a client. Work on Issue 95.
...
Change-Id: I417d9d04398e734d88d3139a91309a8f0eb8084d
2013-02-06 21:47:01 -08:00
f7d048d4ee
Merge "rm TODO"
2013-02-03 22:50:42 +00:00
73870dc3c2
Enable the use of localhost as a proxy for debugging purposes.
...
This is useful to do blackbox performance analysis at the http layer.
Change-Id: Idf26f17e1fd0cb9421b3def9c846c4bffecaa433
2013-02-03 12:26:29 -08:00
a8e3cfde54
update TODO
...
Change-Id: I5075de51cc8fa64658c1d74038a37c3f1a92383c
2013-02-03 10:26:18 -08:00
96ea9af952
camput: add debug mechanism CAMLI_NO_FILE_DUP_SEARCH to disable fileMapFromDuplicate
...
Change-Id: I1772ed69cccd4245e660887e257da2ab30bd59e4
2013-02-02 21:34:05 -08:00
ddcec4e92c
Add --version flags to tools.
...
Change-Id: I1c86c3de09fb59bc20539f7d1efbd4d4710bf494
2013-02-02 10:59:39 -08:00
ed40fc89f7
Clean up the clients dir a bit
...
Change-Id: I1fb8e188044e8a552aa02acedb92c1c9b1d75330
2013-02-02 09:02:43 -08:00
9ef7e5d2b9
rm TODO
...
Verified: if a claimDate is set, the client will create
a signRequest that has a signature time set to that claimDate.
Then when the opengpg signing is done, it will use that time
instead of time.Now().
Change-Id: I658d52dadfcea7a9b6f9291c4d8020ac0efa83b5
2013-02-01 16:09:58 +01:00
0b2532d96a
camput: s/androidStatusRecevier/androidStatusReceiver/
...
Change-Id: I9cefc638b821948fad911c3e51e4a07c4da655b0
2013-02-01 15:52:39 +01:00
e8ef81c626
android/camput: more stats
...
Change-Id: I668b8ee0b7c32d33075e421445af2973be1f576a
2013-01-30 17:22:53 -08:00
dfb45e6784
camput/android: more stat tracking. camput side only so far.
...
Change-Id: Ifbafda6f5fae01db44cfd557f7b3e81433f85e70
2013-01-30 10:56:10 -08:00
0c2aa8058a
android: TODO notes for camput
...
Change-Id: I751756a6d69bda73ecc5e6b3bd52e7b8b523cc1c
2013-01-29 17:12:32 -08:00
5945b3a7dc
campt: in android mode, also notify on skipped chunk uploads
...
Change-Id: I25e6991c3d8a0703f00091154258e1d5073a28a2
2013-01-29 16:39:18 -08:00
14f038c667
android: working but kinda-sucky progress bars. need camput to broadcast better stats.
...
Change-Id: I28e6a16eba835116d8e8e205fe71073e0357b9c7
2013-01-28 21:52:23 -08:00
502026bd89
android: UploadThread re-work. use pipe-to-camput-stdin model, and be more Go-like.
...
Change-Id: I0f598dae28e47036bf713bfde488a7af0884e932
2013-01-28 19:06:37 -08:00
34d1588a6f
camput: add android child process input/input mode
...
Change-Id: I54c80b958c3610b5f9e9b229f68f626553241d47
2013-01-28 17:17:46 -08:00
f1dc25c9c5
camput: move initCache to its own function
...
Change-Id: I1d2029652fe4215a71b89bce178e6df8c40826b9
2013-01-28 07:14:11 -08:00
f5a588527b
android: untested: more upload / status work
...
Change-Id: I15dbf861b0bd8684a4e61a09f910315b874db1ed
2013-01-27 17:12:08 -08:00
dc49611c53
android: clean-up and start of protocol from camput to android host
...
Change-Id: Icda33c1d54bdcd0a57f6ac62c09fd3e9cf9cd6f7
2013-01-27 15:27:17 -08:00
2ef8612595
camput: android dialer cleanup
...
Change-Id: I7f3749cc81029b5a24a3d8696660c79aefdffa84
2013-01-26 17:12:07 -08:00
51e88c7434
camput: run in an Android linux non-cgo environment.
...
Teaches it about Android DNS resolution and Android SSL root certs.
Change-Id: I22b53d4f836f58cd3dfc8c97e993138a8a5fe318
2013-01-26 16:50:24 -08:00
c7bbccf435
config: Rename blobServer config key and flag to just server. use auth from environment when an explicit server is given
...
Change-Id: I07cfe3df3b644e143fcb92456e56d238bbbdd52e
2013-01-25 16:23:31 -08:00
cfc32e4a05
schema, camget: more work on deleting the Superset type.
...
not much more remains.
Change-Id: I6cfe4145f67b100a0e2509f88ce6e1c580b7f9fe
2013-01-22 09:32:40 -08:00
1bf01d7315
schema: lot of Blob/Builder work, and kill schema.Map.
...
Next up: unexport schema.Superset.
Change-Id: Ia4bb34790abba75bdb4d4101df7a4f15875f4237
2013-01-21 20:56:12 -08:00
02e3c08947
schema: start of eliminating/unexporting schema.Superset and schema.Map
...
For now, unfortunately, there are even more types. The old ones will hopefully die soon.
Change-Id: I9d0bb1743c94a0de429175ef0b05ad6a964e07c0
2013-01-21 17:31:08 -08:00
7ceaaa0012
blobref: simplify the FromHash func. Make type implicit.
...
Change-Id: I2e01c3663bdb1151c11dfc9a1d59c7081940ffac
2013-01-20 13:36:27 -08:00
7c319ca284
pkg/cacher: unify the cammount and camget disk caching code
...
Change-Id: I8e53d00f1f5459856a4e2b863d74c2b7c3f515bc
2013-01-20 09:52:03 -08:00
ff382ceef5
camput: concurrent tagging
...
Change-Id: Ice7a77d030087648187c1c8ad37888a1a9f406ca
2013-01-17 23:20:40 +01:00
8268b2a298
camsync: set logger for all clients
...
Change-Id: I9d3e9eeeadbceebbd7ef371e4e405a2cea202282
2013-01-17 01:24:42 +01:00
bb665b9584
camsync: added --all flag
...
and some guessing/discovery for --src.
This option enables camsync to discover the sync handlers on the
source server, and use that information to run all possible syncs.
Change-Id: Ie1f3c309780a168372a674c6c3a18f679a67d804
2013-01-17 00:37:40 +01:00
dd72cd4434
schema: verify no slashes in fileName in FileNameString accessor.
...
Change-Id: I1b02e67144fed1f80123530786af55c74bd64f36
2013-01-13 20:43:55 -08:00
77d044f4b7
schema: add FileTime to guess the file
...
creation time from potential file metadata.
camput: add the -exiftime option that allows
the use of the above for -filenodes and -vivify.
Fixes: http://code.google.com/p/camlistore/issues/detail?id=77
Change-Id: I05d1af3d875bb984e47e06775fd149e5ed0ed7b1
2013-01-10 17:43:05 +01:00
d689b8df88
camget: say the cache dir if verbose
...
Change-Id: Ic9e8fa9379e354d920e8c896fbcd4f3a861dfe20
2013-01-06 10:44:50 -08:00
3e5940a7ff
camget, schema: read chunks more aggressively
...
Change-Id: I31bc3a145fc2f3a51c3cecbaa33872da077e7bd7
2013-01-05 23:12:42 -08:00
ff9492c6cd
camget: buffer the workc channel
...
Change-Id: Ic6bf443c4d91f240b0b86c34a33bf74a1f2b627f
2013-01-05 16:44:43 -08:00
9f6776dddb
camget, httputil: more verbose HTTP debugging
...
Change-Id: I2fc1a1fe0fff99202245d391a8a76e9fab07adc2
2013-01-05 16:44:34 -08:00
39218e3cd5
Merge "Changed auth to take into account not only the credentials, but the requested operation/action too."
2013-01-04 21:09:37 +00:00
12213c058e
Changed auth to take into account not only the credentials,
...
but the requested operation/action too.
This allows to restrict vivify credentials to only upload
(as well as get and stat, because they're needed) to the
blobserver.
Change-Id: Idaed60d1f0d679cb9795ba9a11f094f964774335
2013-01-04 22:17:12 +01:00
c89dbc7130
camget: put a local disk cache in front of the client.
...
a simple localhost test with only 4.4 MB and 4 files drops from 350 HTTP requests to 76.
more work to be done, but this is better.
Change-Id: I66ed67fe95ddd30aeefe742395874b2eabc237c7
2013-01-03 17:55:30 -08:00
fe13c7c712
move camput's statsTransport to httputil.StatsTransport, use it in camget too.
...
Change-Id: Iab48e32ccb8a99b8ec6b058061cff9a5b99e8712
2013-01-02 20:32:13 -08:00
8866e88db0
camget: do 10 at once. better, much pref work remains.
...
Change-Id: Ib2e159a8a290aff249fbec109d4e76099fa0491a
2013-01-02 15:13:22 -08:00
827feaa3ac
camget, client, schema: start of camget --shared support. see flag docs.
...
Change-Id: I5dd43129cb0032821a5913a8f20da0ddb38c63da
2013-01-02 12:55:12 -08:00
edd5cff32a
camget: fix directory restoring
...
Change-Id: I16c433fdda8a3220befd33309b1a31bbb69c0e58
2013-01-02 11:23:00 -08:00
378b854479
camget: use bytes.NewReader instead of bytes.NewBuffer
...
Change-Id: If51c47f356460b32355836e67a1ce443c071b65c
2013-01-02 09:20:35 -08:00
6b7d73d757
camput: ignore atimes
...
Change-Id: Ia55d6fef6a8e3aa95b6265c958853da7b170e3c4
2013-01-01 21:05:02 -08:00
6b466d0faa
camput, client: push havecache down deeper, make it track blob sizes.
...
also, clean up some camput HTTP stuff.
Change-Id: I2762e5a1b83139132b5f660d306ed45584a7799a
2013-01-01 20:23:44 -08:00
f8df0845ea
camput: Don't use statcache when using vivify
...
Change-Id: I2c450afd36efdcd2d0fbd16023435936fbc43397
2013-01-01 13:29:15 -08:00
57a60aa944
camput: flatcache TODO.
...
Change-Id: I4e2368f8e22d5aa71bee8f5319f59444f3c0e638
2012-12-31 13:53:44 -08:00
519c46a388
camput: add FileInfo.Sys fields to stat fingerprint
...
Change-Id: Idc6af6e945c922aacbf2b51dbed70f4f8e20ab2a
2012-12-31 13:52:17 -08:00
93e8a6c5c2
camput: remove debug log
...
Change-Id: I1c335ea4c291fe59b56b3b12a0f50a1fbcbc822d
2012-12-31 13:46:49 -08:00
9bee164972
camput: enabling client caching by default
...
make havecache/statcache named on disk by remote server's generation,
hide debug options in camput unless CAMLI_DEBUG=1 is set,
push havecache down into pkg/client.
Change-Id: Ic152397abc11b4ff3931468ffe93cf1eba62b438
2012-12-31 13:45:13 -08:00
ed20da7592
vivify: support in camput and upload handler.
...
no special credentials/auth yet.
pkg/blobserver/handlers/get.go moved to
pkg/blobserver/gethandler/get.go to avoid
a dependency loop with the json sign helper.
pkg/server/sig.go was moved to pkg/jsonsign/signhandler
because it seemed inapproriate to import in
pkg/blobserver/handlers/upload.go something from
pkg/server
Change-Id: Ifeb14512e182e8a101d4fced6d6d4184e2b9cb99
2012-12-30 23:39:22 +01:00
8560c6d513
camput: move interfaces to their own file
...
Change-Id: I44276ecd1d3474efeb35308638a317605cedf05a
2012-12-28 09:49:18 -08:00
22621b80f6
Work around io.Copy / ioutil.Discard data race. :(
...
Change-Id: I3c2f7fb265ad70aa6df39d7ffe089f9654541dd0
2012-12-26 12:39:33 -08:00
b8dc625560
fs: implement fuse nodeReader.Read in terms of FileReader.ReadAt
...
Change-Id: I136543554ca5337568ec94d8888528220d5f88cc
2012-12-26 10:02:22 -08:00
e3247edafb
Change blobref.SeekerFromStreamingFetcher signature to not return an error.
...
Change-Id: I77f693e3b3d0d116e08bca3d3f4cb45ef2a00b27
2012-12-25 10:27:35 -08:00
826bb388b4
camget: clean up, fix file dumping (to use FileReader), add --contents flag, update docs
...
Change-Id: I1fb2ed8691e24f7cd89e92b9a1ae3afbeb7fb21a
2012-12-25 10:20:14 -08:00
22a4d5e050
camput: more logging
...
Change-Id: I0098d42279dcf6eac07f3cedefd20213bd468b7c
2012-12-24 10:55:00 -08:00
896dba2cc8
camsync, client: check size consistency in sync pass too.
...
Change-Id: I4509f1799bbe578d76b94285309c5ad364311a42
2012-12-23 09:22:25 -08:00
0cc7ec78ed
camsync: verbose logging changes
...
Change-Id: I8169259ce37522b2512e9c1f86841635d7c796f7
2012-12-23 09:00:51 -08:00
0b022b433a
camput, schema, client: query server for dups before uploading, like the JavaScript UI.
...
Change-Id: Id12f78cf2a095adcfad90ca4c6416bb80b6e9b14
2012-12-22 22:48:21 -08:00
b32008ed34
camput, client: start of making camput file check searchRoot for wholedigest dups
...
Change-Id: Ic014ef54705b14482c42936bf6957dc87efa1cf7
2012-12-22 18:42:35 -08:00
a536e2a6d3
Merge "camput: Preparation for vivify."
2012-12-23 00:11:08 +00:00
6cb7ecd644
camput: Preparation for vivify.
...
Refactor writeFileMapRolling so we can write the chunks,
then the top level contents, separately.
Change-Id: Ib56cd4b2aa295516fafbe7072ad5b352d1aaaa89
2012-12-23 01:15:29 +01:00
392a6649da
Make camsync verbose more verbose; echo position every second.
...
Change-Id: I32fbfc53dd45c5c6870ea2aa3ba68babad32c98b
2012-12-22 15:22:22 -08:00
e2d0fd6aa3
camsync: formatting
...
Change-Id: Idd69072bcc0ec38ee5d0cc4dc8c8f7a74581edf3
2012-12-21 16:41:15 -08:00
12ddaab1d5
camput file: use abspath when uploading file
...
Change-Id: Ia42f2c09a6c76d2e5f2b80ad309ab3e98fa17c49
2012-12-20 00:22:29 +01:00
682390dffa
Add a GIF mime type, TODOs, add camdebug mime command.
...
Change-Id: Ic8930d4250dec30ed8c9a056c1cecd70cc447f9d
2012-12-06 17:53:03 -08:00
03a8e7370b
default to localhost auth type when generating config for camlistored and clients
...
Change-Id: I95e74bcde6c1ca564f76e1ed9a916de48c1b6513
2012-11-16 16:49:19 +01:00
9abbbef45e
camput: gofmt
2012-11-07 22:21:52 -06:00
ff60ec2785
Get rid of some old camput config cruft
2012-11-07 21:19:07 -06:00
281a6c40f4
client: correct case of SetHTTPClient
2012-11-07 21:03:46 -06:00
9b35026497
camput: rename tinkerTransport to statsTransport
2012-11-07 21:00:22 -06:00
295cbade1b
Add postgres support
...
Also added the -short option for make presubmit
because postgres test is too slow.
Change-Id: I6be21c4d4cd67671eb44ab65333fc32c1e261bbd
2012-11-03 19:59:22 +01:00
22495f8bfd
schema: adjust rollsum splitting algorithm, make it the default
...
We might tweak the parameters yet, and I want to add a bunch more
tests, but this is a major relief for me. I've been reluctant to
put too much data into my personal Camlistore instance until I
figured out how I wanted the file chunks to split.
This also simplifies (deletes) a fair bit of code.
Change-Id: I35c38cc5d39a8a43e3f62445434f8d7fd5de5d17
2012-10-29 02:03:05 +01:00
0ddc9dcadb
camget: add a graphviz output mode with -graph
...
Change-Id: I472a1e7fda0f1a15bdcba1a7be8e0d586741bf08
2012-10-28 19:32:25 +01:00
f91f1f88c4
camput: final error printing
...
Change-Id: Icc51560545b20315a0383d7bda80170c5612e1f2
2012-10-24 16:02:37 +02:00
202c45a04a
camput file:
...
This change introduces 2 features, illustrated by those examples:
1) camput file -filenodes *.jpg
This was not previously possible because -filenodes option only worked
with directories.
2) camput file -filenodes -tag 'nice pics' /media/foo/DCIM/
This was not previously possible because the -tag option could only be
used with the -permanode option.
Change-Id: I80ed95fdc819d359cacfe97ba3b7849b96e406f4
2012-10-22 19:25:56 +02:00
6be58c9ce0
camput: start of proper tests
...
Change-Id: Icc9257154fecaaf2018971ab94e0dc54086d3719
2012-10-21 14:14:51 -07:00
ef8e746cb5
merge
...
Change-Id: I06ab89274e35bad61944e208e366cd7436381e63
2012-10-21 13:59:13 -07:00
e9b5bca10e
camput: start of making it more testable
...
Change-Id: Ie65afeaa14ea483231674433cda5a90669ca0bc7
2012-10-21 13:58:06 -07:00
159c2a5216
camput: moved/added examples, added -help as a global flag to display usage
...
Change-Id: I3de13057c555d68752101043f671d7694a311021
2012-10-19 17:57:17 +02:00
a534e02c2d
camput: client becomes StatReceiver.
...
also added ReaderSize function to
guess before reading the size of
a source.
Change-Id: I659e174821696b3c58759b132b82710a49583018
2012-10-11 11:51:12 +02:00
228c0ca56f
schema: change PopulateSymlinkMap to be Map.SetSymlinkTarget
...
Change-Id: I072ce65331048b54c90c32e951f5034060c1764c
2012-08-22 04:38:25 +10:00
439db110ec
schema: more cleanup; ditch the Claimer type
...
Change-Id: I2a52564c1a99c96a2cd8bc84a791784f0a4e4b2e
2012-08-22 04:31:48 +10:00
11eac8721e
Convert more code to use schema.Map instead of map[string]interface{}
...
Change-Id: I285fd1b0e730ebd32069688a71cb992c43770ade
2012-08-22 03:47:38 +10:00
3cd7f75f89
schema: introduce type Map
...
Change-Id: I8209930d8505a4cb74b8ea29b5b57f2cd6bd14d2
2012-08-22 03:36:36 +10:00
0a0b012eba
camput,schema: cleanup, docs
...
Change-Id: Ifa94cb1e9e086f85222692508ab12616f4cc3211
2012-08-22 03:31:44 +10:00
dc684dfa6a
camput: updated comments, TODO
...
Change-Id: I8df73d4c1c44c71424bcfaf7d3cb124c1738c791
2012-08-22 02:39:36 +10:00
8fec29c974
camput: make file permanodes consistent between runs (fix claim times)
...
Change-Id: Idd2bbda2372802772703eaeaf6fe2e74f9bcdd83
2012-08-21 20:43:38 +10:00
b815a2b758
camput: add -filenodes option for per-file content-based planned permanodes
...
Change-Id: Ic03b990654d860e03b4cc710de92f7d40c392df8
2012-08-21 16:11:10 +10:00
51e88cac65
Support for "planned permanodes" in pkg/jsonsign, pkg/schema and cmd/camput.
...
A planned permanode involves setting the contents of the permanode
(instead of a random string) as well as the OpenPGP signing time so
the resultant bytes of the blob (and thus its blobref) is deterministic.
This allows multiple independent devices (my laptops) to create the
same permanodes for the same files (photos backed up from my phone)
when offline (airplane) and then when they sync later, still only have
one permanode per unique file. This means that tagging and other
metadata applied to permanodes on one laptop merge cleanly with
metadata from the other.
2012-07-28 16:32:31 -07:00
8c293e34b6
Per Go initialism style, rename Json to JSON
2012-07-28 15:42:56 -07:00
326d62aea4
camput: actually catch errors when trying to upload,
...
also added startupcheck for remote blobserver
Change-Id: Id9f5383c6fd3fdf697f2b75052f3e63b0972205d
2012-07-23 11:05:57 +02:00
e3c0d11011
camput share, argument bugfix
...
Change-Id: Ic1bc9889b50dc69509810a619f34b4896e68587b
2012-05-13 15:55:12 +02:00
4b8df05859
fs,fuse: new 'welcome' root node, where you can navigate/search anywhere.
...
Change-Id: I66f076a85d8e474bb5d93ad8743fc8181de7502f
2012-04-29 14:29:51 +10:00
f19c1f9026
cammount: fix fmt pattern
...
Change-Id: I8bb3df8a14cd3d3b44c899f3e2754b50b35b0028
2012-04-26 19:34:29 -07:00
5315bbf232
camput: re-do the stat/have flatcaches.
...
still waiting on leveldb-go, but this works for now.
Change-Id: I630d7bd2757a02d3a711c48075486c5cc6339118
2012-04-22 17:18:19 -07:00
b4bff6a7d9
camput: check decode errors, exposes bugs in flat cache
...
Change-Id: I5fd812300c0c92892447a57d1da49cceedca9e3f
2012-04-21 08:54:26 -07:00
df339b446a
camput: show blob stats during upload too
...
Change-Id: I988e5e12cfee355dce0170165c42cb4d162dc259
2012-04-21 08:30:04 -07:00
da42bdf523
camput: more code movement
...
Change-Id: I775096c39b5c9671a3c054b7d02562cfa787f0a1
2012-04-21 07:22:32 -07:00
e6e26f2017
camput: move common uploader stuff to uploader.go
...
Change-Id: I6137a945f89f4a0603189a5fc04b49a570a3403b
2012-04-21 07:14:57 -07:00
cd79828912
camput: move nodeworker into its own file
...
Change-Id: Ic61d35955fe4dd59ada65a314f0c94cc3f7b0f87
2012-04-21 07:07:07 -07:00
2274bd97b1
camput: bunch of cleanups and bugs and usability stuff.
...
Change-Id: I029cc38d5b10a2d322e26148eb4971b3c838c889
2012-04-20 14:43:23 -07:00
0d989a6ae6
camput: fix up statcache to work with Go 1. crashed before with gob error on os.FileInfo.
...
Change-Id: I9b88435e00352988e5f89e64f45ea4cb3d2c76ab
2012-04-20 14:27:52 -07:00
9168eb84f0
camput: fix logging bug
...
Change-Id: I8781b523013dca814b81f99fed85ba59a489862c
2012-04-20 14:03:09 -07:00
5005dcf751
camput: fix statcache/havecache flag mix-up.
...
Change-Id: Ie7caea7841d055ed47774d0279095f218740ab5a
2012-04-20 13:05:34 -07:00
0baa073b44
camput: switch to new tree upload code.
...
Change-Id: Iae653e4442bab63979d584caecdce58df942385d
2012-04-19 17:59:41 -07:00
0f3ce52c54
camput: memoize SumBytes
...
Change-Id: Ie0ea05a895f97de2b069c088d4195b533b70ea8e
2012-04-18 07:24:45 -07:00
7e94879e86
camput: in du mode, only run 1 upload worker.
...
Change-Id: I65b642095ca9f162f14db512f34bb7073ef7a13b
2012-04-17 17:46:05 -07:00
715c5278e0
camput: working diskusage mode.
...
Change-Id: I7c3df20fed7720e3535b5e618d20842d1ddaadcc
2012-04-15 17:21:19 -07:00
4f0f53a90e
camput: more TreeUpload work; adds NodeWorker helper.
...
Change-Id: Ie5919faa34fd2f6eb1b61669598e0a06e1d8ba07
2012-04-15 17:11:25 -07:00
69ce6654e8
camput: fix bug so default switch case can be hit now.
...
otherwise all other files were hitting the not-directory case.
Change-Id: I03e6021e947eee301e21db3f20c2da3c2f92ab2d
2012-04-15 14:25:41 -07:00
92b10a39a4
camput: remove dead flag
...
Change-Id: Idf6ec9416b5878379de19b53a93b86982125f7cb
2012-04-15 14:21:38 -07:00
d05a336cb0
camput: move more blob-only stuff to blobs.go
...
Change-Id: I967e9dc975db18088b88c00b2672353a5a72f458
2012-04-15 14:21:00 -07:00
90c254038c
camput: remove dead code
...
Change-Id: Ifab0247b6c23d050abd822b479558a0d75930b8b
2012-04-15 14:10:08 -07:00
77fc9e8abb
camput: only support '-' meaning stdin for the 'blob' subcommand, not 'file'
...
Change-Id: I9a8923792385e232dc42c7707c1b748f82a5ac37
2012-04-15 14:01:39 -07:00
88cff46f28
camput: more TreeUpload work
...
Change-Id: I8638ccdc342642447661e68918259878e04ef756
2012-04-15 13:54:21 -07:00
61a895b1f8
camput: some more TreeUpload work
...
Change-Id: I2b4e74f7892720bbf6b544a7abb9867d5ad3415b
2012-04-15 11:26:36 -07:00
f366dfb850
some TreeUpload TODO notes
...
Change-Id: If26ddcb6ee8ebc1b35f569ecbf835da5f666c3a7
2012-04-15 08:55:11 -07:00
72ac3fc8b1
camput: TODOs for VFS
...
Change-Id: I9b403ec5e3e66f150da4277eabe99e079c61d2b8
2012-04-15 08:28:18 -07:00
683e762c79
camput: go through virtual filesystem. will be good for filtering skipDirs & testing.
...
Change-Id: I2fa56615b6b199ab054de388dde652749049d5e1
2012-04-15 08:11:37 -07:00
147cdb05c9
camput: unindent
...
Change-Id: I151b136a97a129a75f3f83fca8f1be841f7bec2b
2012-04-15 07:47:38 -07:00
18e53cd72f
rename MapToCamliJson to Go style initialism MapToCamliJSON
...
Change-Id: I674944c9fafa6b78c495d22c525b9ffd0b75dbf9
2012-04-15 07:45:10 -07:00
a8687d112c
camput: move rollSplits to Uploader field, not passed along.
...
Change-Id: I8e78fe88e179235a03117927bd7b36217242e055
2012-04-15 07:18:40 -07:00
91501cdbd1
remove unused code
...
Change-Id: I4e35ce6dbb34f3db17024f49dd98bed7d3f9b4f0
2012-04-15 07:10:10 -07:00
77f9544936
gofmt
...
Change-Id: If71fa78e10464bc93fc7ec748e73d935d3c555ba
2012-04-14 22:04:00 -07:00
5b379561a7
Fix bogus r60->go1 gofix.
...
Change-Id: Ieb229a6767dbb955b898006d10926d8715bbe307
2012-04-14 22:03:54 -07:00
6f3826f6af
camput: flag for diskUsage mode
...
Change-Id: I9a8bdd907ddcb9debb8381c11691a7485ecba6c2
2012-04-14 22:01:19 -07:00
f8f95554d5
camput: fix panic on errors.
...
I think maybe I should just change handleResult to panic with its own error type and catch it above.
Change-Id: Ib0622321e29e7926173c4957dc0e56f3b514e7d2
2012-04-12 22:18:38 -07:00
e4a17bc8ac
camput: fix init and some config stuff.
...
Change-Id: I121bd9c2dfac3c076db0db75d8f939b5ec571ca4
2012-04-12 19:36:00 -07:00
c6e6f7a92d
camdb: gofmt and change panic to log.Fatalf
...
Change-Id: I8c8ced3cb5f74a7742a2a9f86874276b3c1fd0c2
2012-04-11 19:13:39 -07:00
9447a8cb78
camput: reorder file type switch; fixes symlink bug
...
Change-Id: Ic4d0f6c2be02d153f933d72c1a02e72926c36786
2012-03-27 10:41:34 -07:00
7f90db0291
rename mysqlindexer to mysql to match import
...
Change-Id: I23cd9cfc40a0e01cb0086fc5c9a3ad108f3705a8
2012-03-27 09:37:32 -07:00
69328c751f
fix camdbinit; use correct mysqlindexer import
...
Change-Id: I227e219ec32f81a11cc6192cd60de8b89878bedc
2012-03-27 09:17:26 -07:00
10cf23c700
Get the MySQL indexer working again.
...
It now uses the generic indexing layer, rather than its own schema.
Also, move it under pkg/index/, like mongo.
Also, link in mongo to the main binary.
Not using the tests yet (leaving for mpl :))
Change-Id: I47a2203ca6c0b5244cd458aedec46227c4363c62
2012-03-26 13:57:53 -07:00
899f8a0e7a
move camdbinit to cmd/
...
Change-Id: I21657e1cc91bda7a7b18b609b1bfac37680256cb
2012-03-24 19:46:38 -07:00
374c1f5360
cammount: move to cmd/ directory; update dev-cammount script.
...
Change-Id: Ifa0c4d586c30ebb807a8094594e4872dcd5039ef
2012-03-18 23:58:36 -07:00
a229abeebd
camget: fix build. update dev-camget.
...
Change-Id: I5289801de7b50caedeabd9cc2513d768aafd424b
2012-03-05 08:04:42 -08:00
c825acde06
move camdebug to cmd; it compiles as Go 1 already.
...
Change-Id: Ic9aa1112a504da96738b7ec3fc224df54b92cdc8
2012-02-21 02:14:34 -08:00
0714a463c9
Update from r60 to [almost] Go 1.
...
A lot is still broken, but most stuff at least compiles now.
The directory tree has been rearranged now too. Go libraries are now
under "pkg". Fully qualified, they are e.g. "camlistore.org/pkg/jsonsign".
The go tool cannot yet fetch from arbitrary domains, but discussion is
happening now on which mechanism to use to allow that.
For now, put the camlistore root under $GOPATH/src. Typically $GOPATH
is $HOME, so Camlistore should be at $HOME/src/camlistore.org.
Then you can:
$ go build ./server/camlistored
... etc
The build.pl script is currently disabled. It'll be resurrected at
some point, but with a very different role (helping create a fake
GOPATH and running the go build command, if things are installed at
the wrong place, and/or running fileembed generators).
Many things are certainly broken.
Many things are disabled. (MySQL, all indexing, etc).
Many things need to be moved into
camlistore.org/third_party/{code.google.com,github.com} and updated
from their r60 to Go 1 versions, where applicable.
The GoMySQL stuff should be updated to use database/sql and the ziutek
library implementing database/sql/driver.
Help wanted.
Change-Id: If71217dc5c8f0e70dbe46e9504ca5131c6eeacde
2012-02-18 21:53:06 -08:00
mpl