Rooba
/
perkeep
mirror of https://github.com/perkeep/perkeep.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,288 Commits 44 Branches 11 Tags 59 MiB
Commit Graph

70 Commits

Author SHA1 Message Date
Brad Fitzpatrick d2bc6f6cd3 fs: use schema.DirReader, rather than doing it by hand 
Change-Id: I7e79621d621314ed1f50219f0cafc44b1b82ab1f
 2013-02-18 10:17:38 -08:00
Brad Fitzpatrick 9efdf83d3e schema: use better types in superset: blobref and Time3339 
Change-Id: I7004220789e1048c1ebcc8703bf2948a733bae10
 2013-02-10 12:58:51 -08:00
Brad Fitzpatrick 0309c22285 schema: work-around fix for Issue 97/100 until 102 is fixed 
Change-Id: Ied51ffe2b50acc390d068696ebbe88e60061ef91
 2013-02-07 17:12:08 -08:00
Brad Fitzpatrick 16c70de194 schema: fix hang in WriteFileChunks, used by vivify & the Android app 
Change-Id: I6099633a300119ef3f0e47266bb6196c475b6378
 2013-02-03 22:28:11 -08:00
Brad Fitzpatrick 9eed2f6082 schema: remove FileWriter's serial byte/file schema uploads after chunks are uploaded 
Change-Id: I1c3a9c496ce168cc8b6d087ef1a0de99cac22127
 2013-02-02 22:35:08 -08:00
Brad Fitzpatrick 6a4eff5b84 schema: make serverHasBlob just use blobserver.StatBlob 
Change-Id: I3012763660c9ed87eb48b2d95b8265c3c77f32df
 2013-02-02 21:50:03 -08:00
Brad Fitzpatrick 9d339355f0 schema: make FileWriter upload different chunks in parallel 
Change-Id: I4608a21ffc45324512a696dfe070b38416aae130
 2013-02-02 20:17:14 -08:00
Brad Fitzpatrick f266b843ed schema: simplify WriteFileChunks 
Change-Id: Icf9077db0adb5081b93f89ddc9d74a4cc45f20e1
 2013-02-02 19:34:23 -08:00
Brad Fitzpatrick ebb6fc85c2 schema: simplify uploadBytes signature 
Change-Id: I508934ab83b25ef3a066de9128e16a18a7d82dd3
 2013-02-02 10:58:47 -08:00
mpl 83fa932f3c signhandler: inject the claimdate (if present) in the signrequest 
This allows to set a signature time (instead of using time.Now()),
so that identical files will result in identical permanodes.

Change-Id: I315c06bd30e8ac5e4d36b0e3f758483c22a31063
 2013-02-02 02:48:00 +01:00
Brad Fitzpatrick f1f670079b server.vivify: use the right claim date on all 3 relevant dates. 
Fixes https://code.google.com/p/camlistore/issues/detail?id=93

Change-Id: I9d9ab02d47f2ef78f6f690414a04fc5c24f95fac
 2013-01-31 17:52:11 -08:00
Brad Fitzpatrick 070592444a schema: unexport Superset 
Change-Id: If45569b1b6d72b1cb6450b9d634f59e88fae4851
 2013-01-22 10:32:15 -08:00
Brad Fitzpatrick af2b2987c5 pkg/fs: stop using Superset 
Change-Id: I890b2376124de129f8035af5b9424f3ba3861895
 2013-01-22 10:20:34 -08:00
Brad Fitzpatrick f9ac0ab109 pkg/client: stop using Superset for sharing 
Change-Id: I4f034b73584413d3d37f399680c67330783951b2
 2013-01-22 09:52:01 -08:00
Brad Fitzpatrick cfc32e4a05 schema, camget: more work on deleting the Superset type. 
not much more remains.

Change-Id: I6cfe4145f67b100a0e2509f88ce6e1c580b7f9fe
 2013-01-22 09:32:40 -08:00
Brad Fitzpatrick 1bf01d7315 schema: lot of Blob/Builder work, and kill schema.Map. 
Next up: unexport schema.Superset.

Change-Id: Ia4bb34790abba75bdb4d4101df7a4f15875f4237
 2013-01-21 20:56:12 -08:00
Brad Fitzpatrick 02e3c08947 schema: start of eliminating/unexporting schema.Superset and schema.Map 
For now, unfortunately, there are even more types. The old ones will hopefully die soon.

Change-Id: I9d0bb1743c94a0de429175ef0b05ad6a964e07c0
 2013-01-21 17:31:08 -08:00
Brad Fitzpatrick 7ceaaa0012 blobref: simplify the FromHash func. Make type implicit. 
Change-Id: I2e01c3663bdb1151c11dfc9a1d59c7081940ffac
 2013-01-20 13:36:27 -08:00
Brad Fitzpatrick dd72cd4434 schema: verify no slashes in fileName in FileNameString accessor. 
Change-Id: I1b02e67144fed1f80123530786af55c74bd64f36
 2013-01-13 20:43:55 -08:00
mpl 77d044f4b7 schema: add FileTime to guess the file 
creation time from potential file metadata.
camput: add the -exiftime option that allows
the use of the above for -filenodes and -vivify.

Fixes: http://code.google.com/p/camlistore/issues/detail?id=77

Change-Id: I05d1af3d875bb984e47e06775fd149e5ed0ed7b1
 2013-01-10 17:43:05 +01:00
Brad Fitzpatrick bdb54d7cef schema: various FileReader optimizations and bug fixes 
Now much faster.

Change-Id: I94920b7ab6903e986b045a6ed862eff0796a7f7e
 2013-01-06 10:50:41 -08:00
Brad Fitzpatrick e95e6db11d schema: actually populate the superset cache map. whoops. 
Change-Id: I461c789bdf01a3341867faa30cac07e645c510e5
 2013-01-06 08:53:29 -08:00
Brad Fitzpatrick 3e5940a7ff camget, schema: read chunks more aggressively 
Change-Id: I31bc3a145fc2f3a51c3cecbaa33872da077e7bd7
 2013-01-05 23:12:42 -08:00
Brad Fitzpatrick a0b7d2e0b2 schema: use singleflight for FileReader superset loading 
Change-Id: Ia3db1a3521ea805440f6ceb884c4f4e6ed76f409
 2013-01-05 20:07:38 -08:00
Brad Fitzpatrick 3cd8af2c7d schema: more incremental work on making FileReader faster 
Change-Id: I033df1654a062dfa57dfe0afce7030f605e88450
 2013-01-04 20:20:41 -08:00
Brad Fitzpatrick e8ab934464 schema: start of more efficient/aggressive file reading 
Change-Id: I027881c9844b0df3d70c1e05da69b04d853da883
 2013-01-04 18:11:40 -08:00
Brad Fitzpatrick 3a8b9078ec schema: delete the old FileReader.Skip method 
Change-Id: I025bee0d2a4cc8b2cae4e825d0c3b37f14ee8b57
 2013-01-04 16:29:16 -08:00
Brad Fitzpatrick 971abb5566 schema: tighten the definition of schema.File 
Change-Id: I14212e55fec217953c6a0979f910e5fa93ee6825
 2013-01-04 16:26:14 -08:00
Brad Fitzpatrick 997e52d6b1 schema: split DirReader off into its own file 
Change-Id: I69fcbe43912f37cc32567b3d46cabe8579ae00de
 2013-01-03 17:16:26 -08:00
Brad Fitzpatrick 2fca641619 schema: minor cleanups 
Change-Id: I7565ccc925e6ae816799242310e26d604a4c42ae
 2013-01-03 17:14:23 -08:00
Brad Fitzpatrick 8e44c62047 camget: finish --shared fetching support 
Change-Id: I32edf63e01068a0e96f3255ba4d1313682cf03c4
 2013-01-02 14:50:52 -08:00
Brad Fitzpatrick 827feaa3ac camget, client, schema: start of camget --shared support. see flag docs. 
Change-Id: I5dd43129cb0032821a5913a8f20da0ddb38c63da
 2013-01-02 12:55:12 -08:00
Brad Fitzpatrick 898e522126 Close FileReaders. Hunting an fd leak, but this isn't it, 
since FileReader.Close is back to doing nothing.

Change-Id: I65e906d75cf2825b9476ed5008ce042f44582113
 2012-12-31 18:02:13 -08:00
mpl ed20da7592 vivify: support in camput and upload handler. 
no special credentials/auth yet.
pkg/blobserver/handlers/get.go moved to
pkg/blobserver/gethandler/get.go to avoid
a dependency loop with the json sign helper.
pkg/server/sig.go was moved to pkg/jsonsign/signhandler
because it seemed inapproriate to import in
pkg/blobserver/handlers/upload.go something from
pkg/server

Change-Id: Ifeb14512e182e8a101d4fced6d6d4184e2b9cb99
 2012-12-30 23:39:22 +01:00
Brad Fitzpatrick 08f09ebf2b schema: fix FileReader bugs, simplify, add exhaustive tests 
Change-Id: I5d098d7b12892d278190391dd5abd7272ed997c2
 2012-12-25 17:27:52 -08:00
Brad Fitzpatrick 0a0cbe72c1 schema: start of more stressful FileReader tests, chasing a bug 
Change-Id: I2aa2b815c785d37674c309a9d6b7c26c67ec3b6e
 2012-12-25 12:17:45 -08:00
Brad Fitzpatrick 0b022b433a camput, schema, client: query server for dups before uploading, like the JavaScript UI. 
Change-Id: Id12f78cf2a095adcfad90ca4c6416bb80b6e9b14
 2012-12-22 22:48:21 -08:00
mpl 6cb7ecd644 camput: Preparation for vivify. 
Refactor writeFileMapRolling so we can write the chunks,
then the top level contents, separately.

Change-Id: Ib56cd4b2aa295516fafbe7072ad5b352d1aaaa89
 2012-12-23 01:15:29 +01:00
Brad Fitzpatrick 22495f8bfd schema: adjust rollsum splitting algorithm, make it the default 
We might tweak the parameters yet, and I want to add a bunch more
tests, but this is a major relief for me.  I've been reluctant to
put too much data into my personal Camlistore instance until I
figured out how I wanted the file chunks to split.

This also simplifies (deletes) a fair bit of code.

Change-Id: I35c38cc5d39a8a43e3f62445434f8d7fd5de5d17
 2012-10-29 02:03:05 +01:00
Brad Fitzpatrick 044194b746 schema: document PopulateParts, add a missing error check 
Change-Id: Ic9d65fe3e56c4233f481254c6f730a8b382fb4ba
 2012-10-29 00:10:31 +01:00
mpl 83ce85b59f old TODOs 
Change-Id: Ifa8877f594e65e081536b97f536106301f5c9836
 2012-10-19 23:37:14 +02:00
Brad Fitzpatrick e557066f1d schema: more cleanup, making FileReader a ReaderAt. 
testing TODOs and further cleanup TODOs remain.

Change-Id: I997153c66805cfe3220d3d735322be14b68b75dc
 2012-08-23 20:09:31 -07:00
Brad Fitzpatrick faf3c90d6f schema: more docs and some FileReader work 
Change-Id: Id9e754e3b6358a651a826a4cdeb00efb330e3c3a
 2012-08-23 18:44:03 -07:00
mpl b355a29150 just one missing map[string]interface{} -> Map conversion 
Change-Id: I616ef7260dac9187ea5157f8d47c1b05815ad723
 2012-08-22 15:09:42 +02:00
Brad Fitzpatrick 1a51f44196 schema: fix test 
Change-Id: Iaacc42167e40233dcb9f5f03bfcfcfe86655a2b1
 2012-08-22 04:38:51 +10:00
Brad Fitzpatrick 228c0ca56f schema: change PopulateSymlinkMap to be Map.SetSymlinkTarget 
Change-Id: I072ce65331048b54c90c32e951f5034060c1764c
 2012-08-22 04:38:25 +10:00
Brad Fitzpatrick a8629ec5dd schema: unexport NewBytes 
Change-Id: Ia6ec7c9aec33319bf277b93a189b4fa43d0540b9
 2012-08-22 04:33:06 +10:00
Brad Fitzpatrick 439db110ec schema: more cleanup; ditch the Claimer type 
Change-Id: I2a52564c1a99c96a2cd8bc84a791784f0a4e4b2e
 2012-08-22 04:31:48 +10:00
Brad Fitzpatrick b67d753364 schema: docs and cleanup 
Change-Id: Ib720b7f711d71bdeaaf33374f376f14b30a5a97c
 2012-08-22 04:14:47 +10:00
Brad Fitzpatrick 11eac8721e Convert more code to use schema.Map instead of map[string]interface{} 
Change-Id: I285fd1b0e730ebd32069688a71cb992c43770ade
 2012-08-22 03:47:38 +10:00