Rooba
/
perkeep
mirror of https://github.com/perkeep/perkeep.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,211 Commits 44 Branches 11 Tags 59 MiB
Commit Graph

2211 Commits

Author SHA1 Message Date
Brad Fitzpatrick a0b7d2e0b2 schema: use singleflight for FileReader superset loading 
Change-Id: Ia3db1a3521ea805440f6ceb884c4f4e6ed76f409
 2013-01-05 20:07:38 -08:00
Brad Fitzpatrick 056d1efb26 httputil: more Transport debugging 
Change-Id: If3ea30237abf8614d41bb340c24a65f7725234c2
 2013-01-05 17:17:56 -08:00
Brad Fitzpatrick 906388d91a get handler: remove old sniffing for demos which no longer works and makes things much slower 
Change-Id: I1d9e7368881894cba9a1febab6c5efcf765b0d59
 2013-01-05 17:17:38 -08:00
Brad Fitzpatrick ff9492c6cd camget: buffer the workc channel 
Change-Id: Ic6bf443c4d91f240b0b86c34a33bf74a1f2b627f
 2013-01-05 16:44:43 -08:00
Brad Fitzpatrick 9f6776dddb camget, httputil: more verbose HTTP debugging 
Change-Id: I2fc1a1fe0fff99202245d391a8a76e9fab07adc2
 2013-01-05 16:44:34 -08:00
Brad Fitzpatrick 6f9f0bdda9 singleflight: open soure this package. 
I wrote this for servers at Google but forgot I hadn't
open sourced it yet.  Maybe I should move this out of Camlistore
in the future, but putting it here for now.

Change-Id: I9120b2bc9c4a1be34d8166f00fb1933318db9783
 2013-01-05 16:23:53 -08:00
Brad Fitzpatrick 3cd8af2c7d schema: more incremental work on making FileReader faster 
Change-Id: I033df1654a062dfa57dfe0afce7030f605e88450
 2013-01-04 20:20:41 -08:00
Brad Fitzpatrick 7791d5bc27 Update template header's copyright date to 2013. 
Change-Id: I5c5ea1bb78473a23c622c4ce12af974e40001ffe
 2013-01-04 19:29:34 -08:00
Brad Fitzpatrick e8ab934464 schema: start of more efficient/aggressive file reading 
Change-Id: I027881c9844b0df3d70c1e05da69b04d853da883
 2013-01-04 18:11:40 -08:00
Brad Fitzpatrick 75a0361165 cacher: add missing error checks 
Change-Id: I4cd13d3c971d23b2ed226897928ec47dc698c366
 2013-01-04 17:05:20 -08:00
Brad Fitzpatrick 3a8b9078ec schema: delete the old FileReader.Skip method 
Change-Id: I025bee0d2a4cc8b2cae4e825d0c3b37f14ee8b57
 2013-01-04 16:29:16 -08:00
Brad Fitzpatrick 971abb5566 schema: tighten the definition of schema.File 
Change-Id: I14212e55fec217953c6a0979f910e5fa93ee6825
 2013-01-04 16:26:14 -08:00
mpl 2a961cfc75 auth: add OpSign operation, and fix sigserver with it 
Change-Id: Ie1f2b15ee72cf571684d12d8c8743a7164d1d80d
 2013-01-04 23:09:55 +01:00
mpl 39218e3cd5 Merge "Changed auth to take into account not only the credentials, but the requested operation/action too." 2013-01-04 21:09:37 +00:00
mpl 12213c058e Changed auth to take into account not only the credentials, 
but the requested operation/action too.

This allows to restrict vivify credentials to only upload
(as well as get and stat, because they're needed) to the
blobserver.

Change-Id: Idaed60d1f0d679cb9795ba9a11f094f964774335
 2013-01-04 22:17:12 +01:00
Brad Fitzpatrick 0333cc093e more TODO 
Change-Id: I0c8535256b5cab6fd3bdca4ce4007dae01d59f7c
 2013-01-03 17:57:10 -08:00
Brad Fitzpatrick c89dbc7130 camget: put a local disk cache in front of the client. 
a simple localhost test with only 4.4 MB and 4 files drops from 350 HTTP requests to 76.

more work to be done, but this is better.

Change-Id: I66ed67fe95ddd30aeefe742395874b2eabc237c7
 2013-01-03 17:55:30 -08:00
Brad Fitzpatrick 997e52d6b1 schema: split DirReader off into its own file 
Change-Id: I69fcbe43912f37cc32567b3d46cabe8579ae00de
 2013-01-03 17:16:26 -08:00
Brad Fitzpatrick 2fca641619 schema: minor cleanups 
Change-Id: I7565ccc925e6ae816799242310e26d604a4c42ae
 2013-01-03 17:14:23 -08:00
Brad Fitzpatrick 09a636caa8 dev-camget: don't pass --blobserver argument if using --shared 
Change-Id: I9198c251b34ba06e03c78653eb83772044441c35
 2013-01-02 20:46:49 -08:00
Brad Fitzpatrick 1053ea5847 more TODO about deleting 
Change-Id: Ib8cb0e815b35f078c306a280bd926a11e16f5620
 2013-01-02 20:41:26 -08:00
Brad Fitzpatrick 4a464e334f more TODO 
Change-Id: I1a3845191c9b17350a11bb671b131d1c4dc8c6bc
 2013-01-02 20:37:37 -08:00
Brad Fitzpatrick fe13c7c712 move camput's statsTransport to httputil.StatsTransport, use it in camget too. 
Change-Id: Iab48e32ccb8a99b8ec6b058061cff9a5b99e8712
 2013-01-02 20:32:13 -08:00
Brad Fitzpatrick 4e6740eac5 Add TODO 
Change-Id: Icbce2e60befb2f7a63010b01039b79d10c10e422
 2013-01-02 20:10:23 -08:00
Brad Fitzpatrick 8866e88db0 camget: do 10 at once. better, much pref work remains. 
Change-Id: Ib2e159a8a290aff249fbec109d4e76099fa0491a
 2013-01-02 15:13:22 -08:00
Brad Fitzpatrick 8e44c62047 camget: finish --shared fetching support 
Change-Id: I32edf63e01068a0e96f3255ba4d1313682cf03c4
 2013-01-02 14:50:52 -08:00
Brad Fitzpatrick 827feaa3ac camget, client, schema: start of camget --shared support. see flag docs. 
Change-Id: I5dd43129cb0032821a5913a8f20da0ddb38c63da
 2013-01-02 12:55:12 -08:00
Brad Fitzpatrick 3d748363f7 pkg/client: don't use config file auth when using an explicit server 
Change-Id: I07e3080adf4fed13deb3ec5a2ee900f4fc3a77fe
 2013-01-02 11:42:39 -08:00
Brad Fitzpatrick edd5cff32a camget: fix directory restoring 
Change-Id: I16c433fdda8a3220befd33309b1a31bbb69c0e58
 2013-01-02 11:23:00 -08:00
Brad Fitzpatrick 378b854479 camget: use bytes.NewReader instead of bytes.NewBuffer 
Change-Id: If51c47f356460b32355836e67a1ce443c071b65c
 2013-01-02 09:20:35 -08:00
Brad Fitzpatrick 6b7d73d757 camput: ignore atimes 
Change-Id: Ia55d6fef6a8e3aa95b6265c958853da7b170e3c4
 2013-01-01 21:05:02 -08:00
Brad Fitzpatrick 6b466d0faa camput, client: push havecache down deeper, make it track blob sizes. 
also, clean up some camput HTTP stuff.

Change-Id: I2762e5a1b83139132b5f660d306ed45584a7799a
 2013-01-01 20:23:44 -08:00
Brad Fitzpatrick f8df0845ea camput: Don't use statcache when using vivify 
Change-Id: I2c450afd36efdcd2d0fbd16023435936fbc43397
 2013-01-01 13:29:15 -08:00
Brad Fitzpatrick caa50142dc sync: fix fd leak, update a TODO 
Change-Id: I772ac951723300e4e5e767d7a25d6d4bcda3e825
 2012-12-31 18:21:50 -08:00
Brad Fitzpatrick 898e522126 Close FileReaders. Hunting an fd leak, but this isn't it, 
since FileReader.Close is back to doing nothing.

Change-Id: I65e906d75cf2825b9476ed5008ce042f44582113
 2012-12-31 18:02:13 -08:00
Brad Fitzpatrick 4ce1263225 more TODO 
Change-Id: I3aff3b6c1fea1ecc6937598b0bce38d126135c0a
 2012-12-31 17:50:01 -08:00
Brad Fitzpatrick 1a8eee298a Misc cleanups and TODOs 
Change-Id: I9903366427e41ebadba289bf66b294690675f78f
 2012-12-31 17:36:28 -08:00
Brad Fitzpatrick e01b19fb32 upload handler: remove r60->go1 notes 
Change-Id: I92ab4406890d2aebe957d168aa1c1032ab99f191
 2012-12-31 17:21:00 -08:00
Brad Fitzpatrick 57a60aa944 camput: flatcache TODO. 
Change-Id: I4e2368f8e22d5aa71bee8f5319f59444f3c0e638
 2012-12-31 13:53:44 -08:00
Brad Fitzpatrick 519c46a388 camput: add FileInfo.Sys fields to stat fingerprint 
Change-Id: Idc6af6e945c922aacbf2b51dbed70f4f8e20ab2a
 2012-12-31 13:52:17 -08:00
Brad Fitzpatrick 93e8a6c5c2 camput: remove debug log 
Change-Id: I1c335ea4c291fe59b56b3b12a0f50a1fbcbc822d
 2012-12-31 13:46:49 -08:00
Brad Fitzpatrick 9bee164972 camput: enabling client caching by default 
make havecache/statcache named on disk by remote server's generation,
hide debug options in camput unless CAMLI_DEBUG=1 is set,
push havecache down into pkg/client.

Change-Id: Ic152397abc11b4ff3931468ffe93cf1eba62b438
 2012-12-31 13:45:13 -08:00
Brad Fitzpatrick 9605d3e950 client: add StorageGeneration accessor 
Change-Id: Ie61a5386f433659421cdf5cc89d216b654f5b0fb
 2012-12-31 11:36:58 -08:00
Brad Fitzpatrick 681b16c556 Merge branch 'master' of ssh://camlistore.org:29418/camlistore 2012-12-31 11:36:29 -08:00
Brad Fitzpatrick 6cd12e9372 Merge "client: fix blobserver path in curl and python clients" 2012-12-31 18:53:04 +00:00
Sarath Lakshman 778a607c67 client: fix blobserver path in curl and python clients 
Change-Id: Ie7a204ee64a9373c696e447ccf93a790584676d6
 2013-01-01 00:27:11 +05:30
Brad Fitzpatrick 1fa720f1e1 Merge branch 'master' of ssh://camlistore.org:29418/camlistore 2012-12-31 10:43:49 -08:00
Brad Fitzpatrick 09a2b367c5 A+C: add Sarath Lakshman (Individual CLA) 
Change-Id: Ic39ba38b55c8604664c8583520f0874644f18419
 2012-12-31 10:43:30 -08:00
mpl 8f62b31ebf Merge "vivify: support in camput and upload handler." 2012-12-30 22:31:39 +00:00
mpl ed20da7592 vivify: support in camput and upload handler. 
no special credentials/auth yet.
pkg/blobserver/handlers/get.go moved to
pkg/blobserver/gethandler/get.go to avoid
a dependency loop with the json sign helper.
pkg/server/sig.go was moved to pkg/jsonsign/signhandler
because it seemed inapproriate to import in
pkg/blobserver/handlers/upload.go something from
pkg/server

Change-Id: Ifeb14512e182e8a101d4fced6d6d4184e2b9cb99
 2012-12-30 23:39:22 +01:00