Rooba
/
perkeep
mirror of https://github.com/perkeep/perkeep.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,372 Commits 44 Branches 11 Tags 59 MiB
Commit Graph

3372 Commits

Author SHA1 Message Date
Tamás Gulácsi cb6f423eeb Add diskpacked-reindex subcommand to camtool 
For checking and/or rebuilding index.kv of diskpacked packs.

Change-Id: I1ad87974b2daf58b1e767bb1df6f7b64e8b6359a
 2013-11-28 20:44:27 +01:00
Brad Fitzpatrick 1defe7b297 serverconfig: add 'memoryIndex' bool option to high-level config 
Change-Id: If5edb272858f186931eeb57579e221bde3efb07c
 2013-11-28 11:09:16 -08:00
Brad Fitzpatrick 3db0eca58c Merge "pkg/index: check for deletions in EdgesTo" 2013-11-28 18:46:07 +00:00
mpl fa32346860 pkg/index: check for deletions in EdgesTo 
Also introduced and used kvEdgeBackward

http://camlistore.org/issue/191

Change-Id: Ifdca9f2453ec0c7db55afc40cf0b5f1441b83158
 2013-11-28 19:41:40 +01:00
Brad Fitzpatrick e2efc15cb8 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-11-28 09:10:49 -08:00
Brad Fitzpatrick 313f8d25cb camlistored: log more version info at start 
Change-Id: I3e1e27f6a475e438bb3bf6c06ca3a0941a2944eb
 2013-11-28 08:55:53 -08:00
mpl 2095a2c913 pkg/index: another delete test case 
This test verifies that a permanode is still considered
deleted until all its deleters have been deleted.

http://camlistore.org/issue/191

Change-Id: Ifa4f17f031884b766d9ac90f1643a62ada591ae5
 2013-11-28 16:34:49 +01:00
mpl 8ee50244f3 Merge "pkg/index: use IsDeleted in some of the search methods" 2013-11-28 15:11:11 +00:00
mpl 15bb5f142e pkg/index: use IsDeleted in some of the search methods 
1) Done in AppendClaims, PermanodeOfSignerAttrValue,
SearchPermanodesWithAttr, PathsOfSignerTarget,
and PathsLookup.

2) camtypes.Path.DateClaim is now a time.Time, and used
as such in Path search methods.

3) Added kvSignerAttrValue, kvPathBackward, and kvPathForward

4) More Delete tests. We probably want more cases, but that's
a start.

http://camlistore.org/issue/191

Change-Id: I1d42485f2c6d1eea024f70378daacaaad9197768
 2013-11-28 16:08:23 +01:00
Brad Fitzpatrick 4346056b73 Merge "genconfig: use same packed-ness for cache as blobs." 2013-11-28 05:33:46 +00:00
Brad Fitzpatrick 9520219fab Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-11-27 21:33:10 -08:00
Brad Fitzpatrick f1558f69b2 index: don't allow schemaVersion 0 if the index is non-empty 
Change-Id: Ifaf3fb6c4f4166028fdf9041abb274f6cb8998ca
 2013-11-27 21:31:05 -08:00
Bill Thiede 270e8f54b6 genconfig: use same packed-ness for cache as blobs. 
Change-Id: I857f7a3071bfd80ed5f4e8cef8ca2307a978655a
 2013-11-27 21:26:04 -08:00
Brad Fitzpatrick fbad538fd8 gofmt 
Change-Id: Ibec8c4b1a78bd93b78e26ff73de7a386699cc6c8
 2013-11-27 20:46:58 -08:00
Brad Fitzpatrick d2cd51a6ff corpus: print out Corpus memory/size stats 
Change-Id: I3e9ed323473488a76992e5d5a9429b7fef5fa9b2
 2013-11-27 16:58:05 -08:00
Brad Fitzpatrick 1ec41b95f0 Speed up in-memory permanode attribute search. 
Finishes Corpus.AppendPermanodeAttrValues and uses it in
PermanodeConstraint.

Change-Id: I031789e0974add24886cc0060e52ba538535c4e1
 2013-11-27 12:47:04 -08:00
Brad Fitzpatrick 3730cbafbb Merge "pkg/index: update the deletes cache when receiving a delete claim" 2013-11-27 18:29:02 +00:00
mpl 5f966c2f99 Merge "pkg/index: change deletes cache structure and meaning" 2013-11-27 17:45:52 +00:00
mpl 42e37d4456 pkg/index: update the deletes cache when receiving a delete claim 
http://camlistore.org/issue/191

Change-Id: I49da2ef4e43675fba6a80db29ba96a473c159403
 2013-11-27 18:44:39 +01:00
mpl 45b630f766 pkg/index: change deletes cache structure and meaning 
1) The deletes cache now simply reflects all the keyDeleted index entries,
instead of trying to always hold the computed deletion state of the
blobs.

2) The IsDeleted and DeletedAt methods have been changed accordingly.
Moreover, while they previously were only taking into account the most
recent delete claim that affected the input blob, they now take
into account all delete claims that concern said blob.
In other words, a blob is considered "active" only if all of its past
deleters are deleted too (or it does not have any deleter).

http://camlistore.org/issue/191

Change-Id: I1b276dac0b3c6a3fb3beaf96828a45a85d27e4c6
 2013-11-27 17:02:03 +01:00
Brad Fitzpatrick c819397cd5 corpus: keep permanode claims sorted in memory 
Change-Id: Ic0387a3789c10036e21f5e2eb10101a0ae23b0b7
 2013-11-27 00:32:24 -08:00
Brad Fitzpatrick 1eff3b0a34 Merge "search & index: change 'before' logic, and tests." 2013-11-27 06:29:04 +00:00
Bill Thiede 0e00fd6198 search & index: change 'before' logic, and tests. 
Change-Id: I62f43f26b295c46b9da63efe018d324f9105606b
 2013-11-26 21:51:06 -08:00
Brad Fitzpatrick c4b7d70ac3 corpus: start of work on AppendPermanodeAttrValues for permanode attr search 
Change-Id: I969778420474e3413813bb49b1bfd095d9a12389
 2013-11-26 20:28:14 -08:00
Brad Fitzpatrick 9a749ce73b search: efficient in-memory scans of permanode modtimes 
Change-Id: I82d34544a86cd9f3a52c2b7140dd89eac32fbfa9
 2013-11-26 19:47:00 -08:00
Brad Fitzpatrick c733835684 more TODO 
Change-Id: Ib43348a234a1902fa94d3d13b7e10aecb49da0e2
 2013-11-26 19:27:03 -08:00
Brad Fitzpatrick 1455a14e72 search: some refactoring to ease future work speeding PermanodeConstraint queries 
Change-Id: Iac98a632bd40c723481f8e635a2c34780e87a408
 2013-11-26 19:05:43 -08:00
Brad Fitzpatrick a918fac281 search: let GetRecentPermanodes take a before param. 
For infinite scroll.  Will be done better later, but this works for now.

Change-Id: I88dc7793b643226a4a5e3a583c5de5d4c7fa4819
 2013-11-25 20:35:59 -08:00
Brad Fitzpatrick 9aaadc73b8 index: put ImageInfo in memory too 
Change-Id: I8110df05dd14ec3ede2e49902b172ed9bf069bfb
 2013-11-25 19:46:28 -08:00
Brad Fitzpatrick 36019847aa Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-11-25 19:18:16 -08:00
Brad Fitzpatrick 03cc0fa8bd sync: wake up early from sleep if a blob arrives 
Change-Id: I49240d1970e537e3ace36f4cd02315ff3ed9d6b2
 2013-11-25 19:18:13 -08:00
Bill Thiede 80234700b2 Import github.com/golang/glog into third_party. 
Revision imported:

    commit 20cea4dfef0c0151bba0422a6244cc1d3e8292b1
    Author: David Symonds <dsymonds@golang.org>
    Date:   Wed Oct 9 08:22:02 2013 +1100

        Create a relative symlink for log files.

        This matches what C++ does in practice, and permits this code to work
        if -log_dir is a relative path.

Change-Id: Ifb81ae766ed5ab58fd45d3c7f485a065b01ec162
 2013-11-25 13:15:15 -08:00
Brad Fitzpatrick 20273566bf serverconfig: fix another /path/to/secring canonicalization case 
Change-Id: Ia0bdbdf0bdd0279055344d2c14a23eed473d7098
 2013-11-25 08:27:11 -08:00
Brad Fitzpatrick 326b24cc7c Merge "Add diskPack as a high level local storage destination" 2013-11-25 16:20:11 +00:00
Tamás Gulácsi 184686e849 Add diskPack as a high level local storage destination 
As a diskPack boolean flag in the high-level server config,
reusing the blobPath parameter for destination.

Change-Id: I9ec0583b160f89952cdad851e66c7fabf22b7bb4
 2013-11-25 11:23:50 +01:00
Bill Thiede 6b9f81e3f8 serverconfig: fix secring path for update_golden. 
Also regenerated the golden config files.

Change-Id: Id87dcad5f85b060eeee477e1fd3c72bff93d3d5e
 2013-11-24 20:38:00 -08:00
Brad Fitzpatrick fd122a5de6 Blind fix of bug in 3fa6d69405. 
See, tests matter.

Change-Id: I8c42be90db246e3529eac9f62711ea5510db605d
 2013-11-24 18:40:51 -08:00
Brad Fitzpatrick 786e2c7c00 Add a test TODO. 
Change-Id: I380b39fa6ce806386409fc50acaa4ec9b360b05a
 2013-11-24 18:28:59 -08:00
Brad Fitzpatrick 3fa6d69405 client: be robust against servers/proxies returning compressed content 
Change-Id: Id87c26a7d6e6d3f794a5421ece7cd344b10e1a8b
 2013-11-24 18:27:41 -08:00
Brad Fitzpatrick 2fb3d06d49 Remove dependency on the testing package from pkg/index. Polluted flags. 
Change-Id: Id5492fcab022c03e40501830d9253b5605a14ee6
 2013-11-24 17:45:45 -08:00
Brad Fitzpatrick 4c01434929 blobserver: run blobhub receive hooks in parallel. 
Change-Id: I918a124a01a2d75a64c26e136d28ed7f2547509d
 2013-11-24 17:12:56 -08:00
Brad Fitzpatrick 9261c9b303 dev-server-config: no longer need to replicate to a sync. the sync subscribes to its source. 
Change-Id: I7734f47f039cbdd40fcb672ba60d6a2488b42bdc
 2013-11-24 17:11:40 -08:00
Brad Fitzpatrick efd51a08ed devcam: print out temp dir on start-up 
Change-Id: I7f84ffdecc5cb63baa931d5b5de8aca564970a67
 2013-11-24 17:11:06 -08:00
Brad Fitzpatrick cf388c2f2a sync: have handler register receive hook with its source. 
Lets legacy configs work, even without replicating directly to it.

Change-Id: I8bdb8651040794ae346f19d6dd67a0da07505f07
 2013-11-24 16:20:11 -08:00
Brad Fitzpatrick b9fd11a8f6 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-11-24 15:13:24 -08:00
Brad Fitzpatrick eae8a74fc9 serverconfig/genconfig: add kv queues to all sync handlers. 
In the future this might be more configurable (e.g. mysql or sqlite or whatever, or different
paths).  But this works for now.

Change-Id: I85853e6bfdd9063fc2eba5922b138d5552b730a9
 2013-11-24 15:12:11 -08:00
Brad Fitzpatrick f4294cecdf Format pkg/serverconfig/testdata golden files consistently. No semantic changes. 
Change-Id: I8c190baf6514a2cf0580da45220d411b81445733
 2013-11-24 15:08:47 -08:00
Brad Fitzpatrick a7f7fb8741 serverconfig test: simplify pretty-printing, use test.Diff, add flag to update golden files 
Change-Id: I8c259dad8fc6f161f25fb4f63f97c4aef4b703b0
 2013-11-24 15:04:20 -08:00
Brad Fitzpatrick 9e3a9fde51 server/camlistored: add start-up test that tests default production config file 
Marked as broken for now. Will fix in subsequent commits.

Change-Id: If41932a72312ce8fbde1d237c3f84fa0acae83ee
 2013-11-24 15:03:34 -08:00
Brad Fitzpatrick 26f9b630f8 Add func test.BrokenTest and RUN_BROKEN_TESTS to doc/environment-vars.txt 
Change-Id: I43dc7b10ac6175ab8ab5eae4cfdd98bc08515ff5
 2013-11-24 15:00:16 -08:00