Commit Graph

13 Commits

Author SHA1 Message Date
mpl ab5e385113 index: move kvfile implementation to sorted
Change-Id: Ie6e676570af088246ef028a2f003b537d85dd6ae
2013-12-13 16:59:45 +01:00
Brad Fitzpatrick 76171ddb3d Change sorted.KeyValue.Find to take an optional end bound; add tests.
The new package sorted/kvtest provides a generic KeyValue test for all
implementations. Memory, SQLite, and kvfile now use it.

This speeds up the index slurping start-up of my personal Camlistore
server from 30 seconds (when it was doing 17,000+ queries in small
windows) to now just 5 seconds. That 5 seconds can be improved yet
further.

Change-Id: Idd55ba9ccd3ed12a26868a41db1af676aff7b67b
2013-12-07 08:43:18 -08:00
Brad Fitzpatrick 11a1745034 Add KeyBytes and ValueBytes accessor to sorted.KeyValue.Iterator.
Goal is to iterate faster (notably: for slurping the index to the in-memory
corpus on start-up), by doing fewer copies and generating less garbage.

Change-Id: I054b0de2b994eb1f2356aa8587a466bafeb6cf82
2013-12-04 14:18:52 +01:00
Brad Fitzpatrick ecc5759758 kvfile: log which db was closed
Change-Id: I932d9b143db0842afe6df55aad8b8c17f85d1254
2013-11-30 09:32:37 -08:00
mpl 15bb5f142e pkg/index: use IsDeleted in some of the search methods
1) Done in AppendClaims, PermanodeOfSignerAttrValue,
SearchPermanodesWithAttr, PathsOfSignerTarget,
and PathsLookup.

2) camtypes.Path.DateClaim is now a time.Time, and used
as such in Path search methods.

3) Added kvSignerAttrValue, kvPathBackward, and kvPathForward

4) More Delete tests. We probably want more cases, but that's
a start.

http://camlistore.org/issue/191

Change-Id: I1d42485f2c6d1eea024f70378daacaaad9197768
2013-11-28 16:08:23 +01:00
Brad Fitzpatrick 1f407edd58 sorted: add Close method to KeyValure; register 'kv' impl type for cznic/kv
Change-Id: Ie4e21450dac9dad3433c65631706f52527502cbc
2013-11-23 21:07:03 -08:00
Brad Fitzpatrick 90c1e48afe Rename index.Storage to sorted.KeyValue and move it into a new package.
Having index.Index and index.Storage both in the same package led to
confusing discussions about "an index". Better names now, and smaller
packages.
2013-11-22 23:24:54 -08:00
mpl fdaf030bc0 index: add deletedAt method, and cache for deletion status
This method will be used to find out when a permanode or claim
was (un)deleted for the last time.

This change also introduces the keyDeletes index entry which
allows to find the permanodes/claims that a delete claim deletes.

Finally, since some index keys have changed, this CL introduces
a check for an index schema version, to force users to reindex
when needed.

http://camlistore.org/issue/191

Change-Id: I6f8e5d3030a6ddc8b21ff5dc5db311f2ea16f613
2013-11-06 23:34:13 +01:00
Brad Fitzpatrick 2b9db0a859 kvfile: say which lock file couldn't be acquired.
Still not a great error message, though.  Should say pid or something.

Change-Id: I4382f23e62e58f8fd472e8d84c3555f4f951d90d
2013-09-22 20:14:23 +01:00
Brad Fitzpatrick ca64e4f508 camlistored: on SIGINT, shut down cleanly, closing the kvfile index db
Change-Id: Ifcb8c22b088586bc72ad98dacc3be2d46e77928b
2013-08-26 21:07:28 -05:00
Brad Fitzpatrick c55bf01f84 Move osutil.Lock to its own package
Now in a small repo at github.com/camlistore/lock and brought into our third_party.

Change-Id: I966ca6126dde8b7c2e03f0d9fab1e36abd66c2ef
2013-08-25 19:53:01 -05:00
Brad Fitzpatrick 87451bb5bd index/kvfile: use new Locker hook in kv to use osutil's Lock func
Change-Id: I7573ea79f5f99ef1b7fe03bd9270b61940339a56
2013-08-25 12:56:36 -05:00
Brad Fitzpatrick 4fa8c3117a Add pkg/index/kvfile: index.Storage impl on top of github.com/cznic/kv
Change-Id: I4adb44c4e70ad8e68b4292fefbbd02e0a52beb6c
2013-08-23 17:13:42 -05:00