Commit Graph

25 Commits

Author SHA1 Message Date
Brad Fitzpatrick 4188fefaa0 sqlite: Empty file to make the go tool happy with a test-only directory.
Not sure if this is still needed but had it sitting on my filesystem, never
checked in.

Change-Id: I06a5efd265032b6a4f867a8d0504cfb4f061304b
2014-08-02 16:51:47 -07:00
mpl 899acb8e72 pkg/sorted: remove API for each distinct implementation
Change-Id: I42446683189cdb00dcfbec1f221885092fdd83d6
2014-04-08 01:37:08 +02:00
mpl 291320d451 pkg/index: cleanup, refactor sub pkgs
http://camlistore.org/issue/263

Change-Id: I319bb097f0ce30b2bd5271b5c3bbff92b8dcc318
2014-04-05 01:22:05 +02:00
Brad Fitzpatrick bfc607fee7 index: reindex blobs when dependent blobs arrive out-of-order
Keep track of missing dependencies both in memory and in the index's
underlying sorted.KeyValue. When we see a dependent blob arrive, see
if we can reindex things.

Fixes camlistore.org/issue/102

Change-Id: I3d8cfc463e4b8c9d158be8f9656e772839b093b9
2014-03-15 08:44:09 -07:00
mpl 62f8a252ff pkg/index: move mysql to sorted
also minor cleanups in sqlite and mongo

Change-Id: I1f371358997d929c72a8f63d2630a1c3fa4a0240
2013-12-18 20:18:20 +01:00
mpl b62c94fdd1 pkg/index: move sqlite implementation to sorted
Change-Id: I1e300ffa14547fabeac8d255fd694054ebcb9e53
2013-12-16 17:35:27 +01:00
Brad Fitzpatrick 76171ddb3d Change sorted.KeyValue.Find to take an optional end bound; add tests.
The new package sorted/kvtest provides a generic KeyValue test for all
implementations. Memory, SQLite, and kvfile now use it.

This speeds up the index slurping start-up of my personal Camlistore
server from 30 seconds (when it was doing 17,000+ queries in small
windows) to now just 5 seconds. That 5 seconds can be improved yet
further.

Change-Id: Idd55ba9ccd3ed12a26868a41db1af676aff7b67b
2013-12-07 08:43:18 -08:00
mpl 15bb5f142e pkg/index: use IsDeleted in some of the search methods
1) Done in AppendClaims, PermanodeOfSignerAttrValue,
SearchPermanodesWithAttr, PathsOfSignerTarget,
and PathsLookup.

2) camtypes.Path.DateClaim is now a time.Time, and used
as such in Path search methods.

3) Added kvSignerAttrValue, kvPathBackward, and kvPathForward

4) More Delete tests. We probably want more cases, but that's
a start.

http://camlistore.org/issue/191

Change-Id: I1d42485f2c6d1eea024f70378daacaaad9197768
2013-11-28 16:08:23 +01:00
Brad Fitzpatrick 90c1e48afe Rename index.Storage to sorted.KeyValue and move it into a new package.
Having index.Index and index.Storage both in the same package led to
confusing discussions about "an index". Better names now, and smaller
packages.
2013-11-22 23:24:54 -08:00
mpl fdaf030bc0 index: add deletedAt method, and cache for deletion status
This method will be used to find out when a permanode or claim
was (un)deleted for the last time.

This change also introduces the keyDeletes index entry which
allows to find the permanodes/claims that a delete claim deletes.

Finally, since some index keys have changed, this CL introduces
a check for an index schema version, to force users to reindex
when needed.

http://camlistore.org/issue/191

Change-Id: I6f8e5d3030a6ddc8b21ff5dc5db311f2ea16f613
2013-11-06 23:34:13 +01:00
Brad Fitzpatrick b34c6198e4 sqlite: add start of an fd leak test
Change-Id: I84857e7a57bc4d95fa1a0ddc1c7538d1924cb93a
2013-09-13 11:21:48 +01:00
mpl d488c576fc search: support for static directory children
This change introduces a new index entry
to help with finding the children of a static directory.
It also fixes ResolvePrefixHop so that it takes
into account static directories, and not only collections.

This is the first step to support publishing static directories.

http://camlistore.org/issue/179

Change-Id: I5666e5caa6c782004054ae4c19a6b6119d4fda8b
2013-09-10 23:06:48 +02:00
Brad Fitzpatrick e20c7f44a8 auth: fix security problem where 'localhost' allowed all access.
Also, add tests. And remove CAMLI_ADVERTISED_PASSWORD, replacing it with explicit devauth mode.

Change-Id: I9cdfe2d537ecefcf75df0b9fb8538258703b1eb7
2013-08-31 13:44:44 -07:00
mpl f4ef8c0375 dev-server: remove. replaced by devcam server.
Fix related docs, config files, buildbot...

Change-Id: I638d3856f42953603b13204fb56782974d0db9ee
2013-08-03 01:12:38 +02:00
Brad Fitzpatrick eb1208a12e sqlite: updated ErrNotCompiled installation hint
Change-Id: I96a5fde5bf3d34cc29f38180272a624c353a6c4f
2013-07-28 18:23:04 -07:00
Brad Fitzpatrick 9468e5ba70 More docs. Every package is documented now.
misc.CountingReader moves into readerutil.

pkg/atomics is folded into pkg/types.

pkg/test/testdep is folded into pkg/test, with better name/docs.

Old cruft from pkg/webserver is deleted.

Change-Id: I3f72d8b29804254ef944995fb085837c878f79f5
2013-07-07 21:12:30 -07:00
Brad Fitzpatrick ded8f8d0d7 Be helpful about how to setup SQLite, especially on a Mac.
Change-Id: I87dbcd077383f0148a8c50f4120e6606139fcacd
2013-06-22 17:51:09 -07:00
mpl a4aba6671a sqlite: Write-Ahead Logging, for improved concurrency
http://camlistore.org/issue/114

Change-Id: I93c3d48f06a6cb0703bd9ce3f89b03fc59826a73
2013-05-01 18:04:03 +02:00
Brad Fitzpatrick 2fd4a22694 Make sqlite (more) reliable. camlistore.org/issue/114
Change-Id: Ia759de784bf4363bd9f5141ddddfa2a441a53f1d
2013-02-17 22:10:28 -08:00
Brad Fitzpatrick 9720652971 Add disabled failing test for http://camlistore.org/issue/114
Change-Id: Iae5dbf1f28ca89793f43bc5e2df2493cf2747a77
2013-02-17 21:42:50 -08:00
Brad Fitzpatrick 75360ee9b3 Rename index.IndexStorage to index.Storage, to reduce stutter.
Change-Id: I52b2a8460c3957a1c8211a470e1df9a8dc5b7067
2013-01-13 20:19:36 -08:00
Brad Fitzpatrick 6ca0efac8c camlistored: much better sqlite support, by default, and help/docs/hints 2013-01-10 16:16:10 -08:00
Brad Fitzpatrick 02b8b35b51 Make sqlite support optional and detectable. 2013-01-10 14:29:20 -08:00
Brad Fitzpatrick f696007a14 index: implement EdgesTo
For finding back references in the graph.
2012-11-05 10:29:42 +01:00
Brad Fitzpatrick a519105585 index/sqlite: new indexer, using sqlite3
Change-Id: If1405fe3ee7c632c05431f4b0be1077d6d42c4a3
2012-04-13 16:11:16 -07:00