Commit Graph

13 Commits

Author SHA1 Message Date
mpl 167bed4277 pkg/index: util_test.go not an index impl
Change-Id: Iccc1bb43d703ec8a05f30bdd3132058fbdff2f1f
2016-10-24 18:12:13 +02:00
Attila Tajti 44b4e855d2 index/Corpus: add Attrs cache to PermanodeMeta
- add an Attrs map to PermanodeMap that is kept in sync with Claims
- update Attrs map on scanFromStorage and whenever a new blobRef is added
- add fast path to AppendPermanodeAttrValues(Locked),
  PermanodeAttrValueLocked and PermanodeHasAttrValueLocked
- fix bugs in and add tests for PermanodeHasAttrValueLocked and
  PermanodeAttrValueLocked slow path

Change-Id: Ib0dcdcf9355dd5fa8588c872721a029ec114cb6b
2016-01-27 17:09:18 +01:00
mpl 0f9a56241a index: automatically fix missing wholeRef in FileInfo rows
Issue #577

Change-Id: I45f210b32bcecb68b0ae3f60540cb33b58a729e5
2015-05-12 23:47:29 +02:00
mpl 443f405962 index: fix data race on BlobSource, make it private.
index.New was starting outOfOrderIndexerLoop in a goroutine. And
outOfOrderIndexerLoop had an if index.BlobSource == nil check, on which
it relied to go on. However, since BlobSource was public and unguarded,
the following sequence was possible:

ix, _ := index.New()
ix.BlobSource = bs

which is racy because the BlobSource assignment may or may not happen
before the check within outOfOrderIndexerLoop.

TestOutOfOrderIndexing was relying on the fact that apparently most
of the time the assignment seems to be happening before the check.

This patch:
-makes BlobSource (now blobSource) private, rendering the race impossible
out of the index package.
-moves the initialization of blobSource, as well as the execution of
outOfOrderIndexerLoop at a unique point, in InitBlobSource (new method).
-makes sure all accesses to blobSource are guarded with the index mutex
(now a RWMutex).

Context: while working on tests for http://camlistore.org/issue/454

Change-Id: I9605f26b41abd62b42880be0620b06ce143761bc
2014-06-27 22:29:29 +02:00
mpl 94f598ddb9 index/corpus: cache sorted (by time) permanodes
Context: http://camlistore.org/issue/298

Change-Id: I599f430e0f7691e54df38e10344c62cbdb6429a0
2014-06-24 02:39:25 +02:00
Brad Fitzpatrick bfc607fee7 index: reindex blobs when dependent blobs arrive out-of-order
Keep track of missing dependencies both in memory and in the index's
underlying sorted.KeyValue. When we see a dependent blob arrive, see
if we can reindex things.

Fixes camlistore.org/issue/102

Change-Id: I3d8cfc463e4b8c9d158be8f9656e772839b093b9
2014-03-15 08:44:09 -07:00
mpl 0fdca769e0 corpus: deleted check when enumerating permanodes
http://camlistore.org/issue/354

Change-Id: I7d76c654eb3b875ad8017a14a08a5c909bd696a0
2014-03-10 18:43:57 +01:00
Brad Fitzpatrick 2f9b87ad59 index: ignore recpn log spam, export SetVerboseCorpusLogging for pkg/search tests
Change-Id: Ie18c8db5ab33e94a6c37e3fa19170ac046d864b2
2013-12-09 11:26:33 +04:00
Brad Fitzpatrick 9d56f61921 Make all corpus row merge funcs take []byte to reduce garbage on startup slurp.
Change-Id: I28cccdbe59792af16264d65e27b2a914e9285db2
2013-12-07 11:08:19 -08:00
Brad Fitzpatrick fb14b09441 index: let benchmarks disable some expensive memstats
Change-Id: I0138148ffa2543985bb78fb69ac2b122b50d0a04
2013-12-03 21:38:03 -08:00
Brad Fitzpatrick c5037b76e6 Fix thumbnails, broken in cb31dd58d2
Change-Id: I82defd54f53d4f6c1a594185d5e9b7d3a4a92c07
2013-11-20 17:44:10 -08:00
Brad Fitzpatrick 17a3faf2df index: export NewMemoryIndex
Change-Id: Iaa6d3268ce0f6c531e37c97799deeb31c11eabdb
2012-11-04 12:18:11 +01:00
mpl f9d0ad487e index packages reorganization
Change-Id: Ie91c4bb5ea02a49e59ad093ac972d84b17a046d4
2012-02-26 13:50:14 +01:00