Rooba
/
perkeep
mirror of https://github.com/perkeep/perkeep.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,477 Commits 44 Branches 11 Tags 59 MiB
Commit Graph

3477 Commits

Author SHA1 Message Date
Brad Fitzpatrick 80ea29de8f mongo: use SKIP_DEP_TESTS again. regressed in recent refactor. 
Noticed because 'make presubmit' was breaking when offline.

Change-Id: I1afd12eb2ae8f504018b8a30a91d2d2e87d35999
 2013-12-14 15:15:09 +01:00
Brad Fitzpatrick 60e05bf90b Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-12-14 14:13:17 +04:00
Brad Fitzpatrick a078ce9406 search: generate an optimized matcher and use it within the matching loop 
benchmark                         old ns/op    new ns/op    delta
BenchmarkQueryRecentPermanodes        27987        27512   -1.70%

benchmark                        old allocs   new allocs    delta
BenchmarkQueryRecentPermanodes           99           87  -12.12%

benchmark                         old bytes    new bytes    delta
BenchmarkQueryRecentPermanodes         6433         6207   -3.51%

But will be better in practice with more blobs.

Change-Id: I4673702231c52864249329818145aad11da2b4fc
 2013-12-14 12:07:35 +04:00
Brad Fitzpatrick d6691c16de search: add a basic query benchmark 
Change-Id: I1da18ef34a5d1f83bb5be2f9684a64bc43312bd7
 2013-12-14 12:06:10 +04:00
Brad Fitzpatrick b406cb3475 query_test: remove some comments, now that they're on RelationConstraint 
Change-Id: Id38dcbb2cfce4ab15ec55c08005356fb2e018ee6
 2013-12-14 10:33:15 +04:00
Nick O'Neill 8ef0276246 Merge "Added some progress towards search." 2013-12-13 18:56:18 +00:00
Nick O'Neill 7b194895da Merge "Upload by way of significant location change service." 2013-12-13 18:56:07 +00:00
mpl 3c5a6e27af Merge "devcam server: properly wipe mongo" 2013-12-13 16:40:24 +00:00
mpl 6d19d64b88 devcam server: properly wipe mongo 
Change-Id: Ib75a460a4be84e21affc662d45b3cf1e321b15bd
 2013-12-13 17:37:19 +01:00
Brad Fitzpatrick 6bc7a83ee0 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-12-13 20:34:09 +04:00
Brad Fitzpatrick 2921a320cb Merge "index: move kvfile implementation to sorted" 2013-12-13 16:33:38 +00:00
mpl b86e95caf4 Merge "camtool: dbinit mongo support" 2013-12-13 16:27:49 +00:00
mpl 8c562e9135 camtool: dbinit mongo support 
Change-Id: I289759ed9bdb34a6e108af1364c4a6bcfbb5982f
 2013-12-13 17:25:03 +01:00
mpl ab5e385113 index: move kvfile implementation to sorted 
Change-Id: Ie6e676570af088246ef028a2f003b537d85dd6ae
 2013-12-13 16:59:45 +01:00
Brad Fitzpatrick addc303226 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-12-13 06:56:01 -08:00
mpl 39996424c7 sorted/mongo: close iter now that we have recent mgo 
Change-Id: If8e7f2ba611f2d3704acece56e03804bf9bd44f7
 2013-12-13 15:44:18 +01:00
Brad Fitzpatrick 9af5a970de Mark boring files generated by FUSE as "hidden", then hide in the UI. 
Makes for prettier FUSE demos w/ files showing up live in the browser
with websockets.

Change-Id: I41cc9ae0d33db39b4dbbf5b60714cee9a79b7248
 2013-12-13 13:27:48 +04:00
Brad Fitzpatrick 94b96d90a8 Merge "third_party/mgo: updated to rev 253" 2013-12-13 07:26:03 +00:00
Brad Fitzpatrick 617bbf5295 replica: support blobserver.Generationer 
Change-Id: Iad3a0e040fcffa7bee2cfe5ff7ce8904d4ded36c
 2013-12-13 11:24:34 +04:00
Brad Fitzpatrick bf203d6ff7 Document sorted.Wiper more. 
Change-Id: Iede5cf3ec12453910594c682a3a18f8c237fbd40
 2013-12-13 11:18:27 +04:00
Brad Fitzpatrick 9f8e9e57fd Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-12-13 11:15:29 +04:00
mpl 96f0e39700 third_party/mgo: updated to rev 253 
Change-Id: I995db6e123b1ef3d6061774aa333c7dbe645a744
 2013-12-13 00:51:30 +01:00
mpl fcbbf2a4df index: move mongo implementation to sorted 
Also devcam server -wipe wasn't wiping for mongo anymore, now fixed.

Change-Id: Iecc9d8025ddfba8d8ae9417ee170baf02be1d52f
 2013-12-13 00:25:40 +01:00
Brad Fitzpatrick 5eeef19472 Make CAMLI_FAST_DEV work again. 
Change-Id: I294de23a50d48c7c2325780b50141a7f02d55aab
 2013-12-12 16:18:24 +04:00
Brad Fitzpatrick 22101b61d8 flickr: clarify in README that the flag is a devcam flag, not server flag 
Change-Id: Iaab951c326ff90f09b6751f8fe4d439743a5e430
 2013-12-12 15:39:44 +04:00
Brad Fitzpatrick 511a599e3b Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-12-12 15:13:57 +04:00
Brad Fitzpatrick 61deeffef5 Make WebSockets work over https/wss, and make Auth work with WebSockets too. 
Change-Id: I321f9d7fe80bf4c88771cbb925f3712229cc09ee
 2013-12-12 15:13:44 +04:00
Aaron Boodman a5f189ef0e Fix an intermittent JavaScript error during server push. 
Change-Id: If942a4d79dc53619e52b205dd462d0fb8f44aeae
 2013-12-12 03:07:56 -08:00
Brad Fitzpatrick 6d77b39d8f Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-12-12 14:04:27 +04:00
Aaron Boodman ae06cdba97 Implements quick and dirty server-push support in the client. 
Change-Id: I32a3d7887b2855fb261b0e6e5c7fbd54e8e0a868
 2013-12-12 13:42:36 +04:00
Aaron Boodman 5e72b0f104 Implements quick and dirty server-push support in the client. 
Change-Id: I32a3d7887b2855fb261b0e6e5c7fbd54e8e0a868
 2013-12-12 09:38:34 +00:00
Brad Fitzpatrick 88f9db482b Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-12-12 13:04:13 +04:00
Brad Fitzpatrick 9ac73660a1 Sort recent permanodes first by time, then by blobref. 
Just in case two permanodes were modified in the same nanosecond (or
more likely: the client's clock resolution sucks when they created the
mutation).

Fixes http://camlistore.org/issue/261

Change-Id: I293fc0f771bccc0473fefdcc2178284066b7a05c
 2013-12-12 13:03:00 +04:00
Brad Fitzpatrick fc4d26ad3b Revert "This patch hangs the server." (the websockets live query code) 
I accidentally committed this earlier, but it was still a WIP.

This reverts commit aef2fba3e1.
 2013-12-12 11:38:01 +04:00
Brad Fitzpatrick a0576f2c0d Fix corpus/search locking bugs. 
The WebSockets live queries exposed a latent locking bug: the search
code was acquiring the Corpus read lock multiple times.  If a write
came in during the middle of a search, between two read locks, the
second RLock would hang (since a writer lock was pending) and then the
search and write would deadlock.

Instead, just acquire the read lock once at the beginning of the query and
never re-acquire it.  For now: new "Locked" versions of all the Corpus APIs.
In the future: will probably just add the locked-or-not into the context, and make
sure all Corpus methods require a *context.Context.

Also fix two potential data races: one with LogicalCondition (which
might've raced on the scratch ss buffer, depending on the query), and
a test-only race that didn't matter in production, but was annoying in
tsan output regardless, so fixed.

More tests to follow later. Unblocking Aaron for now.

Change-Id: Ie1edcbd061235f1ef04f9b4c68b86814c84361af
 2013-12-12 11:34:07 +04:00
Brad Fitzpatrick 3da18bc2da typo: use Errorf instead of Error in test 
Change-Id: Idaa1b8870975d50d22789ed6fd6787f213d0d0d3
 2013-12-12 11:14:09 +04:00
Brad Fitzpatrick a197984bef syncutil: more debugging and paranoia 
Change-Id: I9dcb903e4f0918097130a93c14c498c260832294
 2013-12-12 11:13:03 +04:00
Brad Fitzpatrick fc1dfc71a2 syncutil: more RWMutexTracker logging and tracking 
Change-Id: I02032a49b7413f56f8283b4ab446fcb0474d4ff3
 2013-12-12 09:08:49 +04:00
Nick O'Neill 5aa6fbff61 Added some progress towards search. 
Change-Id: I755f6c05a011461ba78441d5e1cfb8d8dfeb19e1
 2013-12-11 11:57:49 -08:00
Nick O'Neill 4e54a3515b Upload by way of significant location change service. 
Change-Id: I1793da66ce642bd896f1a41e464cb6e97356c7e1
 2013-12-11 11:57:49 -08:00
Brad Fitzpatrick d6c5f4f396 syncutil: add RWMutexTracker debugging type 
Change-Id: Ifac22bbea3ef116685baa617d42a2cd7f16a1dd3
 2013-12-11 22:13:31 +04:00
Brad Fitzpatrick 2d2a0c1479 devcam: add -race flag 
Change-Id: I1d477431b917faba3d2ee99ef5241dda14e748f3
 2013-12-11 22:13:06 +04:00
Brad Fitzpatrick 2fde7c11a0 Add make.go option to build with the race detector. 
Change-Id: Ie972bd7ca74e59cbdbf4000ba505473fdd15dc6f
 2013-12-11 15:53:36 +04:00
Brad Fitzpatrick 1ade80b0f0 add blobhub TODO 
Change-Id: I8d3b69d0264610cfdb35addfefeaad11be43c557
 2013-12-11 15:53:17 +04:00
Brad Fitzpatrick bbaf755995 search: WIP notes on RelationConstraint. Not implemented yet. 
Change-Id: Ic179a046da146706b57d7e6e031e1665deedd036
 2013-12-11 13:46:19 +04:00
Aaron Boodman aef2fba3e1 This patch hangs the server. 
Apply the patch, load the UI, then run ./bin/devcam put -filenodes -permanode doc.
Server reliably hangs for me.

Change-Id: I4057d4575515d1298ab83e58059b669a6f2bdacb
 2013-12-11 13:14:37 +04:00
Brad Fitzpatrick 15386a1d51 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-12-11 12:26:41 +04:00
Brad Fitzpatrick bf54591e8d Remove the memIndex option from genconfig and devcam; default memoryIndex to on. 
We had the legacy "memIndex" option to put the Camlistore index in
memory (the leveldb memdb) as an option in genconfig called
"memIndex", and we also had an option called "memoryIndex" for whether
to slurp the on-disk index to memory on start-up.  Too confusing!

Instead, delete "memIndex" (since it's not the default anyway, now
that we have kv).

Then, also (the original point of this change): default the
memoryIndex option to true, so search works for people by default.
This option might go away in the future if it becomes the only required
way.

Also, document this.

Change-Id: Iddffa6e19adbf09c5aacd063aa44de362d90633b
 2013-12-11 12:20:22 +04:00
Brad Fitzpatrick 23aa50846f Merge "camput init: use osutil.KeyBlobsDir()" 2013-12-11 07:44:49 +00:00
Aaron Boodman 930b8a2258 Merge "Add a README for the flickr importer package" 2013-12-11 05:38:06 +00:00