Rooba
/
perkeep
mirror of https://github.com/perkeep/perkeep.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,400 Commits 44 Branches 11 Tags 59 MiB
Commit Graph

3400 Commits

Author SHA1 Message Date
Brad Fitzpatrick b2d221af44 Start of Camlistore WebSocket support, for showing live search queries in the UI. 
If the JavaScript UI is open and the query you're looking at changes
due to new data arriving, the server should push the update search
results to you live, without needing to poll.

Change-Id: I00faf7737f9a459de39b21fb5226a8000feeee2b
 2013-12-07 20:19:20 -08:00
Brad Fitzpatrick 1641307a6b Import github.com/gorilla/websocket into third_party at its 2119675aadf 
Change-Id: Id3d62b16d3c0186295559205b86c40b95881d0d0
 2013-12-07 19:18:15 -08:00
Brad Fitzpatrick f806008d06 Corpus: after meta rows, scan rest all in parallel. 
This works well enough for MySQL at least. We'll see if it hurts the other sorted.KeyValue
implementations. If so, we might have to fix those.

Change-Id: I71e9f56f2dee9e7334c7a35a4960f55399ca306f
 2013-12-07 19:11:31 -08:00
Brad Fitzpatrick 5f1e7de0d1 serverconfig: also write mem profile with CAMLI_PPROF_START 
Change-Id: Iffe78c66916d6e690a28edfd258c51a5433edff5
 2013-12-07 18:34:13 -08:00
Brad Fitzpatrick 9d56f61921 Make all corpus row merge funcs take []byte to reduce garbage on startup slurp. 
Change-Id: I28cccdbe59792af16264d65e27b2a914e9285db2
 2013-12-07 11:08:19 -08:00
Brad Fitzpatrick b020f60390 search: add test for earlier fix in commit 1e1183a8bf 
Change-Id: I95c0f885079de17cd1c52b9ad40861c96eb5308c
 2013-12-07 10:53:30 -08:00
Brad Fitzpatrick 1e1183a8bf search: add a forgotten Cancel. 
Funny, because this one line is why I finally went and added the Context type
and started plumbing it through lots of stuff.

Change-Id: Ie419362048a485629e79725de9297fe485430d4a
 2013-12-07 08:49:30 -08:00
Brad Fitzpatrick e27a1dd019 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-12-07 08:46:29 -08:00
Brad Fitzpatrick 76171ddb3d Change sorted.KeyValue.Find to take an optional end bound; add tests. 
The new package sorted/kvtest provides a generic KeyValue test for all
implementations. Memory, SQLite, and kvfile now use it.

This speeds up the index slurping start-up of my personal Camlistore
server from 30 seconds (when it was doing 17,000+ queries in small
windows) to now just 5 seconds. That 5 seconds can be improved yet
further.

Change-Id: Idd55ba9ccd3ed12a26868a41db1af676aff7b67b
 2013-12-07 08:43:18 -08:00
Brad Fitzpatrick 1ca8b90d6c Merge "LogicalConstraint: typo in comment" 2013-12-07 15:15:45 +00:00
Aaron Boodman d593b6f98a LogicalConstraint: typo in comment 
Change-Id: I62345763e42832e2575f492a9678dbb0cfb65ec6
 2013-12-07 07:11:16 -08:00
Brad Fitzpatrick 49fced871b Update a comment in client/config.go 
Change-Id: Icc30171776c87107bdf844c4dc908f35ed98b9a4
 2013-12-07 06:48:02 -08:00
Brad Fitzpatrick 0163059814 sqlindex: don't iterate in small 50-row batches on big prefixes 
This speeds Camlistore start-up index slurping considerably. Instead of
17,000 queries in 30 seconds, it now does 6.

A future change will improve this further, giving sorted.KeyValue impls
a hint about the end range too.

Change-Id: Icb5e2ac6edd580469e907fde4e916c00aba0ffe8
 2013-12-07 06:46:02 -08:00
Aaron Boodman 79c1cee670 Merge "Added progress on ios uploader." 2013-12-06 23:14:47 +00:00
mpl 0af0ddd79f ui/toolbar: fix typo that was commenting out the button creation 
Change-Id: I983dd6cfc9bbcf0ae1a9e302476d29ffa918663a
 2013-12-06 23:50:58 +01:00
Brad Fitzpatrick 4e1ad20075 strutil: more common strings, especially 'bytes' 
Change-Id: I034f032f37aaea513d0b8e860d9fd780a681f6a8
 2013-12-06 17:26:33 +04:00
Brad Fitzpatrick 2428dc86fd Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-12-06 11:17:35 +04:00
Brad Fitzpatrick cde3176c0d blob: speed up parsing slightly. 
BenchmarkParseBlob  2000000   727 ns/op   64 B/op   2 allocs/op
BenchmarkParseBlob  5000000   531 ns/op   64 B/op   2 allocs/op

Kinda pointless and mostly the result of a failed refactor anyway,
but checking it in.

Change-Id: I7cdd0a8c5d03d7e223626ef6ddb77d8d10dd06a7
 2013-12-06 11:15:51 +04:00
Brad Fitzpatrick f1a13214af blob: implement the parser's buffer pool TODO 
Change-Id: I24366c5f42fd18103df68676047c7e4cdf6b7af6
 2013-12-06 10:19:20 +04:00
Brad Fitzpatrick 87a85c4543 index: don't log in benchmark, even to ioutil.Discard. just a CPU distraction. 
Change-Id: Id4d8b761cd81a7c8abd7e21d139bffc818aff5e1
 2013-12-06 10:06:18 +04:00
Aaron Boodman a574697b7d Merge "Add a 'clear selection' button per Brad request." 2013-12-05 19:30:26 +00:00
Brad Fitzpatrick 5709ab501f Import lldb panic fix from upstream 
Cherry-pick of fix from bb1bc282aa

Fixes https://code.google.com/p/camlistore/issues/detail?id=233

Change-Id: I2779fc719e3ed23e334e084a18b8320610ec2ec3
 2013-12-05 16:33:25 +04:00
Nick O'Neill d0c0d09f65 Added progress on ios uploader. 
Change-Id: I21e6f00ce2dec1dd430dc5af3a3818a45c96a628
 2013-12-04 22:05:18 -08:00
Aaron Boodman 4637b6676f Add a 'clear selection' button per Brad request. 
Also clean up dispatch of the related events somewhat.

Change-Id: I6c0e5fe20f2d985463cc434f9efd5796108da869
 2013-12-04 20:47:33 -08:00
Aaron Boodman 0e8ed35fe4 Merge "Beginning of infinite scroll for recent permanodes." 2013-12-05 03:49:00 +00:00
Brad Fitzpatrick 67a9cfd68c index: let index-row-to-corpus merger funcs have a []byte version 
Saves a lot of allocations and garbage.

Change-Id: Ic9dfb050a54323312056e8b69e93dfe7aaf715a8
 2013-12-04 23:58:59 +04:00
Brad Fitzpatrick facb7ac062 strutil: add StringFromBytes 
Change-Id: Ie396dd58c48a724e044ecd4f8dc28712e7091f9f
 2013-12-04 23:53:00 +04:00
Brad Fitzpatrick 6c1df9d633 blob: add ParseBytes 
Will be used by index->corpus slurping of blob meta rows on
start-up, to reduce garbage.

Change-Id: Ied6605bd44ae38537d3661579c7a8b6dea2b2d9e
 2013-12-04 14:38:13 +01:00
Brad Fitzpatrick 11a1745034 Add KeyBytes and ValueBytes accessor to sorted.KeyValue.Iterator. 
Goal is to iterate faster (notably: for slurping the index to the in-memory
corpus on start-up), by doing fewer copies and generating less garbage.

Change-Id: I054b0de2b994eb1f2356aa8587a466bafeb6cf82
 2013-12-04 14:18:52 +01:00
Brad Fitzpatrick 09dddbc63a go-sqlite3: temporary manual fix until we update from upstream 
Change-Id: Icacb6f467b6c4058bcfc65e71d5ff8265cdc6c9f
 2013-12-04 14:14:22 +01:00
Brad Fitzpatrick c7a0c5842c little TODO 
Change-Id: Icda704f63db4ad39ea4bfe13bb9b7d62b227d662
 2013-12-04 00:39:09 -08:00
Brad Fitzpatrick 098ab8645a index: add BenchmarkCorpusFromStorage 
Change-Id: Ifba96f21d5a003150062d988129c2c24fcae878c
 2013-12-03 21:56:34 -08:00
Brad Fitzpatrick 429fa4bf4e Fix API typo: ParseTime3339OrNil, not OrZil 
Change-Id: I1255cf4560d133fe5c6128fba0be2d9d4a0c0daa
 2013-12-03 21:56:09 -08:00
Brad Fitzpatrick de3de4c499 index: remove 2 allocs (3% of mem) in kvClaim, via Corpus.mergeClaimRow 
And some tests.

Change-Id: Ifd57b027e193d0356bca140c93c18963be664bae
 2013-12-03 21:52:12 -08:00
Brad Fitzpatrick fb14b09441 index: let benchmarks disable some expensive memstats 
Change-Id: I0138148ffa2543985bb78fb69ac2b122b50d0a04
 2013-12-03 21:38:03 -08:00
Brad Fitzpatrick 749c5aed3d leveldb TODO 
Change-Id: I20f5b0491d3adcc7eca89d819345efd31f11adde
 2013-12-03 21:36:58 -08:00
Brad Fitzpatrick 4c8d79e170 test: let TLog take a testing.B too. add tests. 
Change-Id: I25c47960ef63be85994a133da85072005ca47a85
 2013-12-03 21:12:06 -08:00
Brad Fitzpatrick cad54aa249 search: fix a panic in Query w/ describe, add tests 
Change-Id: I4a2288298fbab8a51c795b9eaed5fc3488024132
 2013-12-03 18:24:02 -08:00
Brad Fitzpatrick db46a44758 test: add TLog function to set log output to a *testing.T 
Change-Id: I68d945634ce7780acabf120fe27989083fb02a04
 2013-12-03 18:23:42 -08:00
Brad Fitzpatrick a7c7e717f6 search: optionally also describe matched blobs in same request 
Change-Id: Id9c16c19fa309d0780c6107b1dd29880d777fd05
 2013-12-02 20:29:50 -08:00
Brad Fitzpatrick bf4d2721ba search: efficient permanode queries, sorted, and with a limit. 
Change-Id: I7f8edfe3636ff3f1344ebb41e27d23073c630ff5
 2013-12-02 20:01:37 -08:00
Brad Fitzpatrick da1abea89e readerutil: fix atomic alignment for 32-bit platforms 
Change-Id: I10eb60a9abbbaf5aeccb1eba9592ff5d354d7985
 2013-12-02 19:01:44 -08:00
Brad Fitzpatrick b82b8efe4c Start of new context package and *context.Context type. 
Will eventually be plumbed through lots of APIs, especially those requiring or benefiting from
cancelation notification and/or those needing access to the HTTP context (e.g. App Engine).

Change-Id: I591496725d620126e09d49eb07cade7707c7fc64
 2013-12-02 13:20:51 -08:00
Brad Fitzpatrick 0c73a65f78 TODO: use uint32 for SizedRef, not int 
Change-Id: I6f8bdfd483c1be4e3ec933f2bf05686163ed2bd7
 2013-12-02 12:01:42 -08:00
Brad Fitzpatrick 7466336f44 storagetest: more tests, and simplify a bit 
Change-Id: Ic0d6c9718ef79ff82f05cf142286022862f30236
 2013-12-02 11:57:02 -08:00
Brad Fitzpatrick 6478e256f7 diskpacked: finish enumeration bug with 'after' param 
Change-Id: I391a33e7672cce6c7c9a4eebc0a09c2f03546b82
 2013-12-02 11:56:23 -08:00
Brad Fitzpatrick d324eecaba blob: add slice sorter types for []blob.Ref and []blob.SizedRef 
Change-Id: I23224a9408a9d6fd2bbd7dd0fa94dba64f4586c1
 2013-12-02 11:55:39 -08:00
Brad Fitzpatrick 88f4f309ac Merge "Finish implementing storagetest" 2013-12-02 19:01:27 +00:00
Aaron Boodman a22b12d9e7 Beginning of infinite scroll for recent permanodes. 
This doesn't have a progress meter yet.

Change-Id: Iee089c99ff0fef2b2c724ccd5d6087eb4fa3f087
 2013-12-01 22:59:39 -08:00
Aaron Boodman 0514b9fd3f Formatting cleanup in blob_item_container.js. 
Change-Id: Id6430c205f2d97e9439105ce7498ae9bc8861557
 2013-12-01 22:52:53 -08:00