1e1183a8bf
search: add a forgotten Cancel.
...
Funny, because this one line is why I finally went and added the Context type
and started plumbing it through lots of stuff.
Change-Id: Ie419362048a485629e79725de9297fe485430d4a
2013-12-07 08:49:30 -08:00
d593b6f98a
LogicalConstraint: typo in comment
...
Change-Id: I62345763e42832e2575f492a9678dbb0cfb65ec6
2013-12-07 07:11:16 -08:00
cad54aa249
search: fix a panic in Query w/ describe, add tests
...
Change-Id: I4a2288298fbab8a51c795b9eaed5fc3488024132
2013-12-03 18:24:02 -08:00
a7c7e717f6
search: optionally also describe matched blobs in same request
...
Change-Id: Id9c16c19fa309d0780c6107b1dd29880d777fd05
2013-12-02 20:29:50 -08:00
bf4d2721ba
search: efficient permanode queries, sorted, and with a limit.
...
Change-Id: I7f8edfe3636ff3f1344ebb41e27d23073c630ff5
2013-12-02 20:01:37 -08:00
4eaeee9ae3
Reduce in-memory size of image info dimensions.
...
For my personal server,
Before:
Corpus stats: 149.884 MiB mem: 787886 blobs (88.853 GiB) (279308 schema (6073 permanode, 150557 file (111819 image), ...)
After:
Corpus stats: 145.172 MiB mem: 787886 blobs (88.853 GiB) (279308 schema (6073 permanode, 150557 file (111819 image), ...)
Change-Id: I47484e46b1eb70c8c4b5ee356cdcb0ceecde94e7
2013-11-29 11:01:41 -08:00
1ee5fd20c5
search: deletions are not modifications
...
1) pkg/search: documented that deletions times do not
qualify as modtimes
2) pkg/index: got rid of DeletedAt, and keyDeletes
http://camlistore.org/issue/191
Change-Id: I39578913345454d36af4599e29e7053f46577846
2013-11-29 00:29:57 +01:00
1ec41b95f0
Speed up in-memory permanode attribute search.
...
Finishes Corpus.AppendPermanodeAttrValues and uses it in
PermanodeConstraint.
Change-Id: I031789e0974add24886cc0060e52ba538535c4e1
2013-11-27 12:47:04 -08:00
9a749ce73b
search: efficient in-memory scans of permanode modtimes
...
Change-Id: I82d34544a86cd9f3a52c2b7140dd89eac32fbfa9
2013-11-26 19:47:00 -08:00
1455a14e72
search: some refactoring to ease future work speeding PermanodeConstraint queries
...
Change-Id: Iac98a632bd40c723481f8e635a2c34780e87a408
2013-11-26 19:05:43 -08:00
a918fac281
search: let GetRecentPermanodes take a before param.
...
For infinite scroll. Will be done better later, but this works for now.
Change-Id: I88dc7793b643226a4a5e3a583c5de5d4c7fa4819
2013-11-25 20:35:59 -08:00
9aaadc73b8
index: put ImageInfo in memory too
...
Change-Id: I8110df05dd14ec3ede2e49902b172ed9bf069bfb
2013-11-25 19:46:28 -08:00
2fb3d06d49
Remove dependency on the testing package from pkg/index. Polluted flags.
...
Change-Id: Id5492fcab022c03e40501830d9253b5605a14ee6
2013-11-24 17:45:45 -08:00
650bd9b61e
Display a folder icon and make camliPath attributes navigable.
...
Like we do for camliMember.
Change-Id: I2db51d7397e0f8ab86ea71716fe06099ba22005a
2013-11-22 22:49:07 -08:00
c5037b76e6
Fix thumbnails, broken in cb31dd58d2
...
Change-Id: I82defd54f53d4f6c1a594185d5e9b7d3a4a92c07
2013-11-20 17:44:10 -08:00
531b4dc32b
search: remove MIME from DescribedBlob; fix Handler tests.
...
Change-Id: I2d92155e3446325520d91379ac24becef49ca56f
2013-11-18 16:36:08 -08:00
48dba353e9
query: make all tests test in all three modes.
...
Change-Id: I945830ddb3bfe68a7223994b94d780b46503c95e
2013-11-17 20:20:27 -08:00
cb31dd58d2
In-memory search: include FileInfo. More tests pass.
...
Change-Id: Ic33357b63d1d57dc174fe18360e920730016c903
2013-11-17 19:49:19 -08:00
e8603b1293
Put claims in memory too for in-memory search. Required index schema version bump.
...
Change-Id: I194d65476bddea111277cd0b1472c56b5527226b
2013-11-17 16:52:51 -08:00
0b07171c0c
index: change GetOwnerClaims interface for efficiency, rename to AppendClaims
...
Change-Id: Icec235961ed8a09256df87c5fe1455cbad81ae3f
2013-11-17 14:54:30 -08:00
d82f02db50
Search optimization: keep blobs arranged by camliType in memory; use when possible.
...
Change-Id: I8b3ebcf7dfece21d17074da32dc20e4d600e4c50
2013-11-17 10:52:37 -08:00
3eb493599e
in-memory search: better structure for keeping memory corpus and kv
...
index in sync, both at start-up and while running and receiving blobs.
They both use the same mechanism now.
Also adds KeyId to the index and Corpus, as the next step. Plenty more
row types remain...
Change-Id: Id79955ba25dc79d5fbd94b0e5248d33dcf71d97e
2013-11-17 09:41:45 -08:00
272db6545a
index: change GetBlobMIMEType to GetBlobMeta. mime types on a blob was wrong anyway.
...
And some more of in-memory search, which this makes easier / less garbage-y.
Change-Id: If1fa7dd44842244ae5e5440f84e92d66dc3321fb
2013-11-16 19:40:14 -08:00
f3cc3c7ed9
search: more in-memory search work. make tests verify Scan doesn't hit Storage.
...
also some string interning work.
Change-Id: I7864b56eb97318bce943afdca3b1212f4729a9a8
2013-11-16 18:50:01 -08:00
e26eacfa2f
index: return FileInfo and ImageInfo as values, not pointers.
...
Change-Id: I6aba3a74ce233cc834550b561f597e549fb2bfab
2013-11-16 17:49:28 -08:00
2984897ac7
search: more in-memory search work.
...
keep blob metadata in memory, and start of testing all search queries in three modes:
classic index.Storage scanning, all in-memory with corpus scanned from the index.Storage,
and the in-memory corpus built up over time as blobs arrive.
Change-Id: I40536e498a63bece5bd4897cdbbd0cef78085f44
2013-11-16 17:24:02 -08:00
c573feb226
Make the search handler conditionally request the index be slurped into memory.
...
Change-Id: I7fadeba41c3b1bbcfbb0fd3470ff30cdf04b1b90
2013-11-16 15:18:16 -08:00
705107ad80
search/index: invert depedency. search now depends on index.
...
creates new package types/camtypes for misc types needed by both. might eventually go away as
search matures.
Change-Id: Ib771ead7bea39936ba478b7e5d58de997060861b
2013-11-16 15:00:30 -08:00
25eca52283
Merge "pkg/search: add modtime in WANT to fix tests"
2013-11-16 22:26:28 +00:00
7f8e035be6
search: little TODO
...
Change-Id: I38a98c9c9b77a899536cf0d7601ce2030d6f4dda
2013-11-16 12:54:11 -08:00
8a5161ec30
search: multiple constraints at top level
...
Change-Id: I964a0d394c3c03e2409c91fb1d240847ff07ddb3
2013-11-16 11:21:30 -08:00
17a9d061a0
search: ditch BlobSizeConstraint, use IntConstraint instead
...
Change-Id: I99bece643841feab777175479743b3bd1fc421b8
2013-11-16 11:10:09 -08:00
1e627ce7c3
search: change BlobMeta to have a CamliType, not a misleading MIMEType.
...
Change-Id: I1c6e111ef27aafd5a15f9b38746361093a78e9e6
2013-11-16 10:35:18 -08:00
01a8dbb163
pkg/search: add modtime in WANT to fix tests
...
Also fixed fakeindex clock to use a time.Time.
And synchronized the fakeindex and the indextests clocks to the
same origin.
Change-Id: I986abca300d61241069132619028f470cacd61da
2013-11-15 23:31:24 +01:00
9587b8e62e
search: rename AttributeConstraint to PermanodeConstraint, add ModTime constraint
2013-11-14 15:31:44 -08:00
e0c3cf9b01
search: optimize ClaimList.Less sort. duh.
2013-11-14 15:30:59 -08:00
6e7df1c6ee
search: IntConstraint
2013-11-14 15:30:40 -08:00
f718e64289
search: start of DirConstraint. mostly notes. not implementation.
...
Change-Id: I6403b4f9be2aa06778fb1ef96edde2896557c3c9
2013-11-09 18:56:52 -05:00
33b11f86b9
search: skip TestHandler in short mode
...
Change-Id: I6f0c651cf7bf4324c9c4d785400588c2ff08557e
2013-11-09 17:24:35 -05:00
c9443d7c7b
search: TimeConstraint
...
Change-Id: Ic50290109043c1ad906f25935e5fad4c2170ea77
2013-11-09 17:08:46 -05:00
4c70e14310
search: add LimitReader around JSON decoding. add some TODOs
...
Change-Id: I98069ef0aeb8c96cdc997c138222e0344981ead8
2013-11-08 14:49:23 -05:00
4d620d2c60
Merge.
...
Change-Id: I15452c93c287eacf42e50dac59549a23d5e219c2
2013-11-08 14:47:51 -05:00
c477ef43bb
search: FileConstraint, StringConstraint, start of TimeConstraint, EXIFConstraint
...
Change-Id: I8464a392f2671e75898b64295b0c2e2261292ff9
2013-11-08 14:46:00 -05:00
3fb2a0be4f
Adds server-side search query handler.
...
Change-Id: Id1bdaa0f5f5a208ccf41ce25e5c50970f66f9469
2013-11-08 13:39:20 -05:00
59f20db9b2
search: delete unimplemented and now-undesired BlobTypeConstraint
...
The CamliType and AnyCamliType are better.
Change-Id: I0fbe9e645fa0ca13d107f7012bcb0008dfd5d43d
2013-11-08 13:17:37 -05:00
948bc3ba48
search: query support for AttributeConstraint.ValueMatches (subqueries)
...
e.g. show me permanodes with a camliContent pointing to a jpeg over 2.5 MB
Change-Id: I8b9c5c4b3595b64538281f63db7852461fd0367c
2013-11-08 13:11:16 -05:00
9e1948be25
search: ValueSet
...
Change-Id: Ie8b1b2c24ab9dcefb4f5e5d2db885568646ca228
2013-11-08 11:35:21 -05:00
13a732c46b
search: query permanode attributes (a start)
...
Change-Id: I1e6109df0d09edf894a32934eec5be7b6ad9fc1c
2013-11-08 11:25:51 -05:00
5e2a4d8e9a
search: add Handler.Describe, make serveDescribe HTTP handler use it
...
Change-Id: I6f5a45b066aa02ad40cc7fdd2067ac6c178a62ab
2013-10-25 00:23:10 -07:00
abdfd02bbb
search: add camliImportRoot to set of indexed properties
...
Change-Id: I63a16e704569b2c834cda7cc9c84ce14bb31511d
2013-10-24 22:49:31 -07:00
a78704c979
search: add AnyCamliType constraint
...
Change-Id: Ib129d4d365d040e0b12b2e3aa35c38a59ed7bc04
2013-10-18 19:10:59 -07:00
ea079282d6
search: tests for logical operators
...
Change-Id: I3d389c3b58206fd98140834162aec17218bbce14
2013-10-18 18:50:19 -07:00
f515ec9861
search: add test for blob prefix queries
...
Change-Id: If53acf332f8eb72d82735d8707a77a21694e83ba
2013-10-18 18:28:05 -07:00
b9d4b7dc09
search: search on blob size
...
Change-Id: Id87fb65ebe0fec304637c6acb29ea27bd99d3bb1
2013-10-18 18:12:23 -07:00
7101fe1b64
search: search on CamliType
...
Change-Id: Ie4c0655da9407b49a67cd6d11240b697bd9bd6f0
2013-10-18 17:56:56 -07:00
17ecf8df07
search: start of general Query support. very rough draft.
...
Start of something that builds and 'works', but barely fleshed out at all.
Likely to change drastically yet.
Change-Id: I7fba25528a341ff2dbbc4dbb579f2450d7889320
2013-10-18 17:17:35 -07:00
7742892670
Merge branch 'master' of https://camlistore.googlesource.com/camlistore
2013-10-05 20:46:43 -10:00
c8badb4404
Misc cleanups
...
Change-Id: I9183ca1ec7a98df210373f27ec3493c996f00f74
2013-10-05 20:46:39 -10:00
74eb8e6c3c
Further visual refinements to layout of homepage:
...
* Increase default image size a few settings
* Increase gutter between items
* Remove extraneous padding and whitepsace
* Remove unneeded gigantic heading
* Move build version into statusbar
* Change algorithm for fitting rows -- instead of preferring to clip
edges of image, instead prefer to adjust height of rows. I find it
looks more interesting visually when the widths of the images vary
more.
* Replace folder and file icon with something better looking
* Swap order of + and - buttons in toolbar, they were backward from
what is typical
Change-Id: Idfe85a813926c2c9eea774a86046df62ecacd45c
2013-10-04 14:04:38 -07:00
304f5c4b38
camput attr: support -del
...
Change-Id: I28460fed5b572ba9a83cfcb04f38e42124fa723f
2013-10-03 15:07:43 +02:00
128f56887b
Handle GET and HEAD the same in more (all?) places.
...
Change-Id: I486a95f0a6fc127d17dcf0abeb94eab292980156
2013-09-15 20:12:26 +01:00
0375ee2d47
Do directory describe queries in parallel.
...
Change-Id: I9e3a4d97a97b177356d11d4ff0a735d80bfc63db
TODO: use syncutil.Group in many more places.
2013-09-12 16:22:00 +01:00
d488c576fc
search: support for static directory children
...
This change introduces a new index entry
to help with finding the children of a static directory.
It also fixes ResolvePrefixHop so that it takes
into account static directories, and not only collections.
This is the first step to support publishing static directories.
http://camlistore.org/issue/179
Change-Id: I5666e5caa6c782004054ae4c19a6b6119d4fda8b
2013-09-10 23:06:48 +02:00
3a5793b743
search: actually use result of SearchPermanodesWithAttr and don't leak a goroutine
...
Change-Id: Ia1ff8acc1d396901c6b589dbe9a972e3dd4df05f
2013-09-03 18:22:05 -07:00
0bdf20884b
all: delete pkg/blobref; convert all from *blobref.BlobRef to new blob.Ref
...
Change-Id: Id2dfb7f19452bedf4f3c9310b36227fd8117b225
2013-08-03 19:54:30 -07:00
24458fca93
Tile images nicely within BlobItemContainer.
...
Screenshots here: http://imgur.com/XN9qFLP,PeEDPnW,CS35Pw6,VS0nHaG,SNT1ZXZ,2nMWVYz,8RmuLuw,NE8b26n,EFErMdg
Change-Id: I6a66b5b782363882261b80e7ac9930ff1046efa6
2013-08-01 22:25:52 -07:00
c28fcb4cda
search: let the HTTP interface describe multiple blobs at once
...
Change-Id: I1998e0fa9a550fb15589bde6da1b549c14bcb97c
2013-07-11 20:17:05 +10:00
4a404a3c60
search: clean up describe handler and DescribeRequest
...
Change-Id: I4e6184765b6139de3bdc153d27a12d3cb205a55e
2013-07-11 10:35:06 +10:00
d2b8d08397
search: make describe follow camliPath attributes
...
Change-Id: I875587a734d118a3f3fc204901ae1788247b91ab
2013-07-11 10:03:56 +10:00
64153d9f02
client,search: add withattr and describe to client
...
Change-Id: I9d55a7283bc7c99560e61aea80024c8f80dacc05
2013-07-10 21:10:18 +10:00
574b5feb96
Merge "publish: serve a zip of all files in a published path"
2013-07-08 13:32:01 +00:00
86b591cab9
publish: serve a zip of all files in a published path
...
Change-Id: I415f4d50936ab419695389dcda1facaf71565a8d
2013-07-08 13:00:41 +02:00
9468e5ba70
More docs. Every package is documented now.
...
misc.CountingReader moves into readerutil.
pkg/atomics is folded into pkg/types.
pkg/test/testdep is folded into pkg/test, with better name/docs.
Old cruft from pkg/webserver is deleted.
Change-Id: I3f72d8b29804254ef944995fb085837c878f79f5
2013-07-07 21:12:30 -07:00
d832b6a06f
Document PrefixHandler and clean up users with accessor functions.
...
Change-Id: Ib0b23ff3cc548a35c35eae1cdc59a8673dc19638
2013-06-12 11:17:30 +02:00
af204bca92
newui: search page, with index page style
...
Change-Id: Iad9a3e914a4ffa79af6629c88a13a9946b7f3a2c
2013-05-19 00:38:19 +02:00
85c0fa9c61
Merge "search: factorize max results sanitation, allow max specified from ui"
2013-05-01 02:22:24 +00:00
e9443ec182
search: refactor serveEdgesTo with annotated structs
...
http://camlistore.org/issue/99
Change-Id: I84ed551db4fecbd1ca06d76759d7512d1651bbf2
2013-04-26 18:33:27 +02:00
7c08f37abe
search: factorize max results sanitation, allow max specified from ui
...
Change-Id: I066369829816189dbd2f682ce003c8b82022d131
2013-04-04 15:38:55 +02:00
0dfd84a7d8
search handler: return correct thumbnail dimensions
...
images: DecodeConfig to get the predicted width
and height after EXIF correction
search&index: add GetImageInfo and use it in search
to predict the thumbnail dimensions
http://camlistore.org/issue/115
Change-Id: I358136a2ab03ea09c8f8fd2fa0dc574921c819c5
2013-03-25 17:06:15 +01:00
5344af8e39
search handler: more tests, and fix for the describe depth race
...
http://camlistore.org/issue/121
Change-Id: I5807df784d9bd47f3cc8720a891d458ee722013b
2013-03-06 19:04:10 +01:00
ace9474d95
index: index file times too, and return in index.GetFileInfo.
...
Change-Id: I59d91f0938c725a4cbdf5ca933cdff3529e25f5f
2013-02-18 21:31:41 -08:00
8155b3867a
Fix case of MimeType to MIMEType, per Go style.
...
Change-Id: I6c67a61cea2c71d7fa5d52c4ae283e33e993fd0f
2013-02-18 11:16:13 -08:00
1b06e32f30
search handler: formalize and split serveSignerPaths
...
Change-Id: I8399b55aed45d3456b0cad89a4cf5bb6f4c57d69
2013-02-13 17:51:39 +01:00
46846e0844
search handler: getClaims: error out if nil permanode
...
Change-Id: Ieeb36ec3c3ed0c9f68abc6a659ba8f0b23a5bbb1
2013-02-12 16:46:16 +01:00
1f2b140bbd
search handler: formalized and split serveClaims
...
Plus a few related cleanups (better httputil use).
Change-Id: I9ea34f2fe658624d05b7a6f347f5d946759a07b5
2013-02-12 01:18:46 +01:00
f1845db855
search: split metaMap into metaMap and metaMapThumbs
...
Change-Id: I5a29166be7ed44a7c6a954ba689f1f3dbfbe57d7
2013-02-10 12:59:05 -08:00
7300184066
httputil, search: more HTTP util usage, deleting code.
...
Change-Id: I90e51a3e8b3fdd868b26a40446f9715aaa7f89c0
2013-02-08 22:21:09 -08:00
e624e24c46
httputil/handler: start of more formal approach to input validation and HTTP error handling
...
Change-Id: I219a6263e1ba2d0e64819c01a885078dbf1c8b4c
2013-02-08 19:04:24 -08:00
3605285924
Add pkg/types with types.Time3339; use it in search/handler JSON types
...
Change-Id: I79bf1484ef8b3548bca2e732f189352bcfbcd6db
2013-02-08 18:10:31 -08:00
0a84607099
search handler: all funcs now rid of manual maps responses.
...
serveDescribe, servePermanodesWithAttr, and serveSignerPaths
rewritten like serveRecent. Got rid of populateJSON.
http://code.google.com/p/camlistore/issues/detail?id=101
Change-Id: Id730d40005476634b138b13260248e497d1a87d0
2013-02-09 02:12:06 +01:00
76735113f9
fs: recent directory works.
...
Change-Id: I13fb418e7ef32c8be8ab635b8179fbded5e18c79
2013-02-07 22:43:50 -08:00
e867ea7dc0
client, search, fs: cammount now getting search queries back
...
Change-Id: I68541e9f9d54fa4240872d440972ad39f6bad33f
2013-02-07 21:55:17 -08:00
a068e4d7aa
search: separate out query logic from HTTP/JSON on GetRecentPermanodes
...
Change-Id: I4afb34aa548e83a2db2374a4cca2a9d5368473be
2013-02-07 21:34:25 -08:00
f8639c5c5c
search: fix error message
...
Change-Id: Ic89e3a32a919414746eeb1c96f3b694fabf9798a
2013-02-07 19:20:37 -08:00
cfa43225d4
search: move thumbnailSize out into a func
...
Change-Id: If7d6ab242fe7fcef2174aa9dad12a85a06ce0091
2013-02-07 09:40:59 -08:00
6cc5f33da1
search: handler response type cleanup (Issue 99).
...
Notably, don't mix 'meta' bag keys with rest of response. Makes it easier for both Go and Closure types to not mix keys from one type with another type.
Change-Id: I65e51f2a6c236c5111b5a9b88830e1220d6b29d4
2013-02-07 09:33:00 -08:00
c34fc8c345
search: use a struct for recent items, not manual map
...
Change-Id: I443983b3689434afc0471a55357a78d03c0bb18f
2013-02-07 08:05:31 -08:00
5a8afdf951
search: rename populateJSONThumbnails to populateJSONAndThumbnails
...
Change-Id: Ibd4db3909c01c8a352df1875de703c71d657c428
2013-02-07 07:54:45 -08:00
6e35c1c26e
search: don't fail the whole search request if dir info isn't available
...
Change-Id: Iae78d099c608dbe41f8ce9ccad1a11d70d212c48
2013-01-20 11:12:32 -08:00
67c5678062
Index directories with "fileinfo", and use this to
...
find their name in a search request.
Fixes http://code.google.com/p/camlistore/issues/detail?id=79
Change-Id: I755afd8f52dbd2f8a48ba72bed0a6b0192d1dd71
2013-01-09 16:59:04 +01:00
Brad Fitzpatrick