Commit Graph

185 Commits

Author SHA1 Message Date
Brad Fitzpatrick bf041c8955 search: escape r.AttrFilter in ClaimsRequest.URLSuffix
Change-Id: I90e5a87330dc6ae820f2f5f6e3887a26f341a2b2
2014-03-19 14:59:13 -07:00
Brad Fitzpatrick 0cbfae59e1 Merge "Added AttrFilter to ClaimsRequest" 2014-03-19 21:57:48 +00:00
Hernan Grecco 16d6bbb065 Added AttrFilter to ClaimsRequest
This allows server-side filtering of claims by claimtype

Change-Id: I1f5368c17f8f9f59ea0add7df67033447e37bc61
2014-03-19 00:45:33 -03:00
Steven L. Speek 146a42cc51 search: accept 'and', 'or', and parentheses in expressions
'and' has precendence over 'or'.
both operators are left associative
parenthesized expressions are evaluated first
Parser refactored, parseAtom split up.

Change-Id: I1f194cc75df49bad9d30d041d689d8ba833076f1
2014-03-18 16:40:44 -07:00
mpl 9cadbcd5bc publish: use generic queries
Use generic queries instead of specialized index queries. This is a step
towards the publisher app, because its client will have to use generic
queries.

Context: http://camlistore.org/issue/365

Change-Id: I2781a345e024174e3bea8511b6cdc6f342d5a7c1
2014-03-18 17:26:29 +01:00
Brad Fitzpatrick bf6031a397 search: add operator "childrenof:sha1-xxxxx" to find permanode children of a parent
Change-Id: I9e756147c70f5230aa843ab215dafe5ed6d80087
2014-03-13 19:02:26 -07:00
Brad Fitzpatrick bfa412de5d search: implement some of RelationConstraint (only the "parent" relation)
Allows searching for permanodes based on their parent(s).

Change-Id: I949f159edc43d895e1657bd6ae4c983758ac59f2
2014-03-13 18:49:28 -07:00
Brad Fitzpatrick bf48a198c8 search: add test for camliContentImage describing
Test was missing from bf5cc362b5

Change-Id: Ie42bef6f7cff19b69e3bad37f42b8d3a694d9a28
2014-03-10 20:11:32 -07:00
Brad Fitzpatrick bf5cc362b5 search: also describe camliContentImage
Change-Id: I64cd4818dead81f5a0dac0e275994df17b7740ef
2014-03-10 19:53:56 -07:00
Brad Fitzpatrick bfe5eafb44 search: remove check on negative search limit.
It's defined and implemented to mean infinite.

Change-Id: Ia50fd2b101d800ba442b4767ddf1c9bf57ea999f
2014-03-10 19:31:33 -07:00
Brad Fitzpatrick bf6d91545c search: remove expr TODO
Change-Id: Iea7826f2924ad229585a1ec3921fceb6130e8207
2014-03-08 21:11:03 -08:00
Steven L. Speek b326c2db11 attribute search
Change-Id: I9638ca9ef325076122721c601aade05ce99c0c24
2014-03-08 20:58:42 +01:00
Brad Fitzpatrick 2b27dd53c0 Handle Photoshop images better.
We still can't find their width/height or thumbnail the, but at least
we don't wedge the UI now.

Fixes camlistore.org/issue/389

Change-Id: I8af6cefa5d84e7f6e26f3920e4dd6e1d5eb8b192
2014-03-04 10:58:51 -08:00
Aaron Boodman 5a1e831a7b Bump is:pano threshold to 2.0.
Previous value of 1.6 was too small because iPhone 5's aspect ratio
is 1.78, meaning all landscape iPhone 5 photos were included in
is:pano. Same with Moto-X and probably other newer phones.

I tried 1.8, but it still "felt" too small. I think the correct
value is atleast 2.0, but perhaps larger. I don't have enough test
data in that range right now to get a good feel for it.

Change-Id: I6a3e054290d6f434afac8142a9abc68e53008229
2014-02-08 15:48:29 -08:00
Brad Fitzpatrick b15cde9bce Merge "pkg/search: Add durationms to MediaTagConstraint comment." 2014-01-26 18:45:33 +00:00
Daniel Erat ef6e0bc696 pkg/search: Add durationms to MediaTagConstraint comment.
Change-Id: I8c097c2c88afaf01b9eb0f62072f5e68bddf5ca0
2014-01-25 17:22:33 -08:00
Brad Fitzpatrick 92b23e391d search: basic lexer to supported quoted tokens in search expressions
Change-Id: I6c8c37e3b27ab947623bbe7431352a14aa029839
2014-01-25 11:04:02 -08:00
Daniel Erat 404548d31a pkg/index: Index more music-related properties.
Add disc and mediaref (a hash of the audio portion of the
file).

Also relocate taglib code to
third_party/github.com/hjfreyer/taglib-go.

Change-Id: I58364f525b787484af894663125163095256d7c6
2014-01-22 21:25:05 -08:00
Brad Fitzpatrick 5b03c3f8fb search, index: let media tags be searchable too.
git push from Dolores Park. Sorry, no tests. Dan Erat will tell me if
this doesn't work.

Change-Id: I557cc3d07983390b8a15b7756ee0825fced2f503
2014-01-20 15:47:36 -08:00
Brad Fitzpatrick 94ff6ac141 pkg/search, pkg/client, cmd/camtool: add 'camtool claims' subcommand
Change-Id: Id9d8381facd97a946c3582b226d433ab59b18da7
2014-01-14 11:48:49 -08:00
mpl 5a58142c2c search: title search
Add the title:foo operator to search, to search permanodes which have a
title containing foo, case insensitively.

http://camlistore.org/issue/329

Change-Id: I661df159a4057a889a7ed110b2e724fc6e43b32b
2014-01-10 17:35:30 -08:00
Daniel Erat 396c8a6211 search: Add ValueMatches{Int,Float} to PermanodeConstraint.
Adds constraints for int- and float-valued attributes.

Change-Id: Ie8354f8ea12f604b45cf2fa6a7fb45170bab7e46
2014-01-04 20:43:32 -08:00
Daniel Erat 0ea90ebec3 search: Add StringConstraint.CaseInsensitive.
Provides support for case-insensitive string comparisons.
The implementation needs work (e.g. substring searches do
potentially-large allocations).

Change-Id: I30f068094b4617b81aee2e8981f87f6ec2bc0000
2014-01-04 18:29:52 -08:00
Brad Fitzpatrick 2002359f73 Merge "search: Fix ZeroMax ValueNum permanode queries." 2014-01-05 00:07:55 +00:00
Daniel Erat 945073baf9 search: Fix ZeroMax ValueNum permanode queries.
Searching for permanodes with zero values for an attribute
returned no results due to a check that at least one value
matched. Only perform that check when a value constraint was
provided.

Change-Id: Ia25a595e4598fe682f2a85babca0b8438681a210
2014-01-04 16:05:54 -08:00
Bill Thiede 9aeb0aa2c7 pkg/client: Search fix zero-value Describe queries.
Adds support for zero value blob.Ref {Unm,M}arshalJSON.

Fix for:
https://camlistore.org/issue/308

Change-Id: I910fc4f05015b0c3ddc57eba9d1d8fe1bfe992bf
2014-01-04 08:11:23 -08:00
Brad Fitzpatrick a576379cb5 search: PermanodeConstraint.Time time constraint, 'before:' and 'after:' operators
Work in progress, but works enough to commit now.  Determing the time
of things has many TODOs, and there's some performance work to be done
(although it still appears to be instant... it just uses more CPU than
it should)

Change-Id: I4b04b5805353dfbde0b841a3a557fd0b7c297780
2013-12-31 18:31:02 -08:00
Brad Fitzpatrick dbe9b83cfa search: some refactor cleanup in prep for sorting work
Change-Id: I71d422774bb68a93c9f9d648de8e4e5bc2ef1d87
2013-12-30 21:03:29 -08:00
Brad Fitzpatrick 8a05c520f8 search: make sort type encode as a string in JSON
Change-Id: I50c5663e850f3e8710381cc59daf246c8a600482
2013-12-29 16:28:06 -08:00
Brad Fitzpatrick 7d4c86d0de client: add a Search method
Change-Id: Iec4c22bf6e4bbe40645ec178b5c2d23392ad270d
2013-12-27 16:22:06 -08:00
Bill Thiede 63096a3133 pkg/search: fix is:pano test.
Fix breakage in
https://camlistore.org/gw/7b6790f56ab3f634fb42908731b211ec2236b6cc

Change-Id: Iba1084944cec54c11702911a88c584ee0efe5fec
2013-12-24 21:25:28 -08:00
Brad Fitzpatrick 7b6790f56a search: make is:pano be a bit wider.
Change-Id: I73a27b296a62d77bd1f8967a690a361f7aa4edde
2013-12-24 17:55:46 -08:00
Brad Fitzpatrick bc15a1d7a6 search: fix two double-RLock deadlocks
Change-Id: I5d083150cb1dc4e43c6a64146dee8a3672b7ea40
2013-12-24 13:46:18 -08:00
Brad Fitzpatrick f8d9e6663d geocode: move logging
Change-Id: I42d901510f9d8085c29792fd6b7ea8e1dc082c06
2013-12-24 13:43:25 -08:00
Brad Fitzpatrick eb3407e3e1 search: add format:foo operator
Change-Id: Ia12dcc43bba019f7d6511cec36e4091c692113fb
2013-12-23 21:07:10 -08:00
Brad Fitzpatrick 04bed02906 search: add negative expressions and has:location
e.g. [is:image -has:location]

Change-Id: I03df0a1d5fea8394a33e3079d131224c116d4c0d
2013-12-23 20:39:06 -08:00
Brad Fitzpatrick 7238bd1652 search: GPS location search
Like loc:hawaii or loc:USA or loc:94128, etc.

Change-Id: I11f47bf464a812f0b62e7799752811144bb7454e
2013-12-23 19:11:55 -08:00
Brad Fitzpatrick 4a53f4d687 search: add width:[min]-[max] and height:[min]-[max] operators
Change-Id: I18a82d8f1d2ba519d78b234a240ff854f36a8c10
2013-12-22 19:26:49 -08:00
Brad Fitzpatrick 06a334c39f search: add is:portrait, is:landscape, and is:pano
Also support Width and Height constraints on raw search (not yet
exposed to the expression syntax, but will be like width:<640 or
height:100-200 probably)

Change-Id: I082e3d27b5ef5c238a8e65a6b836943d9f9ff8a6
2013-12-22 18:30:27 -08:00
Brad Fitzpatrick 82a356ab77 search: add FloatConstraint
Change-Id: Ic367e87cb239258a76a99c9d07f0956d67e242ea
2013-12-22 18:11:44 -08:00
Brad Fitzpatrick f27c12ee6a search: add is:image operator
Change-Id: Iba693538b4e7b3d0cf08d5e8601d3b5819000ca1
2013-12-22 15:11:31 -08:00
Dustin Sallings 84ff81d563 Test search describe with time travel
Change-Id: I6bd8dbfa64a27f4fa491655d52371487f6ea9bb5
2013-12-22 01:21:02 -08:00
Aaron Boodman e15052302b Implement the new "expression" support in the web ui
Change-Id: Ic0521f770ffd45c24505ff386b2c044dc307e284
2013-12-21 22:10:17 -08:00
Brad Fitzpatrick 130be113fd Merge "Include camliPath attributes as members from DescribedBlob.Members" 2013-12-22 01:38:45 +00:00
Brad Fitzpatrick 516751574e Merge "Support requesting claims as of a particular time" 2013-12-22 00:31:04 +00:00
Dustin Sallings f805ed2ba6 Support requesting claims as of a particular time
Change-Id: I44763bc7ffc59d08e577d7bbb74a00b3c798206d
2013-12-21 16:27:37 -08:00
Brad Fitzpatrick 79fb299ff6 search: server-side search expression eval + result set continuation tokens
Search expressions (e.g. "tag:funny location:Portland") can now be
evaluated server-side (expr.go and expr_test.go, still nascent). Next
step is to remove this parsing and construction from the javascript UI
code.

In addition, the search result now contains a continuation token to
continue interating through the result sets in a subsequent query.
The Javascript UI should use this now instead of parsing out
modification times of things and altering the search.  If there's no
continuation token, that either means the end has been reached, or
that search type doesn't [yet] support iteration. Currently only
permanode-based queries are supported at many layers, including
continuation, but will be implemented later.

For now, the web UI should send a search query "expression" value of
non-zero length but just whitespace (like a single space: " ") to get
the home page (recently-modified permanodes") because setting both to
the zero value is an input validation error.  We should probably make
an expression operator to be more clear here (like "want:permanode
sort:recent").  But the empty space hack works for now.

Change-Id: I3ea92eb8c776159f53c49db1a7439a91c507940a
2013-12-21 15:59:11 -08:00
Ulf Holm Nielsen daa4379bd3 Include camliPath attributes as members from DescribedBlob.Members
Change-Id: I30272acd1cae805cf9477f99611275d32231b52a
2013-12-18 13:53:00 +01:00
Brad Fitzpatrick 30c7d8859a thumbnails: add a cache-busting URL component, support ETag
And quiet noisy logging on normal write failures.

We can now stress test the thumbnail generation by setting
CAMLI_DISABLE_THUMB_CACHE=1 which will make all the thumbnails in the
browsers be unique, and not write them to cache on the server.

Then, when we're happy with the thumbnails, we just increment the
thumbnailVersion string and that busts all the browser- and
server-side caches.

Change-Id: I3cda8e85ab8b1b0b2c9113f6dff613dfbf736028
2013-12-16 20:27:49 -08:00
Brad Fitzpatrick a078ce9406 search: generate an optimized matcher and use it within the matching loop
benchmark                         old ns/op    new ns/op    delta
BenchmarkQueryRecentPermanodes        27987        27512   -1.70%

benchmark                        old allocs   new allocs    delta
BenchmarkQueryRecentPermanodes           99           87  -12.12%

benchmark                         old bytes    new bytes    delta
BenchmarkQueryRecentPermanodes         6433         6207   -3.51%

But will be better in practice with more blobs.

Change-Id: I4673702231c52864249329818145aad11da2b4fc
2013-12-14 12:07:35 +04:00