Brad Fitzpatrick
27fd9e71a7
Merge branch 'master' of https://camlistore.googlesource.com/camlistore
2013-11-17 15:00:21 -08:00
Brad Fitzpatrick
0b07171c0c
index: change GetOwnerClaims interface for efficiency, rename to AppendClaims
...
Change-Id: Icec235961ed8a09256df87c5fe1455cbad81ae3f
2013-11-17 14:54:30 -08:00
Aaron Boodman
de9e5103e7
Grab the userID from the flickr oauth handshake.
...
Remove the mutex from auth.go because I believe the
mutex in importer.go should guaranteeing only one
flickr importer is running at a time.
Change-Id: Ice971c6c041cf21fdb2bc93cb15f861dd1c8c1d7
2013-11-17 14:26:42 -08:00
Aaron Boodman
9c671f76eb
Revert "Revert "Add genconfig support for flickr importer""
...
This reverts commit 8367de6e95
.
Change-Id: I1fdfa4070572b1badc0f0980348404f905fa13a3
2013-11-17 14:15:12 -08:00
Brad Fitzpatrick
8abaddfa80
index: rearrange interface definition.
...
more stable stuff at the top. stuff that needs a re-think below.
Change-Id: Id1682b393718a2e9928088d38a4ddea477698e98
2013-11-17 13:44:49 -08:00
Brad Fitzpatrick
8367de6e95
Revert "Add genconfig support for flickr importer"
...
This reverts commit b53e79614d
.
It broke the tests.
2013-11-17 13:43:06 -08:00
Brad Fitzpatrick
1d91f3a627
Merge "Add genconfig support for flickr importer"
2013-11-17 21:31:05 +00:00
Aaron Boodman
b53e79614d
Add genconfig support for flickr importer
...
Change-Id: I8be13f81fae15acfe87cce9d65318524efc3035d
2013-11-17 13:29:49 -08:00
Brad Fitzpatrick
04a116fed0
TODOs
...
Change-Id: I6a7866d889f7446bc89c25a19c57d0c8220bf99f
2013-11-17 11:01:09 -08:00
Brad Fitzpatrick
d82f02db50
Search optimization: keep blobs arranged by camliType in memory; use when possible.
...
Change-Id: I8b3ebcf7dfece21d17074da32dc20e4d600e4c50
2013-11-17 10:52:37 -08:00
Brad Fitzpatrick
8e8ac9092a
corpus: add a generation number for future query caching
...
Change-Id: I07dd61756537688f4f5e68bbcaf93bff75aaddc9
2013-11-17 10:05:40 -08:00
Brad Fitzpatrick
494c9f37e5
Merge branch 'master' of https://camlistore.googlesource.com/camlistore
2013-11-17 09:49:51 -08:00
Brad Fitzpatrick
3eb493599e
in-memory search: better structure for keeping memory corpus and kv
...
index in sync, both at start-up and while running and receiving blobs.
They both use the same mechanism now.
Also adds KeyId to the index and Corpus, as the next step. Plenty more
row types remain...
Change-Id: Id79955ba25dc79d5fbd94b0e5248d33dcf71d97e
2013-11-17 09:41:45 -08:00
Brad Fitzpatrick
6fbe62f081
Merge "Bugfix: incorrect baseURL passed to InstallHandlers"
2013-11-17 16:28:07 +00:00
Aaron Boodman
3f7543853a
Bugfix: incorrect baseURL passed to InstallHandlers
...
Change-Id: Iaf26a806358cd82ca09c48d83ca86155e27c289d
2013-11-17 00:41:01 -08:00
Aaron Boodman
fde2936507
Remove some unnecessary code from dummy importer.
...
Change-Id: Ifd1cb70f6f5e11261353e8fe16d17c348ea95f12
2013-11-16 22:43:17 -08:00
Brad Fitzpatrick
b33a785e62
Merge "Rearrange the Flickr importer to use OAuth instead of Flickr's custom importer."
2013-11-17 06:39:24 +00:00
Aaron Boodman
9ae23e52b0
Rearrange the Flickr importer to use OAuth instead of Flickr's custom importer.
...
Change-Id: I8284431fa6572a08d3c7ff0d0e655297627a4eda
2013-11-16 22:30:21 -08:00
Brad Fitzpatrick
5ce4fab0f9
Work around lack of cgo when cross-compiling to arm.
...
Change-Id: Ic230362b9f19af0c99f296f14330c970bf5d010b
2013-11-16 22:22:48 -08:00
Brad Fitzpatrick
272db6545a
index: change GetBlobMIMEType to GetBlobMeta. mime types on a blob was wrong anyway.
...
And some more of in-memory search, which this makes easier / less garbage-y.
Change-Id: If1fa7dd44842244ae5e5440f84e92d66dc3321fb
2013-11-16 19:40:14 -08:00
Brad Fitzpatrick
bbabc52723
test: add Diff function.
...
forgot to commit earlier.
Change-Id: I9986b6bd2266d455837bc20e47b43e4c971654d7
2013-11-16 19:06:30 -08:00
Brad Fitzpatrick
f3cc3c7ed9
search: more in-memory search work. make tests verify Scan doesn't hit Storage.
...
also some string interning work.
Change-Id: I7864b56eb97318bce943afdca3b1212f4729a9a8
2013-11-16 18:50:01 -08:00
Brad Fitzpatrick
e26eacfa2f
index: return FileInfo and ImageInfo as values, not pointers.
...
Change-Id: I6aba3a74ce233cc834550b561f597e549fb2bfab
2013-11-16 17:49:28 -08:00
Brad Fitzpatrick
2984897ac7
search: more in-memory search work.
...
keep blob metadata in memory, and start of testing all search queries in three modes:
classic index.Storage scanning, all in-memory with corpus scanned from the index.Storage,
and the in-memory corpus built up over time as blobs arrive.
Change-Id: I40536e498a63bece5bd4897cdbbd0cef78085f44
2013-11-16 17:24:02 -08:00
Brad Fitzpatrick
c573feb226
Make the search handler conditionally request the index be slurped into memory.
...
Change-Id: I7fadeba41c3b1bbcfbb0fd3470ff30cdf04b1b90
2013-11-16 15:18:16 -08:00
Brad Fitzpatrick
705107ad80
search/index: invert depedency. search now depends on index.
...
creates new package types/camtypes for misc types needed by both. might eventually go away as
search matures.
Change-Id: Ib771ead7bea39936ba478b7e5d58de997060861b
2013-11-16 15:00:30 -08:00
Brad Fitzpatrick
25eca52283
Merge "pkg/search: add modtime in WANT to fix tests"
2013-11-16 22:26:28 +00:00
Brad Fitzpatrick
2a42678f0c
index: start of optional support for keeping index all tightly in-memory for search
...
Change-Id: Ie7fbafa1376505d10b0c01470089d260a0c0f97f
2013-11-16 12:55:09 -08:00
Brad Fitzpatrick
7f8e035be6
search: little TODO
...
Change-Id: I38a98c9c9b77a899536cf0d7601ce2030d6f4dda
2013-11-16 12:54:11 -08:00
Brad Fitzpatrick
8a5161ec30
search: multiple constraints at top level
...
Change-Id: I964a0d394c3c03e2409c91fb1d240847ff07ddb3
2013-11-16 11:21:30 -08:00
Brad Fitzpatrick
17a9d061a0
search: ditch BlobSizeConstraint, use IntConstraint instead
...
Change-Id: I99bece643841feab777175479743b3bd1fc421b8
2013-11-16 11:10:09 -08:00
Brad Fitzpatrick
1e627ce7c3
search: change BlobMeta to have a CamliType, not a misleading MIMEType.
...
Change-Id: I1c6e111ef27aafd5a15f9b38746361093a78e9e6
2013-11-16 10:35:18 -08:00
mpl
01a8dbb163
pkg/search: add modtime in WANT to fix tests
...
Also fixed fakeindex clock to use a time.Time.
And synchronized the fakeindex and the indextests clocks to the
same origin.
Change-Id: I986abca300d61241069132619028f470cacd61da
2013-11-15 23:31:24 +01:00
mpl
131976f165
Merge "pkg/index: use a map to populate the mutations"
2013-11-15 00:27:26 +00:00
mpl
e03d923fe1
pkg/index: use a map to populate the mutations
...
When indexing upon a blob reception, we first populate
all the mutations in a map instead of in a batch mutation.
Then we transfer all the mutations in a batch and commit
it immediately. This makes the window when the batch mutation
is open much shorter, and will ease future indexing because
it allows reading from the index while writing the mutations
to the map.
Change-Id: I276282388f59ca543835bfa5ec64986453b23fe1
2013-11-15 01:23:21 +01:00
Brad Fitzpatrick
169f38ede6
Merge branch 'master' of https://camlistore.googlesource.com/camlistore
2013-11-14 15:36:50 -08:00
Brad Fitzpatrick
9587b8e62e
search: rename AttributeConstraint to PermanodeConstraint, add ModTime constraint
2013-11-14 15:31:44 -08:00
Brad Fitzpatrick
e0c3cf9b01
search: optimize ClaimList.Less sort. duh.
2013-11-14 15:30:59 -08:00
Brad Fitzpatrick
6e7df1c6ee
search: IntConstraint
2013-11-14 15:30:40 -08:00
Brad Fitzpatrick
e5eade1bd6
Merge "httputil: set the headers before writing them"
2013-11-14 21:53:40 +00:00
Brad Fitzpatrick
469b166129
C+A: add Robert Obryk
2013-11-14 13:52:59 -08:00
Brad Fitzpatrick
6a4c0ec09c
TODO: use glog
...
Change-Id: I94ce97b93c73bee47f90a966f5cb1e860b23233a
2013-11-09 19:55:49 -05:00
Brad Fitzpatrick
d6a10c18f2
index: doc tweak
...
Change-Id: I30836b1a6967ac86f7f16118a9bf81a293b09a27
2013-11-09 19:20:21 -05:00
Brad Fitzpatrick
f718e64289
search: start of DirConstraint. mostly notes. not implementation.
...
Change-Id: I6403b4f9be2aa06778fb1ef96edde2896557c3c9
2013-11-09 18:56:52 -05:00
Brad Fitzpatrick
33b11f86b9
search: skip TestHandler in short mode
...
Change-Id: I6f0c651cf7bf4324c9c4d785400588c2ff08557e
2013-11-09 17:24:35 -05:00
Brad Fitzpatrick
c9443d7c7b
search: TimeConstraint
...
Change-Id: Ic50290109043c1ad906f25935e5fad4c2170ea77
2013-11-09 17:08:46 -05:00
Brad Fitzpatrick
4c70e14310
search: add LimitReader around JSON decoding. add some TODOs
...
Change-Id: I98069ef0aeb8c96cdc997c138222e0344981ead8
2013-11-08 14:49:23 -05:00
Brad Fitzpatrick
4d620d2c60
Merge.
...
Change-Id: I15452c93c287eacf42e50dac59549a23d5e219c2
2013-11-08 14:47:51 -05:00
Brad Fitzpatrick
c477ef43bb
search: FileConstraint, StringConstraint, start of TimeConstraint, EXIFConstraint
...
Change-Id: I8464a392f2671e75898b64295b0c2e2261292ff9
2013-11-08 14:46:00 -05:00
Hunter Freyer
3fb2a0be4f
Adds server-side search query handler.
...
Change-Id: Id1bdaa0f5f5a208ccf41ce25e5c50970f66f9469
2013-11-08 13:39:20 -05:00