Commit Graph

3500 Commits

Author SHA1 Message Date
Aaron Boodman 054b67bc04 Merge "New Navigation UI." 2013-12-17 03:30:07 +00:00
Aaron Boodman c4e74959a2 New Navigation UI.
I'm sure there's still lots of refinement that needs to be done here, but this
seems like a good checkpoint.

Change-Id: Ida808464c0cc40c84f7f79c016d3c6a148d50a32
2013-12-16 19:28:20 -08:00
Brad Fitzpatrick c0a4f1664d Make thumbnails quality 90, not 75
Change-Id: Icd3e102fbb1e95a9e6a46ff187f56ff14d1842d5
2013-12-16 18:54:20 -08:00
Brad Fitzpatrick 0d36539f81 Add CAMLI_DISABLE_THUMB_CACHE option for when working on thumbnailer
Change-Id: Ifa99a57c82c67a83fe0b89abaeee407058b00af8
2013-12-16 18:11:52 -08:00
Brad Fitzpatrick 9366723666 Rename pkg/misc/resize to pkg/images/resize
Change-Id: I9b8105cbae0c6dc66ccbb1dd15a48363ee2f1e85
2013-12-16 17:49:28 -08:00
Brad Fitzpatrick d9e7b03fca ui: subscribe to attribute changes over websocket too
Change-Id: I0ecdbe9ff321cd46e86534c7da1e90f48d0eb6f5
2013-12-16 17:37:17 -08:00
Brad Fitzpatrick 60685a1194 auth: warn when local connection uid doesn't match
Change-Id: Icdcef55c4831b4f77f7df34e58c87a6985401a04
2013-12-16 17:19:31 -08:00
Brad Fitzpatrick 67341654ad Merge "images: fix Decode when resize + rotate + max W/H." 2013-12-16 22:21:42 +00:00
Brad Fitzpatrick 599fb8c6b3 Merge "pkg/index: move sqlite implementation to sorted" 2013-12-16 18:34:40 +00:00
Brad Fitzpatrick 4fca427a28 misc/resize: don't depend on Go 1.2 image/color/palette in test 2013-12-16 10:19:55 -08:00
Bill Thiede 2d4fb25c34 images: fix Decode when resize + rotate + max W/H.
Adds more tests to cover rotations with resize when used with
MaxWidth/MaxHeight, previously only ScaledWidth/ScaledHeight were
tested.

Improve tests to compare bounds when determining equality, otherwise
an image sized 0x0 is equal to all other images.

Sort test image filenames so test order is stable and obvious.

Keep more data in memory when indexing images upon receive.  Some
largish CR2 files need more data or the EXIF parsing will fail.

Should address some or all of https://camlistore.org/issue/274

Change-Id: I80d90c33538c9d62ce4480ccb58c003e18ee6629
2013-12-16 10:01:07 -08:00
mpl b62c94fdd1 pkg/index: move sqlite implementation to sorted
Change-Id: I1e300ffa14547fabeac8d255fd694054ebcb9e53
2013-12-16 17:35:27 +01:00
Brad Fitzpatrick 49c2eea50a add TODO for a new test
Change-Id: I6fadfa6bfcbcc27753e12f3def96123ce3715689
2013-12-16 08:15:44 -08:00
Brad Fitzpatrick ab635c6ab3 misc UI TODOs
Change-Id: Ic874bd8fac550d35268f006e1e1beb3bd88ba3b5
2013-12-15 20:58:53 -08:00
Brad Fitzpatrick 7c9b72425e Merge branch 'master' of https://camlistore.googlesource.com/camlistore
Conflicts:
	pkg/server/image.go

Change-Id: Ieae89d752f4c6602e0ce31f90e719b07612acb43
2013-12-14 17:20:30 -08:00
Brad Fitzpatrick f202aaa665 Merge "images: performance and memory improvements." 2013-12-15 01:18:02 +00:00
Bill Thiede 036e35a258 images: performance and memory improvements.
Makes the assumption that our old image logic was 'If the image is
huge (>2400px) shrink it to twice the target thumbnail size with
resample (which is quick) then box filter the image in half (which is
slow)'.  There was a bug there that was causing large intermediary
images to be allocated when the source image was just the right size
and in portrait orientation (mainly trigger by photos from my Nexus 4).

Generalizes resize code to always resample to 2 times the thumbnail
size and then smooth down.  Throws more pixels away faster.

Add downsizing routines that operate inplace.  This greatly reduces
our peak memory usage when generating thumbnails.

Add version optimized for our particular resize case (halving);
providing a speed boost over the generalized versions.

Add tests and benchmarks comparing new resize to old.

Test for pkg/misc/resize can be run with --output=output_dir/ to see
the images generated by the old and new resize routines along with
mask of their differences.

Addresses some of the concerns in https://camlistore.org/issue/237

Change-Id: I6464fa637da9db371f15761bb699c045604b6cb8
2013-12-14 16:28:49 -08:00
Brad Fitzpatrick 4edcdd5098 Use singleflight when generating thumbnails.
Change-Id: I3691f912748a85db09f96029584129e0897191ef
2013-12-14 12:05:51 -08:00
Brad Fitzpatrick 79f48ad1f2 More thumbnail cleanup, in prep for putting singleflight in.
Change-Id: I11c60ee7d14f330c20ca916fbce963d3c59d561f
2013-12-14 10:53:09 -08:00
Brad Fitzpatrick 5c5666d037 Thumbnail cleanups and let the meta map persist on disk with a sorted.KeyValue
Previously, every time you restarted the server, it forgot about all
thumbnails previously generated.  (in practice it didn't/doesn't
matter with a single user, though, since they're still cached in the
browser and we always reply to If-Modified-Since immediately without
checking the cache)  But it'll matter more with the Publish handler.

Also, rename some stuff, clean up some stuff, drop an unused interface.

And then necessarily change the serverconfig low-level generator to use
a kvfile for the thumbmeta map when using local disk for blobs.

--

Change-Id: I4dcfcb21429a440aa118794c03f7abf7bd69c33b
2013-12-14 09:37:56 -08:00
Brad Fitzpatrick 1fe9afd480 Unexport UIHandler's PublishRoots
Change-Id: I844d2f0cd7ef2b81caf3035ed8abaeef023b90ee
2013-12-14 17:41:26 +01:00
Brad Fitzpatrick eb483eb9f6 Limit number of image resizes happening at once.
Attempt to ease memory spikes.

Change-Id: I4df7177c063b284ac841335d8f18b1a6da5289d0
2013-12-14 15:25:39 +01:00
Brad Fitzpatrick a159d20b41 server: unexport some image stuff, add docs, TODOs, clean up
Change-Id: I4914de33406da08a4bbf806d9d0386a285e4a8d7
2013-12-14 15:16:29 +01:00
Brad Fitzpatrick 80ea29de8f mongo: use SKIP_DEP_TESTS again. regressed in recent refactor.
Noticed because 'make presubmit' was breaking when offline.

Change-Id: I1afd12eb2ae8f504018b8a30a91d2d2e87d35999
2013-12-14 15:15:09 +01:00
Brad Fitzpatrick 60e05bf90b Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-12-14 14:13:17 +04:00
Brad Fitzpatrick a078ce9406 search: generate an optimized matcher and use it within the matching loop
benchmark                         old ns/op    new ns/op    delta
BenchmarkQueryRecentPermanodes        27987        27512   -1.70%

benchmark                        old allocs   new allocs    delta
BenchmarkQueryRecentPermanodes           99           87  -12.12%

benchmark                         old bytes    new bytes    delta
BenchmarkQueryRecentPermanodes         6433         6207   -3.51%

But will be better in practice with more blobs.

Change-Id: I4673702231c52864249329818145aad11da2b4fc
2013-12-14 12:07:35 +04:00
Brad Fitzpatrick d6691c16de search: add a basic query benchmark
Change-Id: I1da18ef34a5d1f83bb5be2f9684a64bc43312bd7
2013-12-14 12:06:10 +04:00
Brad Fitzpatrick b406cb3475 query_test: remove some comments, now that they're on RelationConstraint
Change-Id: Id38dcbb2cfce4ab15ec55c08005356fb2e018ee6
2013-12-14 10:33:15 +04:00
Nick O'Neill 8ef0276246 Merge "Added some progress towards search." 2013-12-13 18:56:18 +00:00
Nick O'Neill 7b194895da Merge "Upload by way of significant location change service." 2013-12-13 18:56:07 +00:00
mpl 3c5a6e27af Merge "devcam server: properly wipe mongo" 2013-12-13 16:40:24 +00:00
mpl 6d19d64b88 devcam server: properly wipe mongo
Change-Id: Ib75a460a4be84e21affc662d45b3cf1e321b15bd
2013-12-13 17:37:19 +01:00
Brad Fitzpatrick 6bc7a83ee0 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-12-13 20:34:09 +04:00
Brad Fitzpatrick 2921a320cb Merge "index: move kvfile implementation to sorted" 2013-12-13 16:33:38 +00:00
mpl b86e95caf4 Merge "camtool: dbinit mongo support" 2013-12-13 16:27:49 +00:00
mpl 8c562e9135 camtool: dbinit mongo support
Change-Id: I289759ed9bdb34a6e108af1364c4a6bcfbb5982f
2013-12-13 17:25:03 +01:00
mpl ab5e385113 index: move kvfile implementation to sorted
Change-Id: Ie6e676570af088246ef028a2f003b537d85dd6ae
2013-12-13 16:59:45 +01:00
Brad Fitzpatrick addc303226 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-12-13 06:56:01 -08:00
mpl 39996424c7 sorted/mongo: close iter now that we have recent mgo
Change-Id: If8e7f2ba611f2d3704acece56e03804bf9bd44f7
2013-12-13 15:44:18 +01:00
Brad Fitzpatrick 9af5a970de Mark boring files generated by FUSE as "hidden", then hide in the UI.
Makes for prettier FUSE demos w/ files showing up live in the browser
with websockets.

Change-Id: I41cc9ae0d33db39b4dbbf5b60714cee9a79b7248
2013-12-13 13:27:48 +04:00
Brad Fitzpatrick 94b96d90a8 Merge "third_party/mgo: updated to rev 253" 2013-12-13 07:26:03 +00:00
Brad Fitzpatrick 617bbf5295 replica: support blobserver.Generationer
Change-Id: Iad3a0e040fcffa7bee2cfe5ff7ce8904d4ded36c
2013-12-13 11:24:34 +04:00
Brad Fitzpatrick bf203d6ff7 Document sorted.Wiper more.
Change-Id: Iede5cf3ec12453910594c682a3a18f8c237fbd40
2013-12-13 11:18:27 +04:00
Brad Fitzpatrick 9f8e9e57fd Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-12-13 11:15:29 +04:00
mpl 96f0e39700 third_party/mgo: updated to rev 253
Change-Id: I995db6e123b1ef3d6061774aa333c7dbe645a744
2013-12-13 00:51:30 +01:00
mpl fcbbf2a4df index: move mongo implementation to sorted
Also devcam server -wipe wasn't wiping for mongo anymore, now fixed.

Change-Id: Iecc9d8025ddfba8d8ae9417ee170baf02be1d52f
2013-12-13 00:25:40 +01:00
Brad Fitzpatrick 5eeef19472 Make CAMLI_FAST_DEV work again.
Change-Id: I294de23a50d48c7c2325780b50141a7f02d55aab
2013-12-12 16:18:24 +04:00
Brad Fitzpatrick 22101b61d8 flickr: clarify in README that the flag is a devcam flag, not server flag
Change-Id: Iaab951c326ff90f09b6751f8fe4d439743a5e430
2013-12-12 15:39:44 +04:00
Brad Fitzpatrick 511a599e3b Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-12-12 15:13:57 +04:00
Brad Fitzpatrick 61deeffef5 Make WebSockets work over https/wss, and make Auth work with WebSockets too.
Change-Id: I321f9d7fe80bf4c88771cbb925f3712229cc09ee
2013-12-12 15:13:44 +04:00