Commit Graph

29 Commits

Author SHA1 Message Date
Bill Thiede 8810b517da images/fastjpeg: reset checkAvailability in tests.
Reseting checkAvailability each run uncovered the fact that
CAMLI_DISABLE_DEBUG wasn't being properly reset, which affected later
tests.

Change-Id: I19b790ef8182b1bc2d54cbf33fa9dfe84dbe3903
2014-08-14 15:17:18 -07:00
Bill Thiede 4bcaaf82cc images/fastjpeg: use jpeg.Decode when djpeg fails.
Return error of type ErrDjpegFailed when djpeg or parsing the PNM returned
fails.
Attempt to decode image again with standard library.

Change-Id: I2a0cb7b52885732b7cbbffb8e34993d232781bc0
2014-08-12 20:20:20 -07:00
Brad Fitzpatrick deb743d1ec Merge "images: add support for fast jpeg down-sampling." 2014-07-21 17:34:27 +00:00
Brad Fitzpatrick a68f8df380 images: fix broken test
Exif.DateTime now always returns a time.Local location when zone is unknown.

Change-Id: I898516398a88bc8dcd8daf26f87c801a24cc1d7a
2014-07-14 09:36:48 -07:00
Bill Thiede bf6a89c9ad images: add support for fast jpeg down-sampling.
The new package pkg/images/fastjpeg uses djpeg(1) to quickly
down-sample images at load time as described here:

  http://jpegclub.org/djpeg/

Add benchmark comparing std lib jpeg.Decode and fastjpeg.DecodeDownsample
with a factor of 1, 2, 4, and 8.
Benchmark resizes a 4000x4000 pixel JPEG to 128x128.

Refactor in pkg/images to make images.Decode a little more readable.

Change-Id: I571db1f3c3068f99da4a01ec84fd246ef098a18c
2014-07-03 21:06:06 -07:00
Bill Thiede eb7f66fe28 jpeg: enable images/jpeg imported from Go tip.
Addresses https://camlistore.org/issue/463

Change-Id: Ie7b8f937ded78d95875f4cd13b024d0429136981
2014-07-02 21:22:15 -07:00
Brad Fitzpatrick bfe51f3ad5 gofmt
Change-Id: Ie6be315d02a439cbfdd870a0b0b7f16cba12a168
2014-03-20 12:29:45 -07:00
Brad Fitzpatrick 44d8772feb images: minor doc comment update
Change-Id: If7c6ad470c9f3ccb152777225ee955b1f7c28e9c
2014-01-17 16:47:29 -08:00
Bill Thiede cdde1c8803 images/resize: remove commented-out code.
Accidentally snuck in as part of the giant CL
https://camlistore.org/r/1556

Change-Id: I379c613360c4f4c7ca9f3cc67a3b0a570308fa51
2013-12-22 21:22:41 -08:00
Brad Fitzpatrick 30c7d8859a thumbnails: add a cache-busting URL component, support ETag
And quiet noisy logging on normal write failures.

We can now stress test the thumbnail generation by setting
CAMLI_DISABLE_THUMB_CACHE=1 which will make all the thumbnails in the
browsers be unique, and not write them to cache on the server.

Then, when we're happy with the thumbnails, we just increment the
thumbnailVersion string and that busts all the browser- and
server-side caches.

Change-Id: I3cda8e85ab8b1b0b2c9113f6dff613dfbf736028
2013-12-16 20:27:49 -08:00
Brad Fitzpatrick 0d36539f81 Add CAMLI_DISABLE_THUMB_CACHE option for when working on thumbnailer
Change-Id: Ifa99a57c82c67a83fe0b89abaeee407058b00af8
2013-12-16 18:11:52 -08:00
Brad Fitzpatrick 9366723666 Rename pkg/misc/resize to pkg/images/resize
Change-Id: I9b8105cbae0c6dc66ccbb1dd15a48363ee2f1e85
2013-12-16 17:49:28 -08:00
Bill Thiede 2d4fb25c34 images: fix Decode when resize + rotate + max W/H.
Adds more tests to cover rotations with resize when used with
MaxWidth/MaxHeight, previously only ScaledWidth/ScaledHeight were
tested.

Improve tests to compare bounds when determining equality, otherwise
an image sized 0x0 is equal to all other images.

Sort test image filenames so test order is stable and obvious.

Keep more data in memory when indexing images upon receive.  Some
largish CR2 files need more data or the EXIF parsing will fail.

Should address some or all of https://camlistore.org/issue/274

Change-Id: I80d90c33538c9d62ce4480ccb58c003e18ee6629
2013-12-16 10:01:07 -08:00
Bill Thiede 036e35a258 images: performance and memory improvements.
Makes the assumption that our old image logic was 'If the image is
huge (>2400px) shrink it to twice the target thumbnail size with
resample (which is quick) then box filter the image in half (which is
slow)'.  There was a bug there that was causing large intermediary
images to be allocated when the source image was just the right size
and in portrait orientation (mainly trigger by photos from my Nexus 4).

Generalizes resize code to always resample to 2 times the thumbnail
size and then smooth down.  Throws more pixels away faster.

Add downsizing routines that operate inplace.  This greatly reduces
our peak memory usage when generating thumbnails.

Add version optimized for our particular resize case (halving);
providing a speed boost over the generalized versions.

Add tests and benchmarks comparing new resize to old.

Test for pkg/misc/resize can be run with --output=output_dir/ to see
the images generated by the old and new resize routines along with
mask of their differences.

Addresses some of the concerns in https://camlistore.org/issue/237

Change-Id: I6464fa637da9db371f15761bb699c045604b6cb8
2013-12-14 16:28:49 -08:00
Bill Thiede 59ab7e0075 pkg/images: add benchmark for resizing images.
Sample run on my computer:
$ go test -bench . -benchmem
PASS
BenchmarkRescale1000To50              50          45211076 ns/op           92224 B/op          3 allocs/op
BenchmarkRescale1000To100             50          45182185 ns/op          364608 B/op          3 allocs/op
BenchmarkRescale1000To200             50          46602272 ns/op         1445952 B/op          3 allocs/op
BenchmarkRescale1000To400             50          61513579 ns/op         5763136 B/op          3 allocs/op
BenchmarkRescale1000To800             20         104372086 ns/op        23040064 B/op          3 allocs/op
BenchmarkRescale2000To50              10         179838094 ns/op           92224 B/op          3 allocs/op
BenchmarkRescale2000To100             10         180684898 ns/op          364608 B/op          3 allocs/op
BenchmarkRescale2000To200             10         181488279 ns/op         1445952 B/op          3 allocs/op
BenchmarkRescale2000To400             10         186902466 ns/op         5763136 B/op          3 allocs/op
BenchmarkRescale2000To800             10         246418484 ns/op        23040064 B/op          3 allocs/op
BenchmarkRescale4000To50            2000           1305587 ns/op          133251 B/op          5 allocs/op
BenchmarkRescale4000To100            500           5179999 ns/op          528512 B/op          5 allocs/op
BenchmarkRescale4000To200            100          20733512 ns/op         2089088 B/op          5 allocs/op
BenchmarkRescale4000To400             20          82820323 ns/op         8323200 B/op          5 allocs/op
BenchmarkRescale4000To800              5         333289471 ns/op        33280128 B/op          5 allocs/op
BenchmarkRescale8000To50            2000           1397384 ns/op          133251 B/op          5 allocs/op
BenchmarkRescale8000To100            500           5219709 ns/op          528515 B/op          5 allocs/op
BenchmarkRescale8000To200            100          20863617 ns/op         2089089 B/op          5 allocs/op
BenchmarkRescale8000To400             20          83260367 ns/op         8323200 B/op          5 allocs/op
BenchmarkRescale8000To800              5         331035176 ns/op        33280128 B/op          5 allocs/op
ok      camlistore.org/pkg/images       46.780s

Change-Id: Iab21f28f06681faf44b01a75c3bb8b23fb508a81
2013-12-08 22:39:49 -08:00
Bill Thiede 630f6ca61f Document environment variables usage.
Running 'go run dev/envvardoc/envvardoc.go' now shows:
'All environment variables are documented'

I also took the liberty of cleaning-up our mishmash of logic for handling
boolean environment variables, and cleaned up a couple other spots that didn't
seem right.

This change adds docmentation for all variables starting with (CAM|DEV|AWS).
This leaves some variables still undocumented.  If there are variables worth
documenting in the following list, maybe we should rename them to have a
CAM{LI} prefix for consistency's sake:

APPDATA                  pkg/osutil/paths.go:86
APPDATA                  pkg/osutil/paths.go:102
DISPLAY                  pkg/misc/gpgagent/gpgagent.go:126
GOPATH                   pkg/fileembed/genfileembed/genfileembed.go:321
GOPATH                   pkg/osutil/paths.go:168
GOPATH                   pkg/test/world.go:54
GOPATH                   server/appengine/build_test.go:77
GPGKEY                   cmd/camput/init.go:77
GPG_AGENT_INFO           cmd/camput/init.go:153
GPG_AGENT_INFO           pkg/misc/gpgagent/gpgagent.go:50
HOME                     pkg/jsonsign/keys.go:79
HOME                     pkg/jsonsign/signhandler/sig.go:64
HOME                     pkg/osutil/paths.go:36
HOMEPATH                 pkg/osutil/paths.go:34
PKG_CONFIG_PATH          pkg/index/sqlite/dbschema.go:59
RUN_BROKEN_TESTS         pkg/fs/fs_test.go:67
SKIP_DEP_TESTS           pkg/test/testdep.go:29
TERM                     pkg/misc/gpgagent/gpgagent.go:133
TERM                     pkg/misc/pinentry/pinentry.go:99
TESTING_PORT_WRITE_FD    pkg/webserver/webserver.go:135
TEST_GPGAGENT_LIB        pkg/misc/gpgagent/gpgagent_test.go:27
USER                     pkg/netutil/ident.go:135
USER                     pkg/osutil/paths.go:45
USERNAME                 pkg/jsonconfig/eval.go:228
USERNAME                 pkg/osutil/paths.go:43
VERBOSE_FUSE             pkg/fs/fs_test.go:133
VERBOSE_FUSE_STDERR      pkg/fs/fs_test.go:137
XDG_CONFIG_HOME          pkg/osutil/paths.go:104

Change-Id: Ief28710d3deefd1e65247cb5d3b1d8dde73e1f2d
2013-09-06 21:54:03 -07:00
mpl 0dfd84a7d8 search handler: return correct thumbnail dimensions
images: DecodeConfig to get the predicted width
and height after EXIF correction
search&index: add GetImageInfo and use it in search
to predict the thumbnail dimensions

http://camlistore.org/issue/115

Change-Id: I358136a2ab03ea09c8f8fd2fa0dc574921c819c5
2013-03-25 17:06:15 +01:00
mpl fbcb4411df images: move rescaling from server/image to pkg/images
Fixes http://code.google.com/p/camlistore/issues/detail?id=94

Change-Id: Ifa73e0a3ccbbcaef31ae8870d39f63b8a90aad26
2013-02-18 16:44:41 +01:00
mpl 932c13acb7 exif: use DateTimeOriginal if present. Added test for it.
Change-Id: Idfd75b15d7302b5c3a6fbe3464917b16bce56558
2013-01-31 22:06:45 +01:00
mpl d36d5c60d4 images: use the goexif api properly to get orientation tag
Change-Id: Ibcc3880bf93faf5deacded75e2e3b5b44187e0cb
2013-01-21 00:53:29 +01:00
Brad Fitzpatrick d4e4653ae3 images: fix potential crash after decode failure
Change-Id: Ifcf2a09af5a50728bc0b7a9d2208c2580c7c4271
2013-01-19 11:11:59 -08:00
mpl 717dc05fcc images: Decode now returns a Config with a Modified field
This allows to know after a call to images.Decode if the image
was actually rotated or flipped.
Without knowing this, when making a thumbnail, it could happen
(if there was no rescaling required) that an image that needed
being rotated/flipped would not get rotated, because
useBytesUnchanged would still be true.

Change-Id: Ifc5e1e1f5a8543e6754102e3b3a685b736ae8673
2013-01-19 00:19:06 +01:00
Brad Fitzpatrick e16153306a Quiet some image log spam; behind a flag now
Change-Id: Ib43ae05cd98d156e2755536136a991c540d9c97c
2012-12-06 18:01:47 -08:00
mpl f26cd134de Do not error out when "Orientation" tag is absent
Change-Id: I76cfc8066986614c5690f0e27b217fb96b52ae36
2012-11-19 23:40:26 +01:00
mpl fa1269da45 import github.com/camlistore/goexif as a third party
Change-Id: I34842677f4d9335df2478692e0b0d169d00d0942
2012-11-19 23:04:04 +01:00
mpl fbead58f92 fix tests for images and netutil
Change-Id: I89d978ce25cc8748ffab54aa3112e6b854f73b85
2012-11-19 19:52:06 +01:00
mpl 94541905ff use pkg/images (EXIF help) for thumbnails in ui
Change-Id: I60f0102efd1eb5b1c9ca070fe959b499ed8f7c5a
2012-11-15 01:53:58 +01:00
mpl 858326fec1 images: exif, rotation, flipping
This change implements rotation and flipping on images.
It can be done automatically, using the EXIF Orientation as a hint,
or optionally forced.

Change-Id: I97e887599d6a191964344e81cf8e90922313d958
2012-11-12 21:26:53 +01:00
Brad Fitzpatrick a2e01c513b Start of images package and EXIF test files.
Change-Id: Ie623ddc7f6df5cbde890a7c0a47affe91780534b
2012-11-04 16:16:02 +01:00