Commit Graph

24 Commits

Author SHA1 Message Date
Brad Fitzpatrick eb27581da7 httputil: add DecodeJSON and CloseBody helpers, then use them everywhere
httputil.CloseBody closes a http.Response.Body nicely, such that it's
more likely to be re-used.

httputil.DecodeJSON parses JSON from a Response.Body and closes the
body. It also keeps a temporary copy of the input, for error messages.

Then use them everywhere, to improve HTTP TCP connection re-use.

And misc other clean-ups.

Change-Id: I5dc44be0165b13659459ed29ce5ab44c17d9739f
2014-01-28 22:00:52 -08:00
Aaron Boodman 53e12a740c Add jstests.go, a unit test harness for JavaScript files.
Change-Id: I3d755f0001c0528f5db2dbe8cdfc10fa60443c89
2014-01-06 20:21:43 -08:00
Brad Fitzpatrick 3e725e2a98 Change self-signed cert fingerprint to 80 bits of SHA-256.
More secondary preimage resistance. Also better docs and less vague and less promoted Android
UI.

Feedback from Adam Langley, but likely misinterpreted.

Change-Id: I3bf1029b6fc07c0b8dab2af44926aa87b0d4cbad
2014-01-04 16:37:33 -08:00
Brad Fitzpatrick 9366723666 Rename pkg/misc/resize to pkg/images/resize
Change-Id: I9b8105cbae0c6dc66ccbb1dd15a48363ee2f1e85
2013-12-16 17:49:28 -08:00
Brad Fitzpatrick 4fca427a28 misc/resize: don't depend on Go 1.2 image/color/palette in test 2013-12-16 10:19:55 -08:00
Bill Thiede 036e35a258 images: performance and memory improvements.
Makes the assumption that our old image logic was 'If the image is
huge (>2400px) shrink it to twice the target thumbnail size with
resample (which is quick) then box filter the image in half (which is
slow)'.  There was a bug there that was causing large intermediary
images to be allocated when the source image was just the right size
and in portrait orientation (mainly trigger by photos from my Nexus 4).

Generalizes resize code to always resample to 2 times the thumbnail
size and then smooth down.  Throws more pixels away faster.

Add downsizing routines that operate inplace.  This greatly reduces
our peak memory usage when generating thumbnails.

Add version optimized for our particular resize case (halving);
providing a speed boost over the generalized versions.

Add tests and benchmarks comparing new resize to old.

Test for pkg/misc/resize can be run with --output=output_dir/ to see
the images generated by the old and new resize routines along with
mask of their differences.

Addresses some of the concerns in https://camlistore.org/issue/237

Change-Id: I6464fa637da9db371f15761bb699c045604b6cb8
2013-12-14 16:28:49 -08:00
Brad Fitzpatrick f3f48eb70b Support alternate S3 hostnames
Change-Id: I9a27eb5fb9c4e5ef3b516ce9cf5a4b5bb4e1df61
2013-09-01 09:50:35 -07:00
Brad Fitzpatrick 91cdda8475 Match common Go case
Change-Id: I14606ee5ba422cf43f68dc65af2b40a096c7ca35
2013-09-01 09:13:13 -07:00
mpl e682cda008 appengine: fix build issues
split pkg/jsonconfig/sign.go and pkg/server/root.go
because of unsupported stuff on appengine

Change-Id: I6f97ec2e043dabbe54971a88dc401a45ba36a60a
2013-08-11 19:42:57 +02:00
mpl 6c4dc821fa update closure library: automatically generate the list of required files
Also updated parseProvidesRequires to use a bufio.Scanner

http://camlistore.org/issue/149

Change-Id: I13df7f03e3482b77e71687adc2b71b8dd9eeb0db
2013-07-19 19:22:41 +02:00
Brad Fitzpatrick 9468e5ba70 More docs. Every package is documented now.
misc.CountingReader moves into readerutil.

pkg/atomics is folded into pkg/types.

pkg/test/testdep is folded into pkg/test, with better name/docs.

Old cruft from pkg/webserver is deleted.

Change-Id: I3f72d8b29804254ef944995fb085837c878f79f5
2013-07-07 21:12:30 -07:00
mpl 004c261358 closure: GenJSDeps now takes a VFS, delete ui/deps.js
This changes allows to always dynamically serve deps.js, generated
either from the local files, or from the embedded resources.
ui.go and publish.go now make use of it.

http://camlistore.org/issue/151

Change-Id: I129895b2c65a0726c1239bdd7387c62e90649a9d
2013-07-05 19:43:22 +02:00
Brad Fitzpatrick 1c4964c1bc Merge branch 'master' of ssh://camlistore.org:29418/camlistore 2013-06-14 12:56:04 -07:00
Brad Fitzpatrick ca58d8e2e0 Remove noisy var _ = log.Printf lines.
Change-Id: Ia58b8ef5f271f542ae4fe61c7fb1497322770322
2013-06-14 12:55:55 -07:00
mpl ad5f3b854d genjsdeps: now pkg + cmd in misc/closure/
pkg/server/ui.go uses misc/closure too

fixes http://camlistore.org/issue/142

Change-Id: I614e34e6b7d493d66aa61108e3ca940854844c0f
2013-06-14 00:29:45 +02:00
mpl 89e14d8f7d misc/genjsdeps: command to generate closure deps
reusing code from pkg/server/ui.go

Change-Id: I56b8d4b42df2fbcbbe3d654261036cdb907f9f27
2013-06-11 00:00:47 +02:00
mpl ac431439f3 Allow HTTPS with a self-signed certificate.
A new client configuration parameter, "trustedCerts" (list of strings)
is introduced. A certificate fingerprint is defined as the 10 digits
prefix of the sha1 of the whole certificate (in ASN1. DER form).
trustedCerts should contain the list of fingerprints of the
certificates we trust. If not empty, the server's certificate
is checked against that list, instead of using the full x509 validation
on it.

-added a dial function and tls configuration, which check if we're using
SSL, and if we're in "trustedCerts" mode.
pkg/client/client.go
pkg/client/config.go

-moved android specific hacks from camput to the client layer, so that
the dial and tls config could reuse/access them. Also allows future
reuse for other commands, such as camget.
pkg/client/android.go

-adapted camput to the above changes:
cmd/camput/android.go
cmd/camput/camput.go
cmd/camput/files.go

-server prints a hint when it generates the self-signed:
pkg/misc/misc.go
server/camlistored/camlistored.go

-camliactivity:
clients/android/res/xml/preferences.xml
clients/android/src/org/camlistore/Preferences.java
clients/android/src/org/camlistore/SettingsActivity.java
clients/android/src/org/camlistore/UploadService.java
clients/android/src/org/camlistore/UploadThread.java

http://camlistore.org/issue/131

Change-Id: I6be20161549a69aafc8eb7b9e96e9351dc1c5b09
2013-04-23 17:10:37 +02:00
Brad Fitzpatrick 5f6130b13b gethandler: use http.ServeContent; delete pre-Go 1 HTTP Range stuff
Change-Id: Icef01ea04aa7d38fc39c829618ff2856449eb326
2013-01-20 14:06:55 -08:00
Brad Fitzpatrick 37bf028633 s3: verify configured bucket exists at start-up.
Change-Id: I43d0795641fe9dbc345a1f0916b828ede76cf4a6
2013-01-20 11:33:41 -08:00
Brad Fitzpatrick 2fe555aeb9 s3: remove debug logging
Change-Id: Idd4c3538362dd14b59ae6b150992d806ae7ff6b8
2012-12-22 18:45:27 -08:00
Brad Fitzpatrick c83e29cfde s3: improve bucket enumeration
Change-Id: I65ed4a6575250bb2005215df461ada64ac20e41e
2012-12-22 13:50:18 -08:00
Brad Fitzpatrick 21d142e538 s3: some work on Buckets, add TODO
Change-Id: I183a85a0b9639790adec025830dae2f276006290
2012-12-21 16:23:19 -08:00
Brad Fitzpatrick 64bbffc3b5 move fileembed from misc 2012-02-28 12:45:46 +11:00
Brad Fitzpatrick 0714a463c9 Update from r60 to [almost] Go 1.
A lot is still broken, but most stuff at least compiles now.

The directory tree has been rearranged now too.  Go libraries are now
under "pkg".  Fully qualified, they are e.g. "camlistore.org/pkg/jsonsign".

The go tool cannot yet fetch from arbitrary domains, but discussion is
happening now on which mechanism to use to allow that.

For now, put the camlistore root under $GOPATH/src.  Typically $GOPATH
is $HOME, so Camlistore should be at $HOME/src/camlistore.org.

Then you can:

$ go build ./server/camlistored

... etc

The build.pl script is currently disabled.  It'll be resurrected at
some point, but with a very different role (helping create a fake
GOPATH and running the go build command, if things are installed at
the wrong place, and/or running fileembed generators).

Many things are certainly broken.

Many things are disabled.  (MySQL, all indexing, etc).

Many things need to be moved into
camlistore.org/third_party/{code.google.com,github.com} and updated
from their r60 to Go 1 versions, where applicable.

The GoMySQL stuff should be updated to use database/sql and the ziutek
library implementing database/sql/driver.

Help wanted.

Change-Id: If71217dc5c8f0e70dbe46e9504ca5131c6eeacde
2012-02-18 21:53:06 -08:00