Commit Graph

30 Commits

Author SHA1 Message Date
mpl afb47c7cd9 devcam server: add syndtr/goleveldb indexer. make it the default.
Change-Id: Ice835d7505404ad9eeaf040d465409bb0e4c420d
2015-07-26 17:53:36 +02:00
Brad Fitzpatrick f947f8fc19 ui: use WholeRefFetcher for downloads and thumbnails, if available
This makes downloads and the thumbnailer's reading of full images use
blobpacked's WholeRefFetcher fast path, when available. The end result
is many fewer blob fetches.

Updates #532

Change-Id: I3bfd1cb0f04f68f81b80a63378c7b365bd73c974
2015-04-13 20:38:41 +02:00
mpl fd5e8c4048 deploy/gce: query the regions, use them in form, pick random zone
Change-Id: Ie7170c7c12aeb4e32474510d9931f662795bcd41
2015-03-02 19:25:08 +01:00
mpl e3f16b8ead pkg/deploy/gce: add web wizard to deploy on Google Cloud
Issue #531

Change-Id: I47900eb7b82b7775ec1d26b1d6796e5701ee672d
2015-01-19 16:24:37 +01:00
Brad Fitzpatrick 9bea14733d importer: support for disabling importers at start-up
Change-Id: Ia2fc81f496ed020669a1c6ede489197d4a52ae89
2015-01-18 19:02:02 -08:00
Brad Fitzpatrick 488f9721b4 make.go: block darwin/386 builds by default
Fixes Issue 155

Change-Id: I91d471c7557c2225f5feb6b33cc6bb7cc381a68f
2014-10-21 15:02:30 +02:00
mpl bb49185463 serverinit: add support for memory-only indexer. Enable in devcam.
To avoid confusion, the previously named "memoryIndex" configuration
parameter, which actually meant to slurp the index into the memory
corpus, has been renamed to "copyIndexToMemory". "memoryIndex" is now
the parameter to enable a memory-only indexer.

http://camlistore.org/issue/416

Change-Id: I7026e5463fda3a1887a88e0b07d4e3431c60c8b7
2014-08-26 22:46:13 +02:00
mpl d953dee008 devcam test: do not "recurse" temp GOPATH, docs, couple more options.
Problem: make.go creates an isolated temp gopath ./tmp/build-gopath. The
integration tests make use of that gopath (by running make.go) to build
the tools, and run the test world in it. Similarly, devcam test uses
make.go to setup that temp gopath, and runs the tests from the source
files in that gopath. Consequently, when the integration tests are run
through devcam test, even though they're run from the temp gopath, they
would use the make.go in it, which would create a nested temp gopath
(CAMLIROOT/tmp/build-gopath/src/camlistore.org/tmp/build-gopath) in
which to run the tests.

This patch addresses this issue by creating a new flag (-envGoPath), and
the corresponding env var (CAMLI_MAKE_USEGOPATH), which tells make.go
not to create a new temporary gopath (and hence not to mirror any
files), and to rely on the already set GOPATH env var instead.

Also refactored make.go a bit, and added a couple options and doc to
devcam test.

Change-Id: Ia8a5d7a31e6e317f05218d9e18fb886001cd19cb
2014-08-08 22:02:57 +02:00
Bill Thiede bf6a89c9ad images: add support for fast jpeg down-sampling.
The new package pkg/images/fastjpeg uses djpeg(1) to quickly
down-sample images at load time as described here:

  http://jpegclub.org/djpeg/

Add benchmark comparing std lib jpeg.Decode and fastjpeg.DecodeDownsample
with a factor of 1, 2, 4, and 8.
Benchmark resizes a 4000x4000 pixel JPEG to 128x128.

Refactor in pkg/images to make images.Decode a little more readable.

Change-Id: I571db1f3c3068f99da4a01ec84fd246ef098a18c
2014-07-03 21:06:06 -07:00
mpl 6f5133a728 server/app: first look for app executable in CAMLI_APP_BINDIR
Change-Id: I2d8e525c33da5813d67fbd74ca9af8059864a205
2014-06-16 00:02:09 +02:00
mpl e34c4bc828 client: replace CAMLI_CLIENT_IDENTITY with CAMLI_KEYID
And remove all mentions of CAMLI_CLIENT_IDENTITY because there isn't any
need for the both of them.

http://camlistore.org/issue/412

Change-Id: I0d6b8cff0f23738231de98171a23550d82f05fe9
2014-03-31 18:56:11 +02:00
mpl 4ac7601315 sorted/kv: enable VerifyDb* options with env var
http://camlistore.org/issue/393

Change-Id: I97bbe3d4ab6eb186c3960db0995cedeec3a34ddf
2014-03-20 00:04:18 +01:00
Brad Fitzpatrick 344980114a doc: add CAMLI_S3_FAIL_PERCENT var
Change-Id: Ie2c85da2bfeefc9e5ae98c14c5d879d7208a82f1
2014-03-05 08:26:34 -08:00
Brad Fitzpatrick 09266d5960 osutil: add CAMLI_VAR_DIR for CamliVarDir
Change-Id: Idcb0a32ccc49cf24e61bdb81a4fdb1a0f049595c
2014-02-23 10:18:15 -08:00
Brad Fitzpatrick 9abba638a0 Add new environment variables for setting client identity for use with test.World.
Testing clients should now use CAMLI_CLIENT_IDENTITY,
CAMLI_DISABLE_CLIENT_CONFIG_FILE (both new) and existing
CAMLI_SECRET_RING, set using the new test.World accessors.

Change-Id: I9fc0571b008f033674466bbc98129ae1919a0640
2014-01-20 13:54:42 -08:00
Brad Fitzpatrick a384ff188d Get rid of CAMLI_DEV_KEYBLOBS / keyblobs / etc.
Fixes camlistore.org/issue/277

Change-Id: I7f380f9d18785f600fa0c442d5a19cd118782788
2014-01-20 13:47:08 -08:00
Brad Fitzpatrick b7230b2897 client: add CAMLI_DEFAULT_SERVER env option for selecting default server
For demos, when we want to show using camput/camget but select the devcam server
instead of our production servers.

Change-Id: Ie2adfd4f80df3119a705b51bb9736c3b66612d1c
2014-01-11 10:41:36 -08:00
Mathieu Lonjaret ac7b19de20 Merge "camput: allow ignoring files NOT only if --filenodes used" 2013-12-17 15:11:07 +00:00
Brad Fitzpatrick 0d36539f81 Add CAMLI_DISABLE_THUMB_CACHE option for when working on thumbnailer
Change-Id: Ifa99a57c82c67a83fe0b89abaeee407058b00af8
2013-12-16 18:11:52 -08:00
jakubbrzeski 33e925ffa1 camput: allow ignoring files NOT only if --filenodes used
Change-Id: I7856c5f6602e6b439ebcebbcb83eddb1d1865e88
2013-12-16 16:52:59 +01:00
Brad Fitzpatrick 5eeef19472 Make CAMLI_FAST_DEV work again.
Change-Id: I294de23a50d48c7c2325780b50141a7f02d55aab
2013-12-12 16:18:24 +04:00
Brad Fitzpatrick 5f1e7de0d1 serverconfig: also write mem profile with CAMLI_PPROF_START
Change-Id: Iffe78c66916d6e690a28edfd258c51a5433edff5
2013-12-07 18:34:13 -08:00
Brad Fitzpatrick 6e900be6d0 serverconfig: add CAMLI_PPROF_START
Change-Id: Ia3d3e021710f7681de7d43de87c236f8e37e992c
2013-11-30 10:01:32 -08:00
Brad Fitzpatrick 26f9b630f8 Add func test.BrokenTest and RUN_BROKEN_TESTS to doc/environment-vars.txt
Change-Id: I43dc7b10ac6175ab8ab5eae4cfdd98bc08515ff5
2013-11-24 15:00:16 -08:00
Bill Thiede b576dcc481 serverconfig: conditionally install expvar handler.
Add local copy of the standard library's expvar handler.

Change-Id: Ida41553345d7d29665f33268356bf16a80386e38
2013-10-04 21:24:24 -07:00
mpl 0c1dc0d6e4 devcam test: generate keyblob with devcam put init
The integration tests run by devcam test require a
GPG keyblob in the default location (for camput), and
we do not track in git such a keyblob.
This change allows devcam test to create the keyblob
with devcam put init before running the tests.

Also, devcam server is now using CAMLI_KEYID instead
of hardcoding the key id in the dev server config file.

Change-Id: I030b88c9a9321016fedb62424e0e707621206922
2013-09-20 20:03:21 +02:00
mpl c556debcc5 osutil: take into account env var for default secret ring
http://camlistore.org/issue/227

Change-Id: I40eac298d7d5d74760be65d9e13d531b562830b9
2013-09-20 14:22:12 +02:00
Bill Thiede 630f6ca61f Document environment variables usage.
Running 'go run dev/envvardoc/envvardoc.go' now shows:
'All environment variables are documented'

I also took the liberty of cleaning-up our mishmash of logic for handling
boolean environment variables, and cleaned up a couple other spots that didn't
seem right.

This change adds docmentation for all variables starting with (CAM|DEV|AWS).
This leaves some variables still undocumented.  If there are variables worth
documenting in the following list, maybe we should rename them to have a
CAM{LI} prefix for consistency's sake:

APPDATA                  pkg/osutil/paths.go:86
APPDATA                  pkg/osutil/paths.go:102
DISPLAY                  pkg/misc/gpgagent/gpgagent.go:126
GOPATH                   pkg/fileembed/genfileembed/genfileembed.go:321
GOPATH                   pkg/osutil/paths.go:168
GOPATH                   pkg/test/world.go:54
GOPATH                   server/appengine/build_test.go:77
GPGKEY                   cmd/camput/init.go:77
GPG_AGENT_INFO           cmd/camput/init.go:153
GPG_AGENT_INFO           pkg/misc/gpgagent/gpgagent.go:50
HOME                     pkg/jsonsign/keys.go:79
HOME                     pkg/jsonsign/signhandler/sig.go:64
HOME                     pkg/osutil/paths.go:36
HOMEPATH                 pkg/osutil/paths.go:34
PKG_CONFIG_PATH          pkg/index/sqlite/dbschema.go:59
RUN_BROKEN_TESTS         pkg/fs/fs_test.go:67
SKIP_DEP_TESTS           pkg/test/testdep.go:29
TERM                     pkg/misc/gpgagent/gpgagent.go:133
TERM                     pkg/misc/pinentry/pinentry.go:99
TESTING_PORT_WRITE_FD    pkg/webserver/webserver.go:135
TEST_GPGAGENT_LIB        pkg/misc/gpgagent/gpgagent_test.go:27
USER                     pkg/netutil/ident.go:135
USER                     pkg/osutil/paths.go:45
USERNAME                 pkg/jsonconfig/eval.go:228
USERNAME                 pkg/osutil/paths.go:43
VERBOSE_FUSE             pkg/fs/fs_test.go:133
VERBOSE_FUSE_STDERR      pkg/fs/fs_test.go:137
XDG_CONFIG_HOME          pkg/osutil/paths.go:104

Change-Id: Ief28710d3deefd1e65247cb5d3b1d8dde73e1f2d
2013-09-06 21:54:03 -07:00
Brad Fitzpatrick e20c7f44a8 auth: fix security problem where 'localhost' allowed all access.
Also, add tests. And remove CAMLI_ADVERTISED_PASSWORD, replacing it with explicit devauth mode.

Change-Id: I9cdfe2d537ecefcf75df0b9fb8538258703b1eb7
2013-08-31 13:44:44 -07:00
Bill Thiede ac4ed5492b envvardoc: new tool to find undocumented env vars.
This tool will walk over interesting directories (-srcDirs flag) in the
source tree and look for references to environement variables. It then
cross-references these with environment variables found in a documentaion file
(-doc flag).  Any values found in source but not documented are printed to the
screen, and the exit code is set to non-zero.  Not all environment variables
are considered, only those matching one of the prefixes (-prefixes flag).

The change also begins the process of documenting environment variables.

Change-Id: Ie122b64b7befcb3104b960cf3d2d22bae443f67f
2013-08-25 21:28:23 -07:00