Commit Graph

103 Commits

Author SHA1 Message Date
Bill Thiede bf6a89c9ad images: add support for fast jpeg down-sampling.
The new package pkg/images/fastjpeg uses djpeg(1) to quickly
down-sample images at load time as described here:

  http://jpegclub.org/djpeg/

Add benchmark comparing std lib jpeg.Decode and fastjpeg.DecodeDownsample
with a factor of 1, 2, 4, and 8.
Benchmark resizes a 4000x4000 pixel JPEG to 128x128.

Refactor in pkg/images to make images.Decode a little more readable.

Change-Id: I571db1f3c3068f99da4a01ec84fd246ef098a18c
2014-07-03 21:06:06 -07:00
mpl 6f5133a728 server/app: first look for app executable in CAMLI_APP_BINDIR
Change-Id: I2d8e525c33da5813d67fbd74ca9af8059864a205
2014-06-16 00:02:09 +02:00
mpl a34f9e2669 publisher app
http://camlistore.org/issue/365

Change-Id: I281fdcbbe6a2bdf15607e75a21bc93b453f82c85
2014-06-15 00:03:03 +02:00
mpl 173adebc19 Merge "apps: more generic app handler work, to prepare for publisher" 2014-06-10 21:29:01 +00:00
mpl 9723b5c814 apps: more generic app handler work, to prepare for publisher
http://camlistore.org/issue/365

Change-Id: I3c34ba1f09debc3aaaf68ad9fb11b595470b6b5d
2014-06-14 18:36:10 +02:00
Brian Gitonga Marete 53ab352fa7 pkg/schema, cmd/camget, cmd/camput: Support sockets.
This patch:

1. Introduces a JSON schema for unix domain sockets (socket files).
2. Adds support for the new schema to pkg/schema.
3. Adds support for uploading and restoring sockets to camput and
camget respectively.

Depends on CL 2706 (https://camlistore-review.googlesource.com/2706/)

Change-Id: Ib0d6e9f9ebd348f55efee5a08cd774f3969d0760
2014-05-21 18:51:14 +03:00
Brian Gitonga Marete e9dd1af195 cmd/camget, cmd/camput, pkg/schema: Support FIFOs.
This patch:

1. Introduces a JSON schema for FIFOs (named pipes).
2. Adds support for uploading and restoring FIFOs to camput and camget
respectively.

It includes tests for the basic schema support in pkg/schema and
integration tests for camput and camget.

This CL does not add FIFO support to the HTTP server and fs (cammount
support) code. Those will be addressed in a subsequent patch.

Change-Id: Iab926073c36e653d25e693a559178bea9bf19b9a
2014-05-21 14:52:23 +03:00
mpl 21dda2b4ef app/hello: dummy server application (hello world)
Change-Id: I6690b9459325af5a76d1de679d56701eefdd195e
2014-05-08 22:15:01 +02:00
mpl e34c4bc828 client: replace CAMLI_CLIENT_IDENTITY with CAMLI_KEYID
And remove all mentions of CAMLI_CLIENT_IDENTITY because there isn't any
need for the both of them.

http://camlistore.org/issue/412

Change-Id: I0d6b8cff0f23738231de98171a23550d82f05fe9
2014-03-31 18:56:11 +02:00
Steven L. Speek 3442076272 search: width and height exact matches simplified
corrected some copy paste errors in the expr_test
file

Change-Id: I36467d96c5674cddc34009d36a05c42ddd7e3cc2
2014-03-23 21:18:00 +01:00
Steven L. Speek 0f644c4244 search-ui.txt: 'and', 'or' and parenthesis description
Change-Id: Ie6c077e008084dec0817b2de23832646f4ba99c7
2014-03-23 18:59:57 +01:00
mpl 4ac7601315 sorted/kv: enable VerifyDb* options with env var
http://camlistore.org/issue/393

Change-Id: I97bbe3d4ab6eb186c3960db0995cedeec3a34ddf
2014-03-20 00:04:18 +01:00
Brad Fitzpatrick bf6031a397 search: add operator "childrenof:sha1-xxxxx" to find permanode children of a parent
Change-Id: I9e756147c70f5230aa843ab215dafe5ed6d80087
2014-03-13 19:02:26 -07:00
Steven L. Speek 6d9117f1c0 No 'and' or 'or' yet. Added description for attr and tag.
Change-Id: Ib1ab293dabc264657a970c8ef3ca54b9d59868ce
2014-03-12 22:48:19 +01:00
Brad Fitzpatrick bfaf8fa120 Add camliContentImage.
See camlistore.org/issue/395

Change-Id: I7057593c6733f0e6312681399c3d707dd3aac147
2014-03-10 19:11:29 -07:00
Nick O'Neill a30364a243 Merge "Add doc/search-ui.txt" 2014-03-11 00:47:43 +00:00
Brad Fitzpatrick 344980114a doc: add CAMLI_S3_FAIL_PERCENT var
Change-Id: Ie2c85da2bfeefc9e5ae98c14c5d879d7208a82f1
2014-03-05 08:26:34 -08:00
Tamás Gulácsi 5ec7d94a34 Add doc/search-ui.txt
List the usable operators in doc/search-ui.txt.

Change-Id: I291e248aa8a571c90e5ac2259a57d748963c1f11
2014-03-03 09:26:46 +01:00
Brad Fitzpatrick 11cd3127c8 Backwards-compatible protocol + handler change on boundary of enumerating blobs.
The handler was adding 1 to things to see if it was at the end, but that was causing
HTTP requests to Amazon like: limit 1000, limit 1, limit 1000, limit 1, etc.

This makes it twice as fast.

Change-Id: Ibb7e3f6ae7229a21c87817c7438324d36e7b491a
2014-03-01 13:12:05 -08:00
Brad Fitzpatrick 09266d5960 osutil: add CAMLI_VAR_DIR for CamliVarDir
Change-Id: Idcb0a32ccc49cf24e61bdb81a4fdb1a0f049595c
2014-02-23 10:18:15 -08:00
Brad Fitzpatrick 1894d4ef47 doc: minor json-signing fixes
From c6afc5ebe6 and related

Change-Id: I8c9308563c03bde19db4567ebe9c5f1b381c788a
2014-02-09 10:02:30 -08:00
Brad Fitzpatrick 9abba638a0 Add new environment variables for setting client identity for use with test.World.
Testing clients should now use CAMLI_CLIENT_IDENTITY,
CAMLI_DISABLE_CLIENT_CONFIG_FILE (both new) and existing
CAMLI_SECRET_RING, set using the new test.World accessors.

Change-Id: I9fc0571b008f033674466bbc98129ae1919a0640
2014-01-20 13:54:42 -08:00
Brad Fitzpatrick a384ff188d Get rid of CAMLI_DEV_KEYBLOBS / keyblobs / etc.
Fixes camlistore.org/issue/277

Change-Id: I7f380f9d18785f600fa0c442d5a19cd118782788
2014-01-20 13:47:08 -08:00
Brad Fitzpatrick d7e0ad7c5b doc: add two undocumented attributes
Change-Id: I3676c94fdd44a1479bc2c57a7a2015f6df14f835
2014-01-16 20:16:41 -08:00
Brad Fitzpatrick 0841e93514 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2014-01-11 16:22:02 -08:00
Brad Fitzpatrick b7230b2897 client: add CAMLI_DEFAULT_SERVER env option for selecting default server
For demos, when we want to show using camput/camget but select the devcam server
instead of our production servers.

Change-Id: Ie2adfd4f80df3119a705b51bb9736c3b66612d1c
2014-01-11 10:41:36 -08:00
Aaron Boodman d4fbfe7180 Implement generalized navigation handling, use to implement escape handling in detail view.
Change-Id: Iba5359e9d17fcaa200a9cbb586ac9d6461c179aa
2014-01-10 15:50:05 -08:00
Brad Fitzpatrick 91f8b2ad49 Change the upload protocol.
No more dynamic upload URL, which trips up half our new users behind
reverse proxies when the camlistored process doesn't know its
forward-facing URL.

The original camlistore stat + upload protocol was influenced by App
Engine's limitations at the time, and some of our indecision about
where the Camlistore design is going. We understand the Camlistore
design now, and App Engine's former limitations are gone. Time to
clean things up.

More REST-y now too.

See http://camlistore.org/issue/123

Change-Id: I92c6552f830b925cef379c204a982a2213bf2f4b
2014-01-04 20:24:58 -08:00
Dustin Sallings c13717e788 fs: extended attribute support
This includes read-write support for extended attributes for mutable
files, and enough support on all other filesystem paths to prevent OS X
from falling back to AppleDouble files.

Change-Id: I2b275591f3bc69c25e1e9c8d59a7fa4cf0bb97ff
2014-01-02 12:02:10 -08:00
Brad Fitzpatrick 3bdff69dcf doc: add camliDefVis to schema claims docs
Change-Id: I76d66887fb599e80472da489d7dfeb8b629b9b4a
2014-01-02 00:34:12 -08:00
Brad Fitzpatrick 80c0a45ecc Update doc/schema/claims/attributes.txt
Change-Id: I6f7bcce136b38ad33f991e20289e67e2d184353e
2013-12-28 11:20:51 -08:00
Mathieu Lonjaret ac7b19de20 Merge "camput: allow ignoring files NOT only if --filenodes used" 2013-12-17 15:11:07 +00:00
Brad Fitzpatrick 0d36539f81 Add CAMLI_DISABLE_THUMB_CACHE option for when working on thumbnailer
Change-Id: Ifa99a57c82c67a83fe0b89abaeee407058b00af8
2013-12-16 18:11:52 -08:00
jakubbrzeski 33e925ffa1 camput: allow ignoring files NOT only if --filenodes used
Change-Id: I7856c5f6602e6b439ebcebbcb83eddb1d1865e88
2013-12-16 16:52:59 +01:00
Brad Fitzpatrick 5eeef19472 Make CAMLI_FAST_DEV work again.
Change-Id: I294de23a50d48c7c2325780b50141a7f02d55aab
2013-12-12 16:18:24 +04:00
Brad Fitzpatrick 5f1e7de0d1 serverconfig: also write mem profile with CAMLI_PPROF_START
Change-Id: Iffe78c66916d6e690a28edfd258c51a5433edff5
2013-12-07 18:34:13 -08:00
Brad Fitzpatrick 6e900be6d0 serverconfig: add CAMLI_PPROF_START
Change-Id: Ia3d3e021710f7681de7d43de87c236f8e37e992c
2013-11-30 10:01:32 -08:00
mpl 1ee5fd20c5 search: deletions are not modifications
1) pkg/search: documented that deletions times do not
qualify as modtimes

2) pkg/index: got rid of DeletedAt, and keyDeletes

http://camlistore.org/issue/191

Change-Id: I39578913345454d36af4599e29e7053f46577846
2013-11-29 00:29:57 +01:00
Brad Fitzpatrick 26f9b630f8 Add func test.BrokenTest and RUN_BROKEN_TESTS to doc/environment-vars.txt
Change-Id: I43dc7b10ac6175ab8ab5eae4cfdd98bc08515ff5
2013-11-24 15:00:16 -08:00
mpl f8b41e236e Merge "server/publish.go: use go html template for publishing" 2013-11-19 22:22:17 +00:00
mpl a6aeff034c server/publish.go: use go html template for publishing
Change-Id: Ic1c55d35e52e407864d339927b453de58a64e03c
2013-11-19 17:50:26 +01:00
mpl d4b5e413a2 schema: add delete claim
Also rename other claim types for consistent style.

http://camlistore.org/issue/191

Change-Id: I1dc1283bd961481ad793b92262cf5ce641d9fdc8
2013-11-03 00:37:49 +01:00
Bill Thiede b576dcc481 serverconfig: conditionally install expvar handler.
Add local copy of the standard library's expvar handler.

Change-Id: Ida41553345d7d29665f33268356bf16a80386e38
2013-10-04 21:24:24 -07:00
mpl 0c1dc0d6e4 devcam test: generate keyblob with devcam put init
The integration tests run by devcam test require a
GPG keyblob in the default location (for camput), and
we do not track in git such a keyblob.
This change allows devcam test to create the keyblob
with devcam put init before running the tests.

Also, devcam server is now using CAMLI_KEYID instead
of hardcoding the key id in the dev server config file.

Change-Id: I030b88c9a9321016fedb62424e0e707621206922
2013-09-20 20:03:21 +02:00
mpl c556debcc5 osutil: take into account env var for default secret ring
http://camlistore.org/issue/227

Change-Id: I40eac298d7d5d74760be65d9e13d531b562830b9
2013-09-20 14:22:12 +02:00
Bill Thiede 630f6ca61f Document environment variables usage.
Running 'go run dev/envvardoc/envvardoc.go' now shows:
'All environment variables are documented'

I also took the liberty of cleaning-up our mishmash of logic for handling
boolean environment variables, and cleaned up a couple other spots that didn't
seem right.

This change adds docmentation for all variables starting with (CAM|DEV|AWS).
This leaves some variables still undocumented.  If there are variables worth
documenting in the following list, maybe we should rename them to have a
CAM{LI} prefix for consistency's sake:

APPDATA                  pkg/osutil/paths.go:86
APPDATA                  pkg/osutil/paths.go:102
DISPLAY                  pkg/misc/gpgagent/gpgagent.go:126
GOPATH                   pkg/fileembed/genfileembed/genfileembed.go:321
GOPATH                   pkg/osutil/paths.go:168
GOPATH                   pkg/test/world.go:54
GOPATH                   server/appengine/build_test.go:77
GPGKEY                   cmd/camput/init.go:77
GPG_AGENT_INFO           cmd/camput/init.go:153
GPG_AGENT_INFO           pkg/misc/gpgagent/gpgagent.go:50
HOME                     pkg/jsonsign/keys.go:79
HOME                     pkg/jsonsign/signhandler/sig.go:64
HOME                     pkg/osutil/paths.go:36
HOMEPATH                 pkg/osutil/paths.go:34
PKG_CONFIG_PATH          pkg/index/sqlite/dbschema.go:59
RUN_BROKEN_TESTS         pkg/fs/fs_test.go:67
SKIP_DEP_TESTS           pkg/test/testdep.go:29
TERM                     pkg/misc/gpgagent/gpgagent.go:133
TERM                     pkg/misc/pinentry/pinentry.go:99
TESTING_PORT_WRITE_FD    pkg/webserver/webserver.go:135
TEST_GPGAGENT_LIB        pkg/misc/gpgagent/gpgagent_test.go:27
USER                     pkg/netutil/ident.go:135
USER                     pkg/osutil/paths.go:45
USERNAME                 pkg/jsonconfig/eval.go:228
USERNAME                 pkg/osutil/paths.go:43
VERBOSE_FUSE             pkg/fs/fs_test.go:133
VERBOSE_FUSE_STDERR      pkg/fs/fs_test.go:137
XDG_CONFIG_HOME          pkg/osutil/paths.go:104

Change-Id: Ief28710d3deefd1e65247cb5d3b1d8dde73e1f2d
2013-09-06 21:54:03 -07:00
Brad Fitzpatrick e20c7f44a8 auth: fix security problem where 'localhost' allowed all access.
Also, add tests. And remove CAMLI_ADVERTISED_PASSWORD, replacing it with explicit devauth mode.

Change-Id: I9cdfe2d537ecefcf75df0b9fb8538258703b1eb7
2013-08-31 13:44:44 -07:00
Bill Thiede ac4ed5492b envvardoc: new tool to find undocumented env vars.
This tool will walk over interesting directories (-srcDirs flag) in the
source tree and look for references to environement variables. It then
cross-references these with environment variables found in a documentaion file
(-doc flag).  Any values found in source but not documented are printed to the
screen, and the exit code is set to non-zero.  Not all environment variables
are considered, only those matching one of the prefixes (-prefixes flag).

The change also begins the process of documenting environment variables.

Change-Id: Ie122b64b7befcb3104b960cf3d2d22bae443f67f
2013-08-25 21:28:23 -07:00
Brad Fitzpatrick 79f7e014ca Merge "start of permanode attributes doc" 2013-06-15 21:05:04 +00:00
Brad Fitzpatrick fc2597e01f Start of an overview page, describing the project, plus more misc doc/website cleanup
Change-Id: Ic456e73ab3adf549e7280c8ee1c9d6668719720e
2013-06-12 07:22:29 -07:00