Commit Graph

55 Commits

Author SHA1 Message Date
mpl 6d19d64b88 devcam server: properly wipe mongo
Change-Id: Ib75a460a4be84e21affc662d45b3cf1e321b15bd
2013-12-13 17:37:19 +01:00
Brad Fitzpatrick 9f8e9e57fd Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-12-13 11:15:29 +04:00
mpl fcbbf2a4df index: move mongo implementation to sorted
Also devcam server -wipe wasn't wiping for mongo anymore, now fixed.

Change-Id: Iecc9d8025ddfba8d8ae9417ee170baf02be1d52f
2013-12-13 00:25:40 +01:00
Brad Fitzpatrick 5eeef19472 Make CAMLI_FAST_DEV work again.
Change-Id: I294de23a50d48c7c2325780b50141a7f02d55aab
2013-12-12 16:18:24 +04:00
Brad Fitzpatrick 2d2a0c1479 devcam: add -race flag
Change-Id: I1d477431b917faba3d2ee99ef5241dda14e748f3
2013-12-11 22:13:06 +04:00
Brad Fitzpatrick bf54591e8d Remove the memIndex option from genconfig and devcam; default memoryIndex to on.
We had the legacy "memIndex" option to put the Camlistore index in
memory (the leveldb memdb) as an option in genconfig called
"memIndex", and we also had an option called "memoryIndex" for whether
to slurp the on-disk index to memory on start-up.  Too confusing!

Instead, delete "memIndex" (since it's not the default anyway, now
that we have kv).

Then, also (the original point of this change): default the
memoryIndex option to true, so search works for people by default.
This option might go away in the future if it becomes the only required
way.

Also, document this.

Change-Id: Iddffa6e19adbf09c5aacd063aa44de362d90633b
2013-12-11 12:20:22 +04:00
Brad Fitzpatrick efd51a08ed devcam: print out temp dir on start-up
Change-Id: I7f84ffdecc5cb63baa931d5b5de8aca564970a67
2013-11-24 17:11:06 -08:00
Aaron Boodman 93b0bbbee5 Flickr importer: store credentials in Camlistore, not weird JSON file.
Change-Id: I07ed4e9043c9fa448425144886dc3b7047c4e85b
2013-11-22 09:50:03 -08:00
Aaron Boodman 2a747ccd56 Add ability to specify flickr config for devcam.
Change-Id: Ic1139730d969558e2d6375dd86c6741e3dfd2b9b
2013-11-18 21:16:09 -08:00
mpl fdaf030bc0 index: add deletedAt method, and cache for deletion status
This method will be used to find out when a permanode or claim
was (un)deleted for the last time.

This change also introduces the keyDeletes index entry which
allows to find the permanodes/claims that a delete claim deletes.

Finally, since some index keys have changed, this CL introduces
a check for an index schema version, to force users to reindex
when needed.

http://camlistore.org/issue/191

Change-Id: I6f8e5d3030a6ddc8b21ff5dc5db311f2ea16f613
2013-11-06 23:34:13 +01:00
mpl 8ca1378869 devcam: reintroduce --sqlite when building server
Change-Id: I2f0fadc380803237057a0dd774070ff79a980a98
2013-10-25 01:00:43 +02:00
Salman Aljammaz a3686d6ec0 devcam: Add mount command, also factor out build() and vebose flags
Change-Id: Ia501cad493d1707c63ebea9640a4d8fa6355da4c
2013-10-23 17:49:09 +01:00
Brad Fitzpatrick 18de812d8b Merge "devcam: fix up package doc strings" 2013-10-22 15:38:46 +00:00
Salman Aljammaz bfd1a65724 devcam: fix up package doc strings
Change-Id: I5f1b85e872a2d66d3c68babd2a2b53b76fb23dc3
2013-10-22 16:36:57 +01:00
Salman Aljammaz b0a8dee500 Add camtool to devcam.
Change-Id: Ife10222ca8f1ba08be923f9433781910376ddf05
2013-10-22 16:24:28 +01:00
mpl b4a5a36457 devcam put/get: read CAMLI_QUIET from current env
reverts 0015237713

Change-Id: Ie8ec46c1971a014cafe05d160d2f65a84d8cd466
2013-09-23 23:21:48 +02:00
mpl 0015237713 devcam put/get: do not use os.Getenv
Change-Id: I2e5b247835ec3febe7f312cabcfbf73f58bf43dd
2013-09-23 12:21:39 +02:00
mpl 70a167e922 devcam server: do not use os.Getenv
Since 0d854e106c, we do not
use the env directly, but a copy of it, so os.Getenv does
not work to retrieve vars that were set in that copy.

Change-Id: Ibc6ab40ea2ee51e1dea6a5e9ceac782304e0039a
2013-09-23 12:07:04 +02:00
Brad Fitzpatrick 0d854e106c devcam: don't use current environment as scratchpad for future process's environment
Remove calls to os.Setenv.

Change-Id: I9a0c611eb534b278459ddf724b8e7d181f2395ef
2013-09-22 20:13:28 +01:00
Brad Fitzpatrick 894a51a5b1 Remove dev-pass-camput from make-release.
Change-Id: I503895e1555b6c7eefef1e0af2baefceb623557d
2013-09-21 19:56:44 +01:00
mpl 0c1dc0d6e4 devcam test: generate keyblob with devcam put init
The integration tests run by devcam test require a
GPG keyblob in the default location (for camput), and
we do not track in git such a keyblob.
This change allows devcam test to create the keyblob
with devcam put init before running the tests.

Also, devcam server is now using CAMLI_KEYID instead
of hardcoding the key id in the dev server config file.

Change-Id: I030b88c9a9321016fedb62424e0e707621206922
2013-09-20 20:03:21 +02:00
mpl 6210cc8d56 devcam server: sqlite support.
Also makes kv use CAMLI_DBNAME for the index file instead of
CAMLI_KVINDEX_FILE; one less env var to care about.

http://camlistore.org/issue/225

Change-Id: I954cc2e5ff8ed1bb213435af18f1ecc2948c4e59
2013-09-20 14:04:03 +02:00
mpl ba8f88c21a devcam test, replaces make presubmit
http://camlistore.org/issue/223

Change-Id: Iaaea9926a98d7d3335fd6ee52daa225e51ec4703
2013-09-19 17:00:59 +02:00
Brad Fitzpatrick 1decd09846 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-09-11 07:46:58 +01:00
Brad Fitzpatrick 6fca50a9c7 devcam: add -nobuild flag to put.
But really this should be a global option, and/or environment controlled,
like dev/demo.sh set up previously.

Change-Id: Iabf0c02e4a44ed574088e2938f3b4f53cabf3a43
2013-09-10 20:02:15 -07:00
mpl 50c07cf7be devcam: do not depend on GOPATH, rebuild camget and camput
http://camlistore.org/issue/218

Change-Id: I4f48fac38cad095cd6d4c4d5aea4efeb35fff306
2013-09-10 21:15:12 +02:00
Brad Fitzpatrick ff4e491b3b devcam: fix breakage caused from envvar strconv changes.
Change-Id: Id05d00d9b82924242905ccbc7a7e8002dc4859a7
2013-09-08 12:32:13 -07:00
Bill Thiede 630f6ca61f Document environment variables usage.
Running 'go run dev/envvardoc/envvardoc.go' now shows:
'All environment variables are documented'

I also took the liberty of cleaning-up our mishmash of logic for handling
boolean environment variables, and cleaned up a couple other spots that didn't
seem right.

This change adds docmentation for all variables starting with (CAM|DEV|AWS).
This leaves some variables still undocumented.  If there are variables worth
documenting in the following list, maybe we should rename them to have a
CAM{LI} prefix for consistency's sake:

APPDATA                  pkg/osutil/paths.go:86
APPDATA                  pkg/osutil/paths.go:102
DISPLAY                  pkg/misc/gpgagent/gpgagent.go:126
GOPATH                   pkg/fileembed/genfileembed/genfileembed.go:321
GOPATH                   pkg/osutil/paths.go:168
GOPATH                   pkg/test/world.go:54
GOPATH                   server/appengine/build_test.go:77
GPGKEY                   cmd/camput/init.go:77
GPG_AGENT_INFO           cmd/camput/init.go:153
GPG_AGENT_INFO           pkg/misc/gpgagent/gpgagent.go:50
HOME                     pkg/jsonsign/keys.go:79
HOME                     pkg/jsonsign/signhandler/sig.go:64
HOME                     pkg/osutil/paths.go:36
HOMEPATH                 pkg/osutil/paths.go:34
PKG_CONFIG_PATH          pkg/index/sqlite/dbschema.go:59
RUN_BROKEN_TESTS         pkg/fs/fs_test.go:67
SKIP_DEP_TESTS           pkg/test/testdep.go:29
TERM                     pkg/misc/gpgagent/gpgagent.go:133
TERM                     pkg/misc/pinentry/pinentry.go:99
TESTING_PORT_WRITE_FD    pkg/webserver/webserver.go:135
TEST_GPGAGENT_LIB        pkg/misc/gpgagent/gpgagent_test.go:27
USER                     pkg/netutil/ident.go:135
USER                     pkg/osutil/paths.go:45
USERNAME                 pkg/jsonconfig/eval.go:228
USERNAME                 pkg/osutil/paths.go:43
VERBOSE_FUSE             pkg/fs/fs_test.go:133
VERBOSE_FUSE_STDERR      pkg/fs/fs_test.go:137
XDG_CONFIG_HOME          pkg/osutil/paths.go:104

Change-Id: Ief28710d3deefd1e65247cb5d3b1d8dde73e1f2d
2013-09-06 21:54:03 -07:00
Brad Fitzpatrick 4aeb310a4c devcam: fix build
Change-Id: If5f8d6e0e4e315cd8ca01aa7bf9b0f59407c52fe
2013-09-02 08:56:08 -07:00
Brad Fitzpatrick 9ae1a64c38 devcam: exec child binaries when possible
Change-Id: I56d56d9f7416e3d5958e736762063a48dcd8e9f9
2013-09-01 11:37:09 -07:00
mpl 3a26c433db devcam: 'put' and 'get' commands, replace dev-camput and dev-camget
Also added -port flag, and untied the port to the
password in devcam server, always 'pass3179' now.

http://camlistore.org/issue/212

Change-Id: Iadf693951763d47907be17c53d4807a4706eb150
2013-09-01 10:25:09 -07:00
Brad Fitzpatrick e20c7f44a8 auth: fix security problem where 'localhost' allowed all access.
Also, add tests. And remove CAMLI_ADVERTISED_PASSWORD, replacing it with explicit devauth mode.

Change-Id: I9cdfe2d537ecefcf75df0b9fb8538258703b1eb7
2013-08-31 13:44:44 -07:00
Brad Fitzpatrick 2f2c3fdfef Merge "envvardoc: new tool to find undocumented env vars." 2013-08-30 19:26:48 +00:00
Brad Fitzpatrick f0d4a61b6a Use osutil.Username in more places.
Change-Id: Id1154fc16c98b3c72d425a4a8adae9c6c0bb2c04
2013-08-28 12:00:17 -07:00
Brad Fitzpatrick 7963001a9c dev/make-release: change in-correct-directory heuristic
Change-Id: If0ee4fb0aea262e1f283e16e85a489d4f806f51b
2013-08-26 18:05:12 -05:00
Brad Fitzpatrick ac05ce80ae Merge "devcam: bugfix; do not fail when binary does not exist" 2013-08-26 23:01:22 +00:00
Johan Euphrosine 4928a880be dev/devcam/appengine: map -all flag to devappserver2 --host
Change-Id: I1cb352041405f5244159f23b37ce2c215a422943
2013-08-26 15:36:34 -07:00
mpl 6bc558910d devcam: bugfix; do not fail when binary does not exist
http://camlistore.org/issue/213

Change-Id: I432a8ec04cb02c4c5241bd38361fad3d6d39ee02
2013-08-26 15:14:52 +02:00
Bill Thiede ac4ed5492b envvardoc: new tool to find undocumented env vars.
This tool will walk over interesting directories (-srcDirs flag) in the
source tree and look for references to environement variables. It then
cross-references these with environment variables found in a documentaion file
(-doc flag).  Any values found in source but not documented are printed to the
screen, and the exit code is set to non-zero.  Not all environment variables
are considered, only those matching one of the prefixes (-prefixes flag).

The change also begins the process of documenting environment variables.

Change-Id: Ie122b64b7befcb3104b960cf3d2d22bae443f67f
2013-08-25 21:28:23 -07:00
Brad Fitzpatrick 2ddb716b83 Merge "devcam: add -openbrowser flag. Don't open the start page on browser by default." 2013-08-25 17:31:46 +00:00
Burcu Dogan 2de9f8aafd devcam: add -openbrowser flag. Don't open the start page on browser by default.
Change-Id: I1dc3804bf192dc5119c7d05758ccbacd0e7740f1
2013-08-25 18:28:54 +02:00
Brad Fitzpatrick 500ba50b97 devcam: make kvfile the default dev-server indexer.
Memory index requires -memindex now.

Change-Id: I6df424e88c3783e397c528cd8633402a9edf3d95
2013-08-23 17:19:21 -05:00
Brad Fitzpatrick c5761f5ed8 Add dev/push script.
Change-Id: I84f12d8df8d65a3d7a5d3287266849f250283502
2013-08-23 00:13:09 -05:00
Bill Thiede ef4719a78c devcam: handle failed camlistored execution.
Issue https://camlistore.org/issue/208

Change-Id: Ie5bf46d13023bc82f7e2934947393708c9308915
2013-08-21 20:22:33 -07:00
mpl a41c89a24e devcam appengine: replaces ./dev-appengine
Change-Id: Ic4adcb8898fc3ead801aefd679682bcc646f3573
2013-08-20 00:10:01 +02:00
mpl 477bf96f33 devcam server: handle SIGQUIT
Somehow forgot last patch when doing
http://camlistore.org/r/452

Change-Id: I10841ebff29689af19773e8f9d99c5c16659cfc1
2013-08-17 00:43:53 +02:00
Brad Fitzpatrick 7bbc2828fd demo.sh script to source during demos
Change-Id: If7415d16f21fdc4ae16778b3bf2ce5c908067193
2013-08-03 13:04:33 -07:00
mpl f4ef8c0375 dev-server: remove. replaced by devcam server.
Fix related docs, config files, buildbot...

Change-Id: I638d3856f42953603b13204fb56782974d0db9ee
2013-08-03 01:12:38 +02:00
Brad Fitzpatrick 737f033cd6 make-release: remove misc/sqlite from delete list. it's gone.
Change-Id: I4fef7dc948d143e0faff63ee176c84ed61cb51d8
2013-07-28 18:46:26 -07:00
mpl 23777f0b05 dev-server: go replacement: devcam server
http://camlistore.org/issue/140

Change-Id: I6fd14b1185450ff853523db0d207df3bf954ffbf
2013-07-26 21:34:42 +02:00