Commit Graph

40 Commits

Author SHA1 Message Date
Nick O'Neill 2d70c95aef Tweet importing with some basic info
Change-Id: I73455eef50dcdc495ddc042d266b3a4a8020c877
2014-03-20 18:57:43 -07:00
mpl 05058091e9 kvutil: add VerifyDb*, use kvutil in sorted
http://camlistore.org/issue/393

Change-Id: If9482aa05f2346644196e3218dcde571e4d1a1a1
2014-03-20 01:21:27 +01:00
mpl 4ac7601315 sorted/kv: enable VerifyDb* options with env var
http://camlistore.org/issue/393

Change-Id: I97bbe3d4ab6eb186c3960db0995cedeec3a34ddf
2014-03-20 00:04:18 +01:00
Tamás Gulácsi 65fa5d6d3e Picasa: add importer.
Imports all images from all albums, and puts each image in its album.
Each album is linked under "Picasa (<username>)".
Tries to skip already imported files (check based on album/filename).
See Issue #391.

Change-Id: If00751671a429891c05bbe41f59c1421b2ff2de8
2014-03-17 10:52:51 +01:00
Andrew Gerrand d136d91000 add devcam server -fullindex flag
Change-Id: I5ef7968d122c9a64665ab70a5f9571f6e3147836
2014-02-11 10:37:41 +11:00
Brad Fitzpatrick d1f2ae227f Start of Foursquare importer.
Change-Id: I56b0e71a3aad697b82710ebea6fc941436a3c119
2014-01-14 20:45:23 -08:00
Brad Fitzpatrick 48e67cab16 devcam: add --publish=false and --mini option to start a lite server with no blobs
Change-Id: Idcde77f72126800d610c632940552f04c396634d
2014-01-13 20:11:43 -08:00
Aaron Boodman dc562622a7 Enable -secretring and -keyid for devcam server. Also fix bug in devcam server -wipe.
Change-Id: Ic615935c6ea9aa150a23aee0a18361b498574d76
2013-12-28 14:35:16 -08:00
Aaron Boodman 246b2b9ca8 Allow devcam to use a persistent directory rather than a temp dir.
Change-Id: I234ba75e36dd55f339f679e45ba4c1ac3159f1f0
2013-12-27 15:25:08 -08:00
Bill Thiede bab46f1db5 devcam: option to add args to camlistored exec.
Handy catch-all for camlistored args we haven't plummed through to
devcam server in the proper manner.  Example use:

	$ devcam server -extraargs=-reindex

Change-Id: I4af922e4c95728dc7858fa2db9e60c7efd37a3cb
2013-12-24 21:35:48 -08:00
mpl 6d19d64b88 devcam server: properly wipe mongo
Change-Id: Ib75a460a4be84e21affc662d45b3cf1e321b15bd
2013-12-13 17:37:19 +01:00
mpl fcbbf2a4df index: move mongo implementation to sorted
Also devcam server -wipe wasn't wiping for mongo anymore, now fixed.

Change-Id: Iecc9d8025ddfba8d8ae9417ee170baf02be1d52f
2013-12-13 00:25:40 +01:00
Brad Fitzpatrick bf54591e8d Remove the memIndex option from genconfig and devcam; default memoryIndex to on.
We had the legacy "memIndex" option to put the Camlistore index in
memory (the leveldb memdb) as an option in genconfig called
"memIndex", and we also had an option called "memoryIndex" for whether
to slurp the on-disk index to memory on start-up.  Too confusing!

Instead, delete "memIndex" (since it's not the default anyway, now
that we have kv).

Then, also (the original point of this change): default the
memoryIndex option to true, so search works for people by default.
This option might go away in the future if it becomes the only required
way.

Also, document this.

Change-Id: Iddffa6e19adbf09c5aacd063aa44de362d90633b
2013-12-11 12:20:22 +04:00
Brad Fitzpatrick efd51a08ed devcam: print out temp dir on start-up
Change-Id: I7f84ffdecc5cb63baa931d5b5de8aca564970a67
2013-11-24 17:11:06 -08:00
Aaron Boodman 93b0bbbee5 Flickr importer: store credentials in Camlistore, not weird JSON file.
Change-Id: I07ed4e9043c9fa448425144886dc3b7047c4e85b
2013-11-22 09:50:03 -08:00
Aaron Boodman 2a747ccd56 Add ability to specify flickr config for devcam.
Change-Id: Ic1139730d969558e2d6375dd86c6741e3dfd2b9b
2013-11-18 21:16:09 -08:00
mpl fdaf030bc0 index: add deletedAt method, and cache for deletion status
This method will be used to find out when a permanode or claim
was (un)deleted for the last time.

This change also introduces the keyDeletes index entry which
allows to find the permanodes/claims that a delete claim deletes.

Finally, since some index keys have changed, this CL introduces
a check for an index schema version, to force users to reindex
when needed.

http://camlistore.org/issue/191

Change-Id: I6f8e5d3030a6ddc8b21ff5dc5db311f2ea16f613
2013-11-06 23:34:13 +01:00
mpl 8ca1378869 devcam: reintroduce --sqlite when building server
Change-Id: I2f0fadc380803237057a0dd774070ff79a980a98
2013-10-25 01:00:43 +02:00
Salman Aljammaz a3686d6ec0 devcam: Add mount command, also factor out build() and vebose flags
Change-Id: Ia501cad493d1707c63ebea9640a4d8fa6355da4c
2013-10-23 17:49:09 +01:00
Salman Aljammaz bfd1a65724 devcam: fix up package doc strings
Change-Id: I5f1b85e872a2d66d3c68babd2a2b53b76fb23dc3
2013-10-22 16:36:57 +01:00
mpl 70a167e922 devcam server: do not use os.Getenv
Since 0d854e106c, we do not
use the env directly, but a copy of it, so os.Getenv does
not work to retrieve vars that were set in that copy.

Change-Id: Ibc6ab40ea2ee51e1dea6a5e9ceac782304e0039a
2013-09-23 12:07:04 +02:00
Brad Fitzpatrick 0d854e106c devcam: don't use current environment as scratchpad for future process's environment
Remove calls to os.Setenv.

Change-Id: I9a0c611eb534b278459ddf724b8e7d181f2395ef
2013-09-22 20:13:28 +01:00
mpl 0c1dc0d6e4 devcam test: generate keyblob with devcam put init
The integration tests run by devcam test require a
GPG keyblob in the default location (for camput), and
we do not track in git such a keyblob.
This change allows devcam test to create the keyblob
with devcam put init before running the tests.

Also, devcam server is now using CAMLI_KEYID instead
of hardcoding the key id in the dev server config file.

Change-Id: I030b88c9a9321016fedb62424e0e707621206922
2013-09-20 20:03:21 +02:00
mpl 6210cc8d56 devcam server: sqlite support.
Also makes kv use CAMLI_DBNAME for the index file instead of
CAMLI_KVINDEX_FILE; one less env var to care about.

http://camlistore.org/issue/225

Change-Id: I954cc2e5ff8ed1bb213435af18f1ecc2948c4e59
2013-09-20 14:04:03 +02:00
mpl ba8f88c21a devcam test, replaces make presubmit
http://camlistore.org/issue/223

Change-Id: Iaaea9926a98d7d3335fd6ee52daa225e51ec4703
2013-09-19 17:00:59 +02:00
mpl 50c07cf7be devcam: do not depend on GOPATH, rebuild camget and camput
http://camlistore.org/issue/218

Change-Id: I4f48fac38cad095cd6d4c4d5aea4efeb35fff306
2013-09-10 21:15:12 +02:00
Brad Fitzpatrick 9ae1a64c38 devcam: exec child binaries when possible
Change-Id: I56d56d9f7416e3d5958e736762063a48dcd8e9f9
2013-09-01 11:37:09 -07:00
mpl 3a26c433db devcam: 'put' and 'get' commands, replace dev-camput and dev-camget
Also added -port flag, and untied the port to the
password in devcam server, always 'pass3179' now.

http://camlistore.org/issue/212

Change-Id: Iadf693951763d47907be17c53d4807a4706eb150
2013-09-01 10:25:09 -07:00
Brad Fitzpatrick e20c7f44a8 auth: fix security problem where 'localhost' allowed all access.
Also, add tests. And remove CAMLI_ADVERTISED_PASSWORD, replacing it with explicit devauth mode.

Change-Id: I9cdfe2d537ecefcf75df0b9fb8538258703b1eb7
2013-08-31 13:44:44 -07:00
Brad Fitzpatrick f0d4a61b6a Use osutil.Username in more places.
Change-Id: Id1154fc16c98b3c72d425a4a8adae9c6c0bb2c04
2013-08-28 12:00:17 -07:00
Brad Fitzpatrick ac05ce80ae Merge "devcam: bugfix; do not fail when binary does not exist" 2013-08-26 23:01:22 +00:00
mpl 6bc558910d devcam: bugfix; do not fail when binary does not exist
http://camlistore.org/issue/213

Change-Id: I432a8ec04cb02c4c5241bd38361fad3d6d39ee02
2013-08-26 15:14:52 +02:00
Brad Fitzpatrick 2ddb716b83 Merge "devcam: add -openbrowser flag. Don't open the start page on browser by default." 2013-08-25 17:31:46 +00:00
Burcu Dogan 2de9f8aafd devcam: add -openbrowser flag. Don't open the start page on browser by default.
Change-Id: I1dc3804bf192dc5119c7d05758ccbacd0e7740f1
2013-08-25 18:28:54 +02:00
Brad Fitzpatrick 500ba50b97 devcam: make kvfile the default dev-server indexer.
Memory index requires -memindex now.

Change-Id: I6df424e88c3783e397c528cd8633402a9edf3d95
2013-08-23 17:19:21 -05:00
Bill Thiede ef4719a78c devcam: handle failed camlistored execution.
Issue https://camlistore.org/issue/208

Change-Id: Ie5bf46d13023bc82f7e2934947393708c9308915
2013-08-21 20:22:33 -07:00
mpl a41c89a24e devcam appengine: replaces ./dev-appengine
Change-Id: Ic4adcb8898fc3ead801aefd679682bcc646f3573
2013-08-20 00:10:01 +02:00
mpl 477bf96f33 devcam server: handle SIGQUIT
Somehow forgot last patch when doing
http://camlistore.org/r/452

Change-Id: I10841ebff29689af19773e8f9d99c5c16659cfc1
2013-08-17 00:43:53 +02:00
mpl f4ef8c0375 dev-server: remove. replaced by devcam server.
Fix related docs, config files, buildbot...

Change-Id: I638d3856f42953603b13204fb56782974d0db9ee
2013-08-03 01:12:38 +02:00
mpl 23777f0b05 dev-server: go replacement: devcam server
http://camlistore.org/issue/140

Change-Id: I6fd14b1185450ff853523db0d207df3bf954ffbf
2013-07-26 21:34:42 +02:00