Rooba
/
perkeep
mirror of https://github.com/perkeep/perkeep.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
5,319 Commits 44 Branches 11 Tags 59 MiB
Commit Graph

60 Commits

Author SHA1 Message Date
Brad Fitzpatrick 3026631521 serverinit: move incompatible option checks further down, into the right places 
I was getting bogus errors because I replicate to s3, but the
s3+blobpacked error should only come up if s3 is primary.

Also, document the database key used for blobpack's index when mysql
or another db is in use.

Change-Id: I21a1815715f70edc3f98abf7209744703aaf6d3b
 2015-01-18 19:04:31 -08:00
Brad Fitzpatrick 4155ac6a3c serverinit: high-level config support for using blobpacked 
Users need to set:

     "packRelated": true,

... in their server config. This will probably become the default
value in the future.

This is currently mutually exclusive with diskpacked, which isn't good
at deleting things, and blobpacked loves to delete things (from the
loose blobs).

Updates #532

Change-Id: I8f4ea9406859b2705f26e9d1103d3acf9d1a8411
 2015-01-18 18:08:18 -08:00
mpl cfd784269e deploy/gce: use subdirs for config and blobs (instead of buckets) 
http://camlistore.org/issue/528

Change-Id: Icb558d7de032f9367e8fb100be87ef24c2982c23
 2014-12-17 16:20:58 +01:00
Brad Fitzpatrick 5eeed2aba5 Remove some code.google.com references. 
Change-Id: I85ccf92af9389300919eed61f62aae28cc24a820
 2014-12-11 17:33:17 +11:00
mpl 5c7208fb94 Merge "blobserver/cloudstorage: directory support" 2014-12-10 15:02:00 +00:00
mpl 8481c2236d blobserver/cloudstorage: directory support 
http://camlistore.org/issue/528

Change-Id: I9d26b85af1cf91d1f2b102c7bfb80628af4d1e4d
 2014-11-26 00:40:19 +01:00
Emil Hessman e8ffb9a66c pkg: fix various vet reports 
Change-Id: I0b7fa09849650fe696e59ad5c5c73a034ab80faa
 2014-11-12 23:25:53 +01:00
mpl 9b68fd9f32 serverinit: syndtr/leveldb support 
Change-Id: Id48a00f3896c38659a53cf131493d5a19dbfe4b7
 2014-10-11 15:40:29 +02:00
mpl 8cf7e1399b serverinit: TestInstallHandlers 
http://camlistore.org/issue/416

Change-Id: I14934dd4e0a415c359a13583ddc295be2d35c781
 2014-09-25 14:54:29 +02:00
mpl 95885192eb serverinit: enable memory storage in config 
http://camlistore.org/issue/416

Change-Id: Idde32273ed651a5876581ad0ea06010970b92a9b
 2014-09-11 01:11:07 +02:00
mpl bb49185463 serverinit: add support for memory-only indexer. Enable in devcam. 
To avoid confusion, the previously named "memoryIndex" configuration
parameter, which actually meant to slurp the index into the memory
corpus, has been renamed to "copyIndexToMemory". "memoryIndex" is now
the parameter to enable a memory-only indexer.

http://camlistore.org/issue/416

Change-Id: I7026e5463fda3a1887a88e0b07d4e3431c60c8b7
 2014-08-26 22:46:13 +02:00
Brad Fitzpatrick 5903e89490 On GCE, put the UI's thumbnail metadata cache on MySQL too. 
Change-Id: Ic3f1238deeb6b569b30e6c3f6b0def2b68f13d93
 2014-08-23 19:55:04 -07:00
Brad Fitzpatrick 853de21b18 serverinit: allow dbname 'ui_thumbcache' in high-level config to store thumbnail meta 
Change-Id: I0b87c68064691e286734aace9ba794b0dc435523
 2014-08-23 19:54:32 -07:00
Brad Fitzpatrick a2a738c6ba GCE: keep sync-to-index queue on MySQL 
Change-Id: I660894ebd03fb04beaa852d92073fb2ad46a98c5
 2014-08-23 09:46:56 -07:00
Brad Fitzpatrick 30c209a21d serverinit: support sync queues on database in genconfig 
So now the GCE install with MySQL can keep its sync-to-index queue on the
same MySQL instance.

Change-Id: I75357d8e6e24bf1469bb617eb46803c11d0936d8
 2014-08-23 09:46:23 -07:00
Brad Fitzpatrick 956e838cc0 If on GCE, use MySQL if MYSQLDB_PORT env var is set. (to linked Docker container) 
Change-Id: I4fd4718e6912e7ad8127395cdf6972337a8766bc
 2014-08-18 23:17:17 -07:00
Brad Fitzpatrick e86d90dd42 GCE: expose host's systemd journal to camlistored; add /var/logs handler 
The machine now boots with a new pair of cooperating systemd units:

1) cam-journal-gatewayd.service: a copy of the systemd-journal-gatewayd service,
   which runs an HTTP interface to the systemd journal.
2) cam-journal-gatewayd.socket: a unix socket listener listening on unix
   socket /run/camjournald.sock. Incoming connections will forward to 1).

Then the camlistored.service unit running camlistored under Docker now
passes -v /run/camjournald.sock:/run/camjournald.sock to make that unix socket
available to the Docker container.

Then in camlistored, a new handler at /debug/logs (wrapped in auth
checks) then opens that socket and makes an HTTP request to it,
copying its response (of log lines) back to the browser.

This will ease debugging, letting people only use their browser to
debug (or send logs to the Camlistore developers more likely), rather
than sshing in to CoreOS and learning CoreOS and systemd arcana.

Change-Id: Icd5967ae7e9946d36229bdbc5d37644a11ee5e9f
 2014-08-18 15:20:59 -07:00
Brad Fitzpatrick ed4add5cf6 serverinit: add /debug/config handler to see low-level config 
Useful for debugging GCE issues.

Change-Id: Iafd20de39ed64e66c8ff6cb736ec49ed8ae568c5
 2014-08-16 21:40:12 -07:00
Brad Fitzpatrick d2f6f9e057 serverinit: in the GCE env, set the BaseURL based on the IP address for now. 
Change-Id: I3be51c52ac78e6f6d7cf373cbc358ac1244526c4
 2014-08-15 22:06:13 -07:00
Brad Fitzpatrick 9fbec8dd12 serverinit, camlistored: support environment-specific default config files 
When running on GCE now, the default config file is a function of
attributes on the instance.

Change-Id: I62fe23ecf28757a6baee6bd432f080f223a389c5
 2014-08-15 20:50:59 -07:00
Brad Fitzpatrick 03739edb2a serverinit: break WriteDefaultConfigFile up, pull out getOrMakeKeyring 
Change-Id: I8322f975baba0037138af30d0bb59572f852b3ed
 2014-08-15 20:49:26 -07:00
Brad Fitzpatrick 6e7d3e3e1e serverinit: clean up and simplify the high-to-low-level config generator 
Change-Id: Ia45457718fb8832b4da32d4d6c87c8a5ba9e2301
 2014-08-07 18:47:42 -07:00
Brad Fitzpatrick b65868d754 serverinit: allow specifying port numbers in user@server:port:pass 
With heuristics and tests.

Change-Id: Ifacfc725db15100c447a3251c23267a5bfd313cf
 2014-08-06 15:07:43 -07:00
Brad Fitzpatrick 0757937177 serverinit: allow jsonconfig expansions in high-level configs. 
Change-Id: I600fc56b9324143559ba594ed41a391296871689
 2014-08-06 10:49:27 -07:00
Brad Fitzpatrick 966d25cb62 add wkfs.ReadFile, use it in serverinit 
Change-Id: I2870568e41de9ddb879c51606c77aec0ec77c4ef
 2014-08-05 12:51:17 -07:00
Brad Fitzpatrick c8322af356 Add well-known filesystem abstraction package 'wkfs' for cloud deployment ease. 
Part of Camlistore deploy to GCE easily. Will use paths like /gcs/bucket/foo or /gce-meta/attr for some
things previously-assumed to only be stored on the OS filesystem.

Change-Id: I2ed6cf5759c4d277f4e89a33a0fcfeee8bcbbf82
 2014-08-04 21:27:07 -07:00
Brad Fitzpatrick c3a3883395 googlecloudstorage: whren running on GCE, get access tokens automatically from the VM 
Change-Id: Ibfcca0c52ff01a8f82bb0d3081119e0ac6e16ba9
 2014-08-02 19:29:57 -07:00
Brad Fitzpatrick 1bfd1b7d98 camlistored, serverinit: allow server config from URLs. 
paves the way for server config from GCE metadata, too.

Change-Id: I9e54e3ad79eb53e7ae96159f2aa02f4a017e8e11
 2014-08-02 16:53:58 -07:00
Brad Fitzpatrick 14bb3f48e4 serverinit: remove unused configPath field 
Change-Id: Ide4c2eb07d7c3353563e5f354e833f9429542b65
 2014-08-02 15:10:08 -07:00
Brad Fitzpatrick 1577fc1e56 Merge "netutil: add HostPort, serverinit: return app baseURL" 2014-08-01 18:50:59 +00:00
mpl 044ac54a20 pkg/serverinit: fix tests for windows 
Fixes: http://camlistore.org/issue/304

Change-Id: Ieb29ffefd1daf8d62704374d8e1eecd9cfea3156
 2014-08-01 01:05:41 +02:00
mpl 0a869ad067 netutil: add HostPort, serverinit: return app baseURL 
Context: http://camlistore.org/issue/479

This patch allows camlistored to wait for all the apps to be serving,
before printing its own listening address.

Change-Id: I4035b115a03ef6a2a43177b83b5b65ebc50a2188
 2014-08-01 00:01:27 +02:00
tnt 6868a882b9 Fixed calls to formatting functions 
Change-Id: I3ec2af85364674ac6f1a88cd12b6cb369d8e50e1
 2014-07-21 22:06:53 +00:00
Andrew Gerrand 594328e21c serverconfig: only add importer when index available 
Change-Id: I1142afd0bfa1cf581641b4cd4dac1c9e76e6671a
 2014-07-05 08:25:28 +10:00
mpl b8e3890d1f serverinit: bootstrap publisher camliRoot in dev-mode 
http://camlistore.org/issue/455

Change-Id: If6e1977e5fd63d48accd4a7f99f063f6c990d4e1
 2014-06-19 00:27:54 +02:00
mpl 736648ce9c publisher app: SSL support 
http://camlistore.org/issue/458

Change-Id: I89013a345dbe06fd02c877422cadb8d6def1580d
 2014-06-16 19:41:22 +02:00
mpl a34f9e2669 publisher app 
http://camlistore.org/issue/365

Change-Id: I281fdcbbe6a2bdf15607e75a21bc93b453f82c85
 2014-06-15 00:03:03 +02:00
mpl 9723b5c814 apps: more generic app handler work, to prepare for publisher 
http://camlistore.org/issue/365

Change-Id: I3c34ba1f09debc3aaaf68ad9fb11b595470b6b5d
 2014-06-14 18:36:10 +02:00
mpl c627b6325a app: register app authmode during setup 
Fixes: http://camlistore.org/issue/439

Change-Id: I5d78f1fb764b83a0a9afc35b4ffb62a1a7934ed0
 2014-05-21 19:57:09 +02:00
mpl 21dda2b4ef app/hello: dummy server application (hello world) 
Change-Id: I6690b9459325af5a76d1de679d56701eefdd195e
 2014-05-08 22:15:01 +02:00
Brad Fitzpatrick 168c902865 serverinit: in genconfig, always register the importer handler 
Change-Id: Ib6267b93e503dccffbf1a8ce3a3a95c9a04cf454
 2014-04-17 10:23:53 -07:00
Brad Fitzpatrick 5d4f2dfefe serverinit: let genconfig's update_golden flag, even if files matched 
Change-Id: I93cdd1b9a1945a975a1c8d2097e914ae2a8c77c1
 2014-04-17 10:20:05 -07:00
Brad Fitzpatrick e0dad70437 serverinit: re-baseline genconfig testdata 
Just re-run --update_golden, to get it all into canonical format,
to minimize the size of the next diff.

Apparently these have been hand-edited a lot.

Change-Id: Ib7dbd9d629b5501280652c695f538007934708d4
 2014-04-17 10:18:55 -07:00
mpl 9c7bb3e793 Merge "pkg/serverinit: fix published type, add parser test" 2014-04-05 13:09:53 +00:00
mpl 291320d451 pkg/index: cleanup, refactor sub pkgs 
http://camlistore.org/issue/263

Change-Id: I319bb097f0ce30b2bd5271b5c3bbff92b8dcc318
 2014-04-05 01:22:05 +02:00
mpl 58e8a7f78b pkg/serverinit: fix published type, add parser test 
Because when the UI handler constructor parses the publish config, it
wants a jsonconfig List, not a []string.

Context: bfbf660cd1

Change-Id: If4e6553072d95077a5dc73c28ac527a31074d441
 2014-04-04 23:28:34 +02:00
Brad Fitzpatrick bf2a7b60a3 Importer overhaul. Only Foursquare is currently working. 
The rest are currently stubbed out and will need updating
to the new APIs.

Change-Id: I9d70302b3ac1026192413bf9dcd3c8f1eb420349
 2014-03-31 10:56:30 -07:00
Brad Fitzpatrick bf5740afb2 Add BaseURL method to handler Loader. 
Change-Id: Ibc61cd19798aa1b0858cbe40b10544953db67b36
 2014-03-23 17:13:40 -07:00
Philip Snowberger 1ba212d707 Add kv queue to google drive / google cloud storage sync handlers 
Change-Id: I978ae42dbc6ce2840071cbadaf9351498a23415f
 2014-03-21 11:46:22 -07:00
Brad Fitzpatrick bf4116eb93 genconfig: picasaa importer doesn't have a path 
Was removed during codereview, but not removed from here.

Change-Id: I51309f46fe9747bcc0b2476813acac9120c32b28
 2014-03-18 10:10:10 -07:00