Commit Graph

22 Commits

Author SHA1 Message Date
Brad Fitzpatrick 8a0a3e8649 Rename camdeploy to pk-deploy.
Updates #981

Change-Id: I4c6c16135f3239bb4b8d2d187e9537a4abf7a3fe
2018-01-02 22:10:37 -08:00
Brad Fitzpatrick c3d05cdce9 Move more packages out of pkg/ and into internal/
Moved hashutil, httputil, osutil, netutil,
images, media, magic, video, and rollsum.
2018-01-02 21:03:30 -08:00
Brad Fitzpatrick d6a0b05df0 Rename import paths from camlistore.org to perkeep.org.
Part of the project renaming, issue #981.

After this, users will need to mv their $GOPATH/src/camlistore.org to
$GOPATH/src/perkeep.org. Sorry.

This doesn't yet rename the tools like camlistored, camput, camget,
camtool, etc.

Also, this only moves the lru package to internal. More will move to
internal later.

Also, this doesn't yet remove the "/pkg/" directory. That'll likely
happen later.

This updates some docs, but not all.

devcam test now passes again, even with Go 1.10 (which requires vet
checks are clean too). So a bunch of vet tests are fixed in this CL
too, and a bunch of other broken tests are now fixed (introduced from
the past week of merging the CL backlog).

Change-Id: If580db1691b5b99f8ed6195070789b1f44877dd4
2018-01-01 16:03:34 -08:00
mpl 4784b22aeb pkg/serverinit: on GCE, reset name if instance name is in camlistore.net
On GCE, on startup, we did not set the camliNetIP in the high-level
config if the camlistore-hostname instance var vas already set.

The camliNetIP presence in the config is the signal for the server
that it should be configured as part of camlistore.net (and in
particular that it should update the record for its name on the
camlistore.net DNS server). Part of this configuration is to set the
camlistore-hostname var for the instance.

Therefore, a server which had been configured once as described above,
would not, on a subsequent restart, behave as if part of camlistore.net
and would skip the related configuration code path. Unless the
camlistore-hostname var was manually wiped before restart.

As this manual step is not an obvious one, this CL changes the
initialization, so that if a camlistore-hostname var is set, but the
value is a subdomain of "camlistore.net", the var is treated as empty
and initialization proceeds as if with a new server.

Fixes #963

Change-Id: Iab70185d7b90ef7e70bb831d363ff9d525922e35
2017-10-04 17:36:02 +02:00
mpl 9e34d14ef5 pkg/serverinit: enable sharing on GCE by default
The default server config on GCE (as deployed by the launcher) did not
have a share handler. This CL adds one, so that users can benefit from
the new sharing feature from the web UI.

Also, the button for sharing from the web UI does not appear anymore if
the config that the web UI gets from the discovery does not have a
"shareRoot", because it's a strong hint that the server does not have a
share handler.

Change-Id: I6c444995339fda8dba864b1d6729fb7c1b6d72bd
2017-04-05 16:17:46 +02:00
mpl 753bd38ca9 misc/docker/server: build with make.go
So far, when building camlistored docker image for CoreOS, we were not
using make.go, and we were neither running gopherjs nor embedding the
resources (but rather provide the UI resources at their default
filesystem location).

Now that we're using gopherjs for the web UI, it is a hard dependency
for the camlistore server.
We could reproduce the steps in make.go to build gopherjs, run it to
build the web ui resources, and then move the resources at the right
place, but since make.go already does the equivalent work it seems
to make more sense to use it, which is the main point of this CL.

Similarly, it seems to make more sense to now build a binary with the
resources embedded, which is the default make.go behaviour, instead of
building a "raw" camlistored, and provide the resources as additional
directories in the container image, so this CL takes that approach too.

Finally, it was necessary to add the "-static" flag to make.go, so we
can keep on building a static camlistored binary, that does not rely on
libc for DNS. Because our container image is FROM SCRATCH, with just the
necessary binaries, in order to get a container image of a reasonable
size.

One drawback of now using make.go in
misc/docker/server/build-camlistore-server.go is we're doing some
unnecessary (since we're already running in the isolation of a
container) copying to the virtual gopath, but that seems a very tiny
price to pay. Especially considering how rarely we run that code.

Change-Id: I416c86d366cd4ed2d3b8b1636a6a65a83b9f15d7
2017-04-05 02:10:12 +02:00
mpl 8813375592 pkg/deploy: no self-signed certs, name in camlistore.net
This CL changes the GCE launcher to work with the new features of
camlistored: i.e. that it can automatically get a hostname in
camlistore.net, and that it can get an HTTPS certificate from Let's
Encrypt, for said hostname.

In order for the user to easily (without having to look at the logs)
know what their hostname is, camlistored stores it as the
"camlistore-hostname" key in the custom metadata of the GCE instance.
The deployer can then query for that key, to report the hostname on the
instance creation success page.

Change-Id: Iaaef2d51f34fa5e1e0ee90097919abab7ee72a12
2017-01-13 16:47:04 +01:00
mpl 620388bd57 server/camlistored: request a name in camlistore.net
In order to use HTTPS, one must have a certificate, and one must have a
domain name for which the certificate is valid.
The first part is solved by the use of Let's Encrypt. For the second
part, we want to provide to any Camlistore instance a name such as
<gpgKeyId>.camlistore.net, where gpgKeyId is the fingerprint of its GPG
key. The DNS for camlistore.net agrees to add a record for that name if
and only if the Camlistore instance can prove it owns the GPG key, as
well as the IP address bound to that name in the DNS record.

A protocol such as the above is already implemented in pkg/gpgchallenge.

This CL:
- uses the client-side of the gpgchallenge protocol in camlistored, so
that it can claim a hostname in camlistore.net on startup (and then use
that hostname when requesting a certificate from Let's Encrypt).
- adds the configuration parameter "CamliNetIP" for the high-level
config. This parameter specifies the IP address that camlistored will
supply during the gpgpchallenge, so it can  prove to the DNS server that
we own this address.

Fixes #722

Change-Id: I6bf4ec149b6dffd0ae93a6fa7bf208b2e8a05445
2017-01-13 00:43:24 +01:00
mpl ab06dbd80d vendor: rename google.golang.org/cloud to cloud.google.com/go
rm google.golang.org/cloud
add cloud.google.com/go at a47b182e769f5e75f5fc927ff6ee2678f7f552cf
update google.golang.org/api to 63cb68f1e3834e44683ca062ddf06cb9a889380a
update google.golang.org/grpc to
0e6ec3a4501ee9ee2d023abe92e436fd04ed4081
update go4.org to f5283521d7365fb2875408726e9cbf349f173767

fix in cmd/ pkg/ server/

TODO(mpl): fix misc/docker tools as well. next CL.

Fixes #832

Change-Id: I842b968a0afea8a5822913bd614d67cdbe50ee63
2016-09-08 15:14:12 +02:00
Brad Fitzpatrick 06f7accf31 Require Go 1.5 (due out soon), start move from third_party to vendor.
See https://golang.org/s/go15vendor

This commit moves the google.golang.org/cloud/... packages. The rest
of third_party will move later.

This commit is a dependency for some cloud logging work which we'll
develop in Camlistore's tree.
2015-08-15 22:45:11 +02:00
Brad Fitzpatrick 58ec210183 For GCE, don't send and ignore a 'localhost' hostname.
Change-Id: I329e86783c4b14deb9b8dfcb6ac827e3436d5faa
2015-04-02 03:40:19 -07:00
Brad Fitzpatrick c7213346d0 Revert "Temporary GCE-specific log spam for debugging."
This reverts commit 314162705c.
2015-04-02 03:25:46 -07:00
Brad Fitzpatrick 314162705c Temporary GCE-specific log spam for debugging.
Change-Id: I62de01cd0be48636e6cac8cd4e7bf8d46f9abf64
2015-04-02 02:29:08 -07:00
Brad Fitzpatrick c9a0beae45 Add new 'env' package to detect the type of environment.
Also, delete my old gce package from third_party and only use the
google metadata package (which my gce package became, and which was
also already vendored into third_party)

Fixes #596

Change-Id: I64fd6f1e9dc6f433466f91f81efd2ecbf039334f
2015-04-01 08:45:36 -07:00
Brad Fitzpatrick 4e8413c501 When on GCE, get UI assets from Docker image.
(The Docker image is still in development)

Change-Id: I0ba1d7482b42d578d6ddbd7cfb42780e29feab24
2015-04-01 02:49:26 -07:00
mpl b78ce17e13 serverinit: add genconfig for blobpacked on Google Cloud Storage.
And enable it (blobpacked) on GCE.

Change-Id: I7b51c186325e80987e48d32392d88bb4c30ee3d5
2015-02-04 19:08:43 +01:00
mpl cfd784269e deploy/gce: use subdirs for config and blobs (instead of buckets)
http://camlistore.org/issue/528

Change-Id: Icb558d7de032f9367e8fb100be87ef24c2982c23
2014-12-17 16:20:58 +01:00
Brad Fitzpatrick 5903e89490 On GCE, put the UI's thumbnail metadata cache on MySQL too.
Change-Id: Ic3f1238deeb6b569b30e6c3f6b0def2b68f13d93
2014-08-23 19:55:04 -07:00
Brad Fitzpatrick a2a738c6ba GCE: keep sync-to-index queue on MySQL
Change-Id: I660894ebd03fb04beaa852d92073fb2ad46a98c5
2014-08-23 09:46:56 -07:00
Brad Fitzpatrick 956e838cc0 If on GCE, use MySQL if MYSQLDB_PORT env var is set. (to linked Docker container)
Change-Id: I4fd4718e6912e7ad8127395cdf6972337a8766bc
2014-08-18 23:17:17 -07:00
Brad Fitzpatrick d2f6f9e057 serverinit: in the GCE env, set the BaseURL based on the IP address for now.
Change-Id: I3be51c52ac78e6f6d7cf373cbc358ac1244526c4
2014-08-15 22:06:13 -07:00
Brad Fitzpatrick 9fbec8dd12 serverinit, camlistored: support environment-specific default config files
When running on GCE now, the default config file is a function of
attributes on the instance.

Change-Id: I62fe23ecf28757a6baee6bd432f080f223a389c5
2014-08-15 20:50:59 -07:00