Commit Graph

80 Commits

Author SHA1 Message Date
Brad Fitzpatrick 930f049ec9 Merge "website: make camlistore.org go-import meta tags reference a snapshot" 2018-01-02 00:42:06 +00:00
Brad Fitzpatrick 39d904c186 Merge "Rename import paths from camlistore.org to perkeep.org." 2018-01-02 00:41:55 +00:00
Mathieu Lonjaret f75df9284a Merge "pkg/deploy/gce: add Google Cloud Free Tier option" 2018-01-02 00:37:48 +00:00
Brad Fitzpatrick d6a0b05df0 Rename import paths from camlistore.org to perkeep.org.
Part of the project renaming, issue #981.

After this, users will need to mv their $GOPATH/src/camlistore.org to
$GOPATH/src/perkeep.org. Sorry.

This doesn't yet rename the tools like camlistored, camput, camget,
camtool, etc.

Also, this only moves the lru package to internal. More will move to
internal later.

Also, this doesn't yet remove the "/pkg/" directory. That'll likely
happen later.

This updates some docs, but not all.

devcam test now passes again, even with Go 1.10 (which requires vet
checks are clean too). So a bunch of vet tests are fixed in this CL
too, and a bunch of other broken tests are now fixed (introduced from
the past week of merging the CL backlog).

Change-Id: If580db1691b5b99f8ed6195070789b1f44877dd4
2018-01-01 16:03:34 -08:00
Brad Fitzpatrick b9a68c3000 website: make camlistore.org go-import meta tags reference a snapshot
I created https://github.com/camlistore/old-cam-snapshot with a snapshot
of our git repo before we start rearranging things.

This way we won't break anybody depending on camlistore.org/* Go
packages after the move. They just won't get any updates. Actually
things will probably break if they run "go get -u". Need to
investigate what happens there.

Change-Id: I45e109c18323e65bd76706faa08d955dcbc5f6c6
2017-12-29 18:20:09 -08:00
mpl 253e7cb5d4 pkg/deploy: ignore Domain field from page template
Fixes #985

Change-Id: If110d392885fb89d0403ab9eb3aaf8dcdc3fe249
2017-12-16 02:01:36 +01:00
Paul Lindner 04b9c0b1c4 pkg: lint fixes for "context.Context should be the first parameter of a function"
Change-Id: I0185eec5616c72d859978aec03880860421c7193
2017-12-13 11:33:21 -08:00
Paul Lindner ba92702834 all: lint fixes for "should omit 2nd value from range"
Change-Id: I7bb19d376f96a39ecae7dbdb4d6808f704bae5fb
2017-12-13 11:31:25 -08:00
Paul Lindner 15feaeb24c all: lint fixes for 'error strings should not be capitalized or end with punctuation or a newline'
Change-Id: I9c3766a51ac8be694ae76befff4b6fa9a85e34eb
2017-12-11 06:13:25 -08:00
Paul Lindner b9c2da7032 all: replace usage of deprecated oauth2.NoContext with context.Background()
Change-Id: I4fdc9719911498dc6864b4f0189ab8b01db68308
2017-12-09 14:48:53 -08:00
Paul Lindner be2dd5ee43 pkg/deploy/gce: use a single append to concatenate two slices
Change-Id: I3e4e0a8f9e934007513737155d15ee03494ccf6b
2017-12-05 11:30:41 -08:00
Paul Lindner b09cd377d7 Switch to stdlib context from golang.org/x/net/context
This switches most usages of the pre-1.7 context library to use the
standard library.  Remaining usages are in:

  app/publisher/main.go
  pkg/fs/...

Change-Id: Ia74acc39499dcb39892342a2c9a2776537cf49f1
2017-11-26 01:12:26 -08:00
Paul Lindner b24717a83f vendor: update xsrftoken
To rev 9dfe39835686865bff950a07b394c12a98ddc811

The xsrftoken package now lives in golang.org/x/net instead of code.google.com.

Change-Id: I4d98b1e50099dc7a1e1188f5c4311cd28c79f44a
2017-11-20 08:47:16 -08:00
Paul Lindner fa46c3935d Correct various misspelled words
Change-Id: I236e880526e4c2b0bd318da041983d557e0aa885
2017-09-11 08:33:31 -07:00
mpl fdf50ad339 pkg/deploy/gce: update GCE API service name
Fixes #943

Change-Id: I676b76dc6b7e25bb5c418150aedf2766a1089ee8
2017-09-07 16:24:00 +02:00
mpl 56fb792dd1 pkg/deploy/gce: add Google Cloud Free Tier option
Fixes #912

Change-Id: Ie914536c409411871627911d0bf802688a7aa685
2017-06-08 01:10:58 +02:00
mpl ec6f46f59b pkg/deploy: add debugging print
Change-Id: I4e466e29167ff702ce5df0b1cf125aa6ccce5aca
2017-05-05 19:23:54 +02:00
mpl af10e5a8dd pkg/deploy/gce: also create Google Cloud Project for user
This change allows the launcher to create a Google Cloud Project on
behalf of the user, as well as enabling the required APIs.

Radio buttons have been added to the form, such that the user can choose
between creating a new project, or reusing one of their existing
projects. Some javascript (jquery) code has been added to modify the
rest of the form fields depending on which button is selected, and if
the project id field is empty or not.

On project creation, if the user didn't pick one, we generate a project
ID that is a (0,1e10) random number prefixed with "camlistore-launcher-",
and we then use google.golang.org/api/cloudresourcemanager to create
the project.

On project creation success, we redirect back to the form, but with the
project id  input field pre-filled with the newly created project
(passed as url query parameter during redirection). The user can then
set the other fields, and submit again for instance creation.

Note that the user still has to enable billing for the selected project
BEFORE instance creation, even when we create the project for them.

Right before instance creation, we now also take care of enabling the
required APIs (Compute Engine, Cloud Storage, Stackdriver Logging)
through the use of google.golang.org/api/servicemanagement. Note that
suprisingly, even though the docs
(https://cloud.google.com/service-management/getting-started) specify
that we should enable the Service Management API on the project in order
to use it, we apparently don't need to do so. So far.

vendor: add
google.golang.org/api/cloudresourcemanager/v1
google.golang.org/api/servicemanagement/v1
google.golang.org/api/gensupport/
at rev 48e49d1645e228d1c50c3d54fb476b2224477303

Fixes #819

Change-Id: Id58b453e432e4d7c29d3ede04917ab4d244f142e
2017-04-21 18:09:10 +02:00
mpl 1e69b9f784 vendor: update cloud.google.com/go/logging to V2 API
In addition to API changes, the V2 logging API uses
google.golang.org/grpc, which does not rely directly on net/http. This
is a problem for us because, among other things, when needed (on
CoreOS), we insert the system Root CAs in the TLSConfig of the
default http client through httputil.InstallCerts, which has no effect
on google.golang.org/grpc calls. Therefore, we switch to
github.com/bradfitz/grpc-go instead, which does rely on net/http.

And since GRPC requires HTTP/2, we also now need to manually configure
the default transport (since we're already modifying it) to be HTTP/2
enabled.

Related dependencies updated/added:

updated:

cloud.google.com/go/logging
a64eb5d53f434cc548fd6b2898b5a3e8a0c31b78

github.com/golang/protobuf
8ee79997227bf9b34611aee7946ae64735e6fd93

added:

github.com/bradfitz/grpc-go
188a132adcfba339f1f2d5da52498451341f9ee8
as a replacement for google.golang.org/grpc

github.com/googleapis/gax-go
da06d194a00e19ce00d9011a13931c3f6f6887c7

google.golang.org/genproto/googleapis/logging
08f135d1a31b6ba454287638a3ce23a55adace6f

google.golang.org/genproto/googleapis/datastore
08f135d1a31b6ba454287638a3ce23a55adace6f

google.golang.org/api/support/bundler
e4c04685e5d7db47ff294aa9e514b3a638c431c9

google.golang.org/genproto/googleapis/rpc/status
08f135d1a31b6ba454287638a3ce23a55adace6f

Fixes #873

Change-Id: I3ba14c94122dd9f1b88ef61c2dc26430690d28a9
2017-02-06 19:59:24 +01:00
mpl 8813375592 pkg/deploy: no self-signed certs, name in camlistore.net
This CL changes the GCE launcher to work with the new features of
camlistored: i.e. that it can automatically get a hostname in
camlistore.net, and that it can get an HTTPS certificate from Let's
Encrypt, for said hostname.

In order for the user to easily (without having to look at the logs)
know what their hostname is, camlistored stores it as the
"camlistore-hostname" key in the custom metadata of the GCE instance.
The deployer can then query for that key, to report the hostname on the
instance creation success page.

Change-Id: Iaaef2d51f34fa5e1e0ee90097919abab7ee72a12
2017-01-13 16:47:04 +01:00
mpl 6a5f91a6f2 website: launcher: show when in WIP mode
Change-Id: Ie2d743236316ca7974268e38639ebd86b11b3fa2
2016-10-15 02:08:09 +02:00
mpl ab06dbd80d vendor: rename google.golang.org/cloud to cloud.google.com/go
rm google.golang.org/cloud
add cloud.google.com/go at a47b182e769f5e75f5fc927ff6ee2678f7f552cf
update google.golang.org/api to 63cb68f1e3834e44683ca062ddf06cb9a889380a
update google.golang.org/grpc to
0e6ec3a4501ee9ee2d023abe92e436fd04ed4081
update go4.org to f5283521d7365fb2875408726e9cbf349f173767

fix in cmd/ pkg/ server/

TODO(mpl): fix misc/docker tools as well. next CL.

Fixes #832

Change-Id: I842b968a0afea8a5822913bd614d67cdbe50ee63
2016-09-08 15:14:12 +02:00
mpl 72c6f6adcf pkg/deploy: show camli version to be deployed
TODO(mpl): make the image creation tool update the VERSION file. Next
CL.

Change-Id: Ie39052d7132ba053f00a42a0e146243e99b576b6
2016-07-25 18:33:23 +02:00
mpl 1087a9d837 website: mv downloadBox to all.css
Rids us of the IsMonthly hack.

We still need to do something about AddTemplateTheme in handler.go
though, see TODO there. In another change.

Also, change servePage to take a struct for its args, more in line with
Go style of few arguments.

Context: issue #815

Change-Id: Ib6799be31b78af3010eee942034f2ac0e8e1d76e
2016-06-20 17:51:46 +02:00
mpl 1587a8a05b pkg/deploy/gce: temp fix to bring /launch back up
I had forgotten that the launcher also uses website/tmpl/page.html when
I introduced the gross IsMonthly hack. Which bites us now that the
launcher is restarted.

I'll rethink the IsMonthly mechanism right after.

Fixes #815

Change-Id: I7d2f7fd553e0ae9ab386266d310ee8175ce2d4ea
2016-06-17 22:22:26 +02:00
Joseph Anthony Pasquale Holsten 118585fef1 pkg/constants/google: remove, use oauth2/google instead
The google Endpoint is now available in upstream oauth2, we no longer
need to define it ourselves.

Fixes #810

Change-Id: I37db869ff7696c53ba456ecba55e72c8c86dd119
2016-06-08 16:15:39 -07:00
Brad Fitzpatrick 75d60962f6 Move remaining stuff in third_party/* to vendor/*
Change-Id: Ifbcc02817083cba68d8c1acec3e6ec50e8f61149
2016-04-20 16:49:15 -07:00
Tamás Gulácsi 7402cc0efd Delete misc unused objects
Using honnef.co/go/unused/cmd/unused

Change-Id: I672b3cb77f09e9bd80dcdc149cde4f7f2939e451
2016-04-06 17:59:51 +02:00
Will Norris 77ed42edf8 add canonical import paths
The import path was added to the go file that included the package
documentation if one existed.  Otherwise, I used what seemed to be the
primary file for the package.

Fixes #689

Change-Id: If51be0e86529fd6f179e80af6781e639f8550fd2
2016-03-13 19:57:14 -07:00
Mathieu Lonjaret 14a8a9f6a4 Merge "pkg/deploy/gce: fix error overescaping" 2016-03-02 23:57:24 +00:00
mpl 8d37fd1a6f pkg/deploy/gce: remove old TODO
I don't think we will ever want that wiping feature for the production handler.
As for the command-line tool, people who use it know enough to destroy
their instances themselves if they need to.

Change-Id: I94cd92ebc4904cc3249b8dcff946a025daace589
2016-02-25 23:28:55 +01:00
mpl 831d6a9ea1 pkg/deploy/gce: fix error overescaping
Fixes issue #659

Change-Id: I74171bd215b1bffedda88111b8fe1f6772532b18
2016-02-23 19:05:48 +01:00
mpl 243c7c146a deploy/gce: do not ask for password
Issue #656

Change-Id: Ida28675588b30eed786957aaaca674cadbf9c404
2016-02-10 19:43:56 +01:00
Brad Fitzpatrick a3d6ed6fa6 launcher: don't default password to the project ID
Change-Id: I9e68d768a436a5e3d3b445982feba3a9520fb643
2015-12-28 21:35:57 -08:00
Brad Fitzpatrick 5a2fe47b73 cmd/camdeploy: fix build, embed a client ID/secret, accept region or zone
Much easier to use now.
2015-12-28 15:15:11 -08:00
Brad Fitzpatrick 5b5cb4cf5a pkg/deploy/gce: remove an errant period and clarify where to delete 2015-12-27 18:44:59 -08:00
Brad Fitzpatrick 1bed370b47 pkg/deploy/gce: launcher wording & behavior tweaks
Update wording and links. Change default zone to a default
region. Generate a random password if blank.
2015-12-27 18:43:17 -08:00
Brad Fitzpatrick b22a78deda Fix launcher. Work in localhost dev mode more easily. 2015-12-27 13:52:35 -08:00
mpl 32e922ad3d vendor: mv go4.org/cloud/gceutil go4.org/cloud/google/gceutil
Change-Id: I21c27b250b3a272f56b387ef07169a29da85f76c
2015-12-23 17:59:26 +01:00
mpl 38fec12dd5 deploy/gce: fix redundant context import
Change-Id: I37117d5b5641bdc057c2ece093a602be6b8e0850
2015-12-15 16:16:51 +01:00
Tamás Gulácsi 9f7e1df32b Remove pkg/context
Use golang.org/x/net/context instead.

Fixes #648.

Change-Id: I676a2f25458be97610a49d6f954f2102cbd373fa
2015-12-12 23:09:02 +01:00
mpl cf08d789f2 website: sidestep failInTests panics from osutil
Because camweb now depends on pkg/deploy/gce, which has global vars
initialized with calls from osutil.

And test website from devcam test

Change-Id: I233c229e96ded5bfde4e2c1fa7a18bbf757ab1fb
2015-12-07 22:48:12 +01:00
mpl d9a3bbc73a pkg/vendor: update google.golang.org/cloud/storage
rev 2375e186ca77be721a7c9c7b13a659738a8511d2

This is so we can use storage.Client in pkg/wkfs/gcs (and later in
other places) instead of pkg/googlestorage

Change-Id: I1d0eafbffd40e0326c41214692754f6671ec1d5d
2015-12-03 00:55:53 +01:00
mpl 1e2a306e8d vendor: add go4.org/cloud, update google.golang.org/api, and golang.org/x/net/context
Import go4.org/cloud/cloudlaunch and go4.org/cloud/gceutil, which were
previously in pkg/cloudlaunch, at rev
2c497a8c195996d6c48402c150da4919c1d99f33

Because the version on go4.org was updated to use the latest
google.golang.org/api/* , we also update our
vendor/google.golang.org/api/* to rev
030d584ade5f79aa2ed0ce067e8f7da50c9a10d5

This in turns requires to add vendor/golang.org/x/net/context/ctxhttp
(and update vendor/golang.org/x/net/context/) to rev
4f2fc6c1e69d41baf187332ee08fbd2b296f21ed

Also adding vendor/google.golang.org/api/gensupport for the same reason.

Change-Id: I9d6c3059626f7ecc0fbf2df0415043a464d27caa
2015-11-25 19:29:46 +01:00
mpl 12eddf9c19 vendor: go4.org/strutil go4.org/syncutil
They were internal packages (under pkg), which we are now moving to
go4.org, so we in turn need to vendor them in now.

Change-Id: I92224f731404d0bd4ca1c57492bed37cb3367ed4
2015-11-21 00:21:20 +01:00
mpl d3e4a01b4f pkg/deploy/gce: rm debugging handler
I broke it recently by removing gce.DevHandler and I don't think it's of
any use anymore nowadays.

Change-Id: I8d9ff85c980999bc5fb61ea08e1fce2cd4b7bea9
2015-11-18 18:49:34 +01:00
mpl 775da29abe pkg/deploy/gce: always use latest stable CoreOS
Added pkg/cloudlaunch/gceutil package.

Fixes issue #649

Change-Id: I47830489b29de9c555b4706fe387be08e3f57e16
2015-11-18 00:17:33 +01:00
mpl 621388f0bf camweb: send deploy handler logs to GCLogging as well
When logging to Google Cloud Logging, make sure the deploy handler logs
are written there too.

We tag them with the from:camli-gce-launcher label.

Change-Id: I4dab010009f98efd2aeba7c57dc4ab3ca6a566b3
2015-11-16 23:40:05 +01:00
mpl a2aee25cfb website: update (camlistored) deployer to run on GCE
Change-Id: I205c6df51e8d8c4bcf2710fc9da1db68419fc2df
2015-11-13 17:41:46 +01:00
mpl afa5d42469 pkg/deploy/gce: remove now useless debugging code
Change-Id: Ib2f71056330805287932f259b0c2eec3e1b4b09f
2015-10-21 16:08:25 +02:00