I had intended for this to be a small change.
I was going to just add context.Context to the BlobReceiver interface,
but then I saw blob.Fetcher could also use one, so I decided to do two
in one CL.
And then it got a bit infectious and ended up touching everything.
I ended up doing SubFetch in the process by necessity.
At a certain point I finally started using context.TODO() in a few
spots, but not too many. But removing context.TODO() will come in the
future. There are more blob storage interfaces lacking context, too,
like RemoveBlobs.
Updates #733
Change-Id: Idf273180b3f8e397ac5929c6d7f520ccc5cdce08
As the priority is to fix GCE instances, the port for the http-01
challenge is not configurable for now (80) even not on GCE, but it will
be in a follow-up change.
update golang.org/x/crypto/* (for acme) to rev
13931e22f9e72ea58bb73048bc752b48c6d4d4ac
update golang.org/x/sys/* to rev
fff93fa7cd278d84afc205751523809c464168ab (because unix is a dep of
crypto)
remove warning about Let's Encrypt security issue from pkg/deploy/gce
I had to manually exclude vendor/golang.org/x/crypto/acme/jws_test.go
for now because it contains a private key, and git whines about it, and
i could not override it.
Fixes#1033
Change-Id: Ie4f2049e97892dee9ab513300a5f12e64976aec8
followup of 7eda9fd502
We want existing Perkeep instances on GCE to be able to keep on running
with their DBNames-style existing databases.
To that end, we introduce the "perkeep-config-version" metadata key,
which will be set by the launcher from now on.
When perkeep configuration starts, it can lookup that key. If it is set,
it means we're in a newly created instance, and we don't need to care
about DBNames compatibility. If not, we modify the low level
configuration on the fly, so that it keeps on using the old DBNames
values that were set for a GCE instance.
Change-Id: I611811fdb9c68777c2ba799e9047d00ec0bae040
Part of the project renaming, issue #981.
After this, users will need to mv their $GOPATH/src/camlistore.org to
$GOPATH/src/perkeep.org. Sorry.
This doesn't yet rename the tools like camlistored, camput, camget,
camtool, etc.
Also, this only moves the lru package to internal. More will move to
internal later.
Also, this doesn't yet remove the "/pkg/" directory. That'll likely
happen later.
This updates some docs, but not all.
devcam test now passes again, even with Go 1.10 (which requires vet
checks are clean too). So a bunch of vet tests are fixed in this CL
too, and a bunch of other broken tests are now fixed (introduced from
the past week of merging the CL backlog).
Change-Id: If580db1691b5b99f8ed6195070789b1f44877dd4
I created https://github.com/camlistore/old-cam-snapshot with a snapshot
of our git repo before we start rearranging things.
This way we won't break anybody depending on camlistore.org/* Go
packages after the move. They just won't get any updates. Actually
things will probably break if they run "go get -u". Need to
investigate what happens there.
Change-Id: I45e109c18323e65bd76706faa08d955dcbc5f6c6
This switches most usages of the pre-1.7 context library to use the
standard library. Remaining usages are in:
app/publisher/main.go
pkg/fs/...
Change-Id: Ia74acc39499dcb39892342a2c9a2776537cf49f1
To rev 9dfe39835686865bff950a07b394c12a98ddc811
The xsrftoken package now lives in golang.org/x/net instead of code.google.com.
Change-Id: I4d98b1e50099dc7a1e1188f5c4311cd28c79f44a
This change allows the launcher to create a Google Cloud Project on
behalf of the user, as well as enabling the required APIs.
Radio buttons have been added to the form, such that the user can choose
between creating a new project, or reusing one of their existing
projects. Some javascript (jquery) code has been added to modify the
rest of the form fields depending on which button is selected, and if
the project id field is empty or not.
On project creation, if the user didn't pick one, we generate a project
ID that is a (0,1e10) random number prefixed with "camlistore-launcher-",
and we then use google.golang.org/api/cloudresourcemanager to create
the project.
On project creation success, we redirect back to the form, but with the
project id input field pre-filled with the newly created project
(passed as url query parameter during redirection). The user can then
set the other fields, and submit again for instance creation.
Note that the user still has to enable billing for the selected project
BEFORE instance creation, even when we create the project for them.
Right before instance creation, we now also take care of enabling the
required APIs (Compute Engine, Cloud Storage, Stackdriver Logging)
through the use of google.golang.org/api/servicemanagement. Note that
suprisingly, even though the docs
(https://cloud.google.com/service-management/getting-started) specify
that we should enable the Service Management API on the project in order
to use it, we apparently don't need to do so. So far.
vendor: add
google.golang.org/api/cloudresourcemanager/v1
google.golang.org/api/servicemanagement/v1
google.golang.org/api/gensupport/
at rev 48e49d1645e228d1c50c3d54fb476b2224477303
Fixes#819
Change-Id: Id58b453e432e4d7c29d3ede04917ab4d244f142e
In addition to API changes, the V2 logging API uses
google.golang.org/grpc, which does not rely directly on net/http. This
is a problem for us because, among other things, when needed (on
CoreOS), we insert the system Root CAs in the TLSConfig of the
default http client through httputil.InstallCerts, which has no effect
on google.golang.org/grpc calls. Therefore, we switch to
github.com/bradfitz/grpc-go instead, which does rely on net/http.
And since GRPC requires HTTP/2, we also now need to manually configure
the default transport (since we're already modifying it) to be HTTP/2
enabled.
Related dependencies updated/added:
updated:
cloud.google.com/go/logging
a64eb5d53f434cc548fd6b2898b5a3e8a0c31b78
github.com/golang/protobuf
8ee79997227bf9b34611aee7946ae64735e6fd93
added:
github.com/bradfitz/grpc-go
188a132adcfba339f1f2d5da52498451341f9ee8
as a replacement for google.golang.org/grpc
github.com/googleapis/gax-go
da06d194a00e19ce00d9011a13931c3f6f6887c7
google.golang.org/genproto/googleapis/logging
08f135d1a31b6ba454287638a3ce23a55adace6f
google.golang.org/genproto/googleapis/datastore
08f135d1a31b6ba454287638a3ce23a55adace6f
google.golang.org/api/support/bundler
e4c04685e5d7db47ff294aa9e514b3a638c431c9
google.golang.org/genproto/googleapis/rpc/status
08f135d1a31b6ba454287638a3ce23a55adace6f
Fixes#873
Change-Id: I3ba14c94122dd9f1b88ef61c2dc26430690d28a9
This CL changes the GCE launcher to work with the new features of
camlistored: i.e. that it can automatically get a hostname in
camlistore.net, and that it can get an HTTPS certificate from Let's
Encrypt, for said hostname.
In order for the user to easily (without having to look at the logs)
know what their hostname is, camlistored stores it as the
"camlistore-hostname" key in the custom metadata of the GCE instance.
The deployer can then query for that key, to report the hostname on the
instance creation success page.
Change-Id: Iaaef2d51f34fa5e1e0ee90097919abab7ee72a12
rm google.golang.org/cloud
add cloud.google.com/go at a47b182e769f5e75f5fc927ff6ee2678f7f552cf
update google.golang.org/api to 63cb68f1e3834e44683ca062ddf06cb9a889380a
update google.golang.org/grpc to
0e6ec3a4501ee9ee2d023abe92e436fd04ed4081
update go4.org to f5283521d7365fb2875408726e9cbf349f173767
fix in cmd/ pkg/ server/
TODO(mpl): fix misc/docker tools as well. next CL.
Fixes#832
Change-Id: I842b968a0afea8a5822913bd614d67cdbe50ee63
Rids us of the IsMonthly hack.
We still need to do something about AddTemplateTheme in handler.go
though, see TODO there. In another change.
Also, change servePage to take a struct for its args, more in line with
Go style of few arguments.
Context: issue #815
Change-Id: Ib6799be31b78af3010eee942034f2ac0e8e1d76e
I had forgotten that the launcher also uses website/tmpl/page.html when
I introduced the gross IsMonthly hack. Which bites us now that the
launcher is restarted.
I'll rethink the IsMonthly mechanism right after.
Fixes#815
Change-Id: I7d2f7fd553e0ae9ab386266d310ee8175ce2d4ea
The google Endpoint is now available in upstream oauth2, we no longer
need to define it ourselves.
Fixes#810
Change-Id: I37db869ff7696c53ba456ecba55e72c8c86dd119
The import path was added to the go file that included the package
documentation if one existed. Otherwise, I used what seemed to be the
primary file for the package.
Fixes#689
Change-Id: If51be0e86529fd6f179e80af6781e639f8550fd2
I don't think we will ever want that wiping feature for the production handler.
As for the command-line tool, people who use it know enough to destroy
their instances themselves if they need to.
Change-Id: I94cd92ebc4904cc3249b8dcff946a025daace589
mpl