Commit Graph

35 Commits

Author SHA1 Message Date
Brad Fitzpatrick 43f34e5cc5 auth, webserver, serverinit: add Tailscale tsnet support
Updates #1668

Co-authored-by: Will Norris <will@willnorris.com>
Signed-off-by: Brad Fitzpatrick <brad@danga.com>
2024-01-02 12:58:44 -08:00
Brad Fitzpatrick 921894783b all: gofmt for Go 1.19
Signed-off-by: Brad Fitzpatrick <brad@danga.com>
2022-12-31 22:45:40 -08:00
Tamás Gulácsi 94987079a6
runTestHarnessIntegration: Remove TODO, already implemented (#1403) 2021-08-01 22:35:48 -04:00
mpl b46cc46d5d pk-web: enable TLS-ALPN-01 for Lets Encrypt
I'm getting reminders from Let's Encrypt that perkeep.org is using a
cert obtained with TLS-SNI-01, which is now deprecated.

I think this change would be enough to switch to TLS-ALPN-01.

Change tested on the staging instance at https://staging.camlistore.net

Change-Id: I19dc51171228e6598846a778aadb8497d3406446
2019-02-23 20:00:10 +01:00
mpl 3f1b2f475d vendor: update golang.org/x/crypto
to c126467f60eb25f8f27e5a981f32a87e3965053f

For acme tls-alpn-01 support.
And enable it in perkeepd.
Remove http-01 challenge support.

Fixes #1078 (by making it obsolote)

Change-Id: I45220b6269024a2ff7a181f69cf708869705dca7
2018-07-24 04:03:16 +02:00
Brad Fitzpatrick 2606360f1d server/perkeepd: more cleanups
And move the camlistore.net code to a new file, autodns.go.

Change-Id: I7503b72c9c3f1c7a2207cab35c746c34dd90740c
2018-05-13 16:57:56 -07:00
Brad Fitzpatrick 12894d4630 all: Windows fixes (don't listen on file descriptors in test.World, etc)
test/integration: don't listen on file descriptors.
make.go: unrelated, but options to make it much faster.
internal/images: t.Skip on HEIC dependency failures

Fixes #1140
Updates golang/go#25210

Change-Id: I8092155411826d6ed1f8d85230b753d1369044af
2018-05-01 21:38:19 -07:00
Brad Fitzpatrick 57648c6b83 all: update copyright holder from Google Inc to The Perkeep Authors
The AUTHORS file is the list of copyright holders.
2018-01-03 16:52:49 -08:00
Brad Fitzpatrick d6a0b05df0 Rename import paths from camlistore.org to perkeep.org.
Part of the project renaming, issue #981.

After this, users will need to mv their $GOPATH/src/camlistore.org to
$GOPATH/src/perkeep.org. Sorry.

This doesn't yet rename the tools like camlistored, camput, camget,
camtool, etc.

Also, this only moves the lru package to internal. More will move to
internal later.

Also, this doesn't yet remove the "/pkg/" directory. That'll likely
happen later.

This updates some docs, but not all.

devcam test now passes again, even with Go 1.10 (which requires vet
checks are clean too). So a bunch of vet tests are fixed in this CL
too, and a bunch of other broken tests are now fixed (introduced from
the past week of merging the CL backlog).

Change-Id: If580db1691b5b99f8ed6195070789b1f44877dd4
2018-01-01 16:03:34 -08:00
Paul Lindner b077a5916e pkg/webserver: remove usage of the obsolete runsit package
The runsit package is obsolete.  Pull the listen code directly into webserver and
remove support for the runsit specific named ports.  Update TODO.

Change-Id: I0d8ea798375d0eb4abea86ed9e6454376233e992
2017-12-26 15:03:59 -08:00
mpl 885389a10f app/publisher: use autocert too when Camlistore does it
As the requests to the publisher are proxied through Camlistore's app
handler, there's no point in the publisher having its own autocert
Manager to request a certificate. Therefore, the publisher reuses
(readonly) camlistored's autocert CacheDir to get its certificate.

It follows that, for now, Let's Encrypt only works for the publisher if
it is running on the same host as camlistored (or more precisely, if they
share the same filesystem).

Fixes #458

Change-Id: Icf3be2913f85f9ec6f94b831ad58e1949b4d6961
2016-12-14 01:24:30 +01:00
mpl c55c8602d3 server/camlistored: use Let's Encrypt
Or to be more precise, golang.org/x/crypto/acme/autocert

The default behaviour regarding HTTPS certificates changes as such:

1) If the high-level config does not specify a certificate, the
low-level config used to be generated with a default certificate path.
This is no longer the case.
2) If the low-level config does not specify a certificate, we used to
generate self-signed ones at the default path. This is no longer always
the case. We only do this if our hostname does not look like an FQDN,
otherwise we try Let's Encrypt.
3) As a result, if the high-level config does not specify a certificate,
and the hostname looks like an FQDN, it is no longer the case that we'll
generate a self-signed. Let's Encrypt will be tried instead.

To sum up, the new rules are:
If cert/key files are specified, and found, use them.
If cert/key files are specified, not found, and the default values,
generate them (self-signed CA used as a cert), and use them.
If cert/key files are not specified, use Let's Encrypt if we have an
FQDN, otherwise generate self-signed.

Regarding cert caching:

On non-GCE, store the autocert cache dir in
osutil.CamliConfigDir()/letsencrypt.cache
On GCE, store in /tmp/camli-letsencrypt.cache

Fixes #701
Fixes #859

Change-Id: Id78a9c6f113fa93e38d690033c10a749d1844ea6
2016-12-05 19:43:37 +01:00
mpl f3f8aa6f12 pkg/webserver: force tls MinVersion to TLS1.2
Fixes #822

Change-Id: I26481f9666001ed13f9feb92e841c3b4d755a97d
2016-07-19 17:38:46 +02:00
Brad Fitzpatrick 75d60962f6 Move remaining stuff in third_party/* to vendor/*
Change-Id: Ifbcc02817083cba68d8c1acec3e6ec50e8f61149
2016-04-20 16:49:15 -07:00
Will Norris 77ed42edf8 add canonical import paths
The import path was added to the go file that included the package
documentation if one existed.  Otherwise, I used what seemed to be the
primary file for the package.

Fixes #689

Change-Id: If51be0e86529fd6f179e80af6781e639f8550fd2
2016-03-13 19:57:14 -07:00
mpl 40c0d80b07 vendor: go4.org/wkfs
Previously pkg/wkfs

Copied from go4.org at rev 3970dd22c0e7e2397304ad626a20b72f8db41b58

Note that pkg/wkfs/gcs has not been moved yet (for dependencies
reasons).

Change-Id: I737e88a360bccfc643e1fa43541f65f2384c3157
2015-11-27 00:23:03 +01:00
mpl fd2d58e61b vendor: go4.org/net/throttle
Previously known as pkg/throttle

Imported from github.com/camlistore/go4 at rev
4d87fc99168459b13e7c78086d210cf11474ade2

Change-Id: I21298db0790b2cf125efb612befd7dfb24780861
2015-11-24 16:27:11 +01:00
mpl 2e34259de8 vendor: update http2
Replaced github.com/bradfitz/http2 with the latest golang.org/x/net/http2
and updated the following dependencies:

golang.org/x/net @ a179abb31d787abdb406d922dff79635a8d784a4
golang.org/x/oauth2 @ 038cb4adce85ed41e285c2e7cc6221a92bfa44aa
google.golang.org/grpc @ 1684e4a49ba8e6c88a8628b26395f310a1e3d99f

Change-Id: Ic8f294f22a4ec7f7ee513fd1eb29a0b72ac41886
2015-11-08 11:52:24 +01:00
Brad Fitzpatrick 5744b598d4 vendor: update Google Cloud Logging, gocloud, add grpc, move http2, etc
I just wanted to update the Google Cloud Logging code (still in review
at: https://code-review.googlesource.com/#/c/2650/) but that required
updating tons of things. For instance, gocloud now always depends on
grpc, which depends on http2, which we already had in third_party, so
that now moves into vendor.

I was unable to run the tests because of some error. The error message
was useless.

But "go run make.go" compiles everything at least.
2015-09-22 14:48:04 +02:00
Brad Fitzpatrick 09114a9089 Enable HTTP/2 support.
Change-Id: I79b5cc06f502478eb8cdc1e044da97e3968bb856
2014-11-26 08:45:01 -08:00
Brad Fitzpatrick d2e964a9da webserver: use wkfs for tls cert/key
Change-Id: Ifdccd5421f54bad206b03d48bad264c3a4539e70
2014-08-08 10:58:56 -07:00
mpl b7d8d55bb3 pkg/webserver: use a custom logger, if provided
http://camlistore.org/issue/479

Change-Id: I41bd0a02d40e9ad94f655b5c1abdfa5db76c7019
2014-07-29 01:40:51 +02:00
Brad Fitzpatrick 0052ec2366 webserver: also log req.RemoteAddr in verbose mode
Change-Id: I83e2ddcb74208de61d5392a519b69d147f427c1d
2014-01-28 12:31:20 -08:00
Bill Thiede 630f6ca61f Document environment variables usage.
Running 'go run dev/envvardoc/envvardoc.go' now shows:
'All environment variables are documented'

I also took the liberty of cleaning-up our mishmash of logic for handling
boolean environment variables, and cleaned up a couple other spots that didn't
seem right.

This change adds docmentation for all variables starting with (CAM|DEV|AWS).
This leaves some variables still undocumented.  If there are variables worth
documenting in the following list, maybe we should rename them to have a
CAM{LI} prefix for consistency's sake:

APPDATA                  pkg/osutil/paths.go:86
APPDATA                  pkg/osutil/paths.go:102
DISPLAY                  pkg/misc/gpgagent/gpgagent.go:126
GOPATH                   pkg/fileembed/genfileembed/genfileembed.go:321
GOPATH                   pkg/osutil/paths.go:168
GOPATH                   pkg/test/world.go:54
GOPATH                   server/appengine/build_test.go:77
GPGKEY                   cmd/camput/init.go:77
GPG_AGENT_INFO           cmd/camput/init.go:153
GPG_AGENT_INFO           pkg/misc/gpgagent/gpgagent.go:50
HOME                     pkg/jsonsign/keys.go:79
HOME                     pkg/jsonsign/signhandler/sig.go:64
HOME                     pkg/osutil/paths.go:36
HOMEPATH                 pkg/osutil/paths.go:34
PKG_CONFIG_PATH          pkg/index/sqlite/dbschema.go:59
RUN_BROKEN_TESTS         pkg/fs/fs_test.go:67
SKIP_DEP_TESTS           pkg/test/testdep.go:29
TERM                     pkg/misc/gpgagent/gpgagent.go:133
TERM                     pkg/misc/pinentry/pinentry.go:99
TESTING_PORT_WRITE_FD    pkg/webserver/webserver.go:135
TEST_GPGAGENT_LIB        pkg/misc/gpgagent/gpgagent_test.go:27
USER                     pkg/netutil/ident.go:135
USER                     pkg/osutil/paths.go:45
USERNAME                 pkg/jsonconfig/eval.go:228
USERNAME                 pkg/osutil/paths.go:43
VERBOSE_FUSE             pkg/fs/fs_test.go:133
VERBOSE_FUSE_STDERR      pkg/fs/fs_test.go:137
XDG_CONFIG_HOME          pkg/osutil/paths.go:104

Change-Id: Ief28710d3deefd1e65247cb5d3b1d8dde73e1f2d
2013-09-06 21:54:03 -07:00
Brad Fitzpatrick 9468e5ba70 More docs. Every package is documented now.
misc.CountingReader moves into readerutil.

pkg/atomics is folded into pkg/types.

pkg/test/testdep is folded into pkg/test, with better name/docs.

Old cruft from pkg/webserver is deleted.

Change-Id: I3f72d8b29804254ef944995fb085837c878f79f5
2013-07-07 21:12:30 -07:00
mpl ee29f1f8e7 Make newui ready to dev on with closure compiler, plus misc cleanup.
Added env var in dev-server and rules in Makefile to build
all.js with the closure compiler from the js files in newui.

Misc related changes:
-fixed a few closure annotations and added a few as well.
-added missing options to dev-server usage.
-renamed CAM_HTTP_DEBUG into CAMLI_HTTP_DEBUG for consistency.

Change-Id: Ica2c12358150aec24147628201e6f7a5852f9383
2013-02-07 15:25:10 +01:00
Brad Fitzpatrick 978a68148f webserver: if env CAM_HTTP_DEBUG=1, log verbose HTTP request/response info
Change-Id: I62251a13168050b483e50ef1d8d2cb242bbd9988
2013-01-12 09:37:24 -08:00
Brad Fitzpatrick 7ca806661d Connection throttling for dev-server.
And on by default, to keep us honest.

Change-Id: Ic4c1fdfd06b54ec9108bb13c05714e9a81b642cd
2012-12-11 20:36:42 -08:00
Maxime Lavigne 5a7510c384 Modification to allow windows build and deployement
Webserver were initialized with "tcp" and ":3179" by default and
listenURL assumed that it would be treated as IPv6 and replaced [::]
by localhost. Host that were listening on IPv4 0.0.0.0 didn't get
the modification.

Receive in localdisk were using link that failed on windows plateforms.

Camlistored didn't use Json Marshaling which caused problem with the
way Windows stores its paths.

Change-Id: I9f62f7d46399c3514707383efcb2752dbaf1f420
2012-10-30 23:17:58 -04:00
Brad Fitzpatrick 6b8fc9b23a webserver: use runsit listener instead of net.Listen
Change-Id: I635d1ac5dcd89905a2fadf4b71d3bffec88389ad
2012-10-19 13:24:18 -07:00
mpl 844551d0b2 If baseURL was not user defined, this change makes sure it is
dynamically defined from incoming requests.
The listen flag has moved from pkg/webserver to server/camlistored
and is not using runsit anymore.
httputil.ServerError now only outputs the full error if we're
in internal/debug mode.
http://code.google.com/p/camlistore/issues/detail?id=53

Change-Id: I55a1c6d43a549d9ee2022742a83aa142dc19e02a
2012-10-15 16:26:50 +02:00
Brad Fitzpatrick 63418fe0fc webserver: use runsit's listen package, for running under runsit 2012-08-04 11:24:51 +10:00
Brad Fitzpatrick 7b0df24a64 maybe a windows build problem, found by gophorge
Change-Id: I21c5a17eebb1c1efce8bf12f98cab966f1ce0bc9
2012-04-24 14:41:15 -07:00
mpl 7f630657d5 generate low level configuration
Change-Id: I43d1610bdc386954dea724b4b38e184bf16e2d34
2012-03-16 00:05:24 +01:00
Brad Fitzpatrick 0714a463c9 Update from r60 to [almost] Go 1.
A lot is still broken, but most stuff at least compiles now.

The directory tree has been rearranged now too.  Go libraries are now
under "pkg".  Fully qualified, they are e.g. "camlistore.org/pkg/jsonsign".

The go tool cannot yet fetch from arbitrary domains, but discussion is
happening now on which mechanism to use to allow that.

For now, put the camlistore root under $GOPATH/src.  Typically $GOPATH
is $HOME, so Camlistore should be at $HOME/src/camlistore.org.

Then you can:

$ go build ./server/camlistored

... etc

The build.pl script is currently disabled.  It'll be resurrected at
some point, but with a very different role (helping create a fake
GOPATH and running the go build command, if things are installed at
the wrong place, and/or running fileembed generators).

Many things are certainly broken.

Many things are disabled.  (MySQL, all indexing, etc).

Many things need to be moved into
camlistore.org/third_party/{code.google.com,github.com} and updated
from their r60 to Go 1 versions, where applicable.

The GoMySQL stuff should be updated to use database/sql and the ziutek
library implementing database/sql/driver.

Help wanted.

Change-Id: If71217dc5c8f0e70dbe46e9504ca5131c6eeacde
2012-02-18 21:53:06 -08:00