In addition to API changes, the V2 logging API uses
google.golang.org/grpc, which does not rely directly on net/http. This
is a problem for us because, among other things, when needed (on
CoreOS), we insert the system Root CAs in the TLSConfig of the
default http client through httputil.InstallCerts, which has no effect
on google.golang.org/grpc calls. Therefore, we switch to
github.com/bradfitz/grpc-go instead, which does rely on net/http.
And since GRPC requires HTTP/2, we also now need to manually configure
the default transport (since we're already modifying it) to be HTTP/2
enabled.
Related dependencies updated/added:
updated:
cloud.google.com/go/logging
a64eb5d53f434cc548fd6b2898b5a3e8a0c31b78
github.com/golang/protobuf
8ee79997227bf9b34611aee7946ae64735e6fd93
added:
github.com/bradfitz/grpc-go
188a132adcfba339f1f2d5da52498451341f9ee8
as a replacement for google.golang.org/grpc
github.com/googleapis/gax-go
da06d194a00e19ce00d9011a13931c3f6f6887c7
google.golang.org/genproto/googleapis/logging
08f135d1a31b6ba454287638a3ce23a55adace6f
google.golang.org/genproto/googleapis/datastore
08f135d1a31b6ba454287638a3ce23a55adace6f
google.golang.org/api/support/bundler
e4c04685e5d7db47ff294aa9e514b3a638c431c9
google.golang.org/genproto/googleapis/rpc/status
08f135d1a31b6ba454287638a3ce23a55adace6f
Fixes#873
Change-Id: I3ba14c94122dd9f1b88ef61c2dc26430690d28a9
Something changed in x509 in Go1.6 so that when verifying an empty Cert,
one now gets the errNotParsed (x509: missing ASN.1 contents; use
ParseCertificate) error, instead of the SystemRootsError.
Since we rely on getting the SystemRootsError to find out that we're on
CoreOS, we now need to verify a non empty cert, when performing this
check.
Fixes issue #705
Change-Id: I40a4f9aa4ef49bbdd3b7b15b127f890e26f31de4
The import path was added to the go file that included the package
documentation if one existed. Otherwise, I used what seemed to be the
primary file for the package.
Fixes#689
Change-Id: If51be0e86529fd6f179e80af6781e639f8550fd2
Most of it replaced with vendor/go4.org/types and
vendor/go4.org/readerutil
u32 went where needed in pkg/blobserver/*
invertedBool went in pkg/types/serverconfig
atomics64 went in pkg/fs
Change-Id: I230426cda35be4b45ed67e869f14e6fdae89be22
At rev c3c16077b235db4c6b2a2a6121c2d429707f4ed4
And use it in new package pkg/camlegal
And remove pkg/legal
Change-Id: I5e398aa5f145a02a04d33719276fd2f8b82ab12b
Previously pkg/wkfs
Copied from go4.org at rev 3970dd22c0e7e2397304ad626a20b72f8db41b58
Note that pkg/wkfs/gcs has not been moved yet (for dependencies
reasons).
Change-Id: I737e88a360bccfc643e1fa43541f65f2384c3157
We redefine a MultipartReader(*http.Request) function to use instead of
req.MultipartReader, so we can get a (bugfixed) multipart Reader from
our vendored mime/multipart, instead of the buggy one from the stdlib's.
Fixes issue #642
Change-Id: I6a205bff915632d4ee77547e6e26bc0af99665e9
This change makes gce/create.go generate a self-signed certificate
with the hostname from the -hostname argument and upload it to GCS
before creating a new camlistore instance.
It also makes camlistored use baseURL to figure out the hostname
when generating its self-signed certificate.
Change-Id: I64f85853dab34a7ce95e5d5997e58f2e5da43496
Add test that imports three posts. Fix bug the test uncovered. We need
to make a copy of the post in the importBatch loop before handing it off
to asynchronous importPost call.
Change-Id: Id8dea72b03dd3666dde97e977c921a8955e67392
When hacking on e.g. importers, I sometimes want to use devcam with
-hostname, to test with a non local context. In which case, when hitting
an error httputil.ServeError would not print the full error, which
makes it harder to debug.
Change-Id: I2fb8c97748d4f077404fbf12010addd6e7af3572
httputil.CloseBody closes a http.Response.Body nicely, such that it's
more likely to be re-used.
httputil.DecodeJSON parses JSON from a Response.Body and closes the
body. It also keeps a temporary copy of the input, for error messages.
Then use them everywhere, to improve HTTP TCP connection re-use.
And misc other clean-ups.
Change-Id: I5dc44be0165b13659459ed29ce5ab44c17d9739f
Moved BasicAuth parsing and localhost detection code from pkg/auth ->
pkg/httputil for use by buildbot master.
Added user config file for remote access. The file's name is
"masterbot-config.json" and is located in osutil.CamliConfigDir(),
which on Unix will resolve to $XDG_CONFIG_HOME/camlistore/, if
XDG_CONFIG_HOME set, or ~/.config/camlistore/. On Windows it will be
under %APPDATA%\Camlistore\. The expected format is a json object
with usernames as the keys and sha1 sums of the password as the
values, i.e.:
{
"user1": "1234567890abcdef12341234567890abcdef1234",
"user2": "1234abcdef12345678901234abcdef1234567890"
}
This file is polled at a 1 minute interval and reparsed if the file's
modification time is more recent then the previous parse attempt. It
is ok for the file to go missing, it will zero out the remote user
list. A malformed file will result in the master exiting.
New commandline flags, -tlsCertFile & -tlsKeyFile, added. Specifying
both will enable TLS on the listener specified by -host. The go
source contains generate_cert.go in crypto/tls that can be used to
generate self-signed cert.pem and key.pem for testing.
Added -skiptlscheck commandline option to builder. This allows the
builder to report to https:// addresses with self-signed certs as we
don't currently have a way to specify the cert chains to be used
for TLS verification. This is a stop-gap solution.
When launching a master that listens for secure connections, we
currently need tell the builders to skip certificate validation. Add
'-builderopts="-skiptlscheck"' to the master's commandline to skip
cerfication verification.
Change-Id: I0750b5c9fa8f4def67fc05a841087b50abded2f7
but the requested operation/action too.
This allows to restrict vivify credentials to only upload
(as well as get and stat, because they're needed) to the
blobserver.
Change-Id: Idaed60d1f0d679cb9795ba9a11f094f964774335
When you go to /setup on Windows, it currently checks
the "localhostAuthorized" function that checks your
UserID with os.Getuid() and than check with the local
and remote address to see if you are allow. In Windows
os.Getuid will always equals -1 and the function default
to unallowed. On darwin, it does check for uid but
afterward only cares if the local and remote addresses
are loopback addresses.
So, I changed this function so that when the os.Getuid
is not avalaible on the platform and returns -1, it does
the same check as in darwin.
I also modified the "isLocalhost" function to use the
helper function "net.IsLoopback" instead of string matching.
Since, I already had parsed addresses and had to call
AddPairUserId, I checked if it wouldn't be simpler to
pass net.Addr directly instead of strings and reconvert
them afterward. It seemed after looking at all the code
that called this function that it simplified calls quite
a bit to do so.
Finally, I modified "netutil.Localhost" for it to return
the first IP it finds instead of the string representation
of the first IP enclosed in square brackets. Since the square
brackets around the IP are only necessary in a TCPAddr, it
would be simplier and more robust for the user to directly
print TCPAddr instead of appending this string to a post.
Change-Id: Id79de6bebd6380f877074211c0d260782058765f
dynamically defined from incoming requests.
The listen flag has moved from pkg/webserver to server/camlistored
and is not using runsit anymore.
httputil.ServerError now only outputs the full error if we're
in internal/debug mode.
http://code.google.com/p/camlistore/issues/detail?id=53
Change-Id: I55a1c6d43a549d9ee2022742a83aa142dc19e02a
A lot is still broken, but most stuff at least compiles now.
The directory tree has been rearranged now too. Go libraries are now
under "pkg". Fully qualified, they are e.g. "camlistore.org/pkg/jsonsign".
The go tool cannot yet fetch from arbitrary domains, but discussion is
happening now on which mechanism to use to allow that.
For now, put the camlistore root under $GOPATH/src. Typically $GOPATH
is $HOME, so Camlistore should be at $HOME/src/camlistore.org.
Then you can:
$ go build ./server/camlistored
... etc
The build.pl script is currently disabled. It'll be resurrected at
some point, but with a very different role (helping create a fake
GOPATH and running the go build command, if things are installed at
the wrong place, and/or running fileembed generators).
Many things are certainly broken.
Many things are disabled. (MySQL, all indexing, etc).
Many things need to be moved into
camlistore.org/third_party/{code.google.com,github.com} and updated
from their r60 to Go 1 versions, where applicable.
The GoMySQL stuff should be updated to use database/sql and the ziutek
library implementing database/sql/driver.
Help wanted.
Change-Id: If71217dc5c8f0e70dbe46e9504ca5131c6eeacde
mpl