Commit Graph

31 Commits

Author SHA1 Message Date
Brad Fitzpatrick c71b122e12 httputil: simplify the SystemCARootsAvailable function
Change-Id: I88d36cdc2384634a6baa49b0d663d87a5d7fcb52
2014-08-15 17:43:22 -07:00
Brad Fitzpatrick 2ee36e4f1a Embed Mozilla's Root CA set for use in Docker environments without CA files.
Thanks to Adam Langley and Kelsey Hightower for pointers:

https://github.com/kelseyhightower/contributors/blob/master/certs.go
https://medium.com/@kelseyhightower/optimizing-docker-images-for-static-binaries-b5696e26eb07
https://github.com/agl/extract-nss-root-certs

Change-Id: I440cedf476446da852c409958a877fbcdb464340
2014-08-15 17:15:09 -07:00
mpl 1fd526b999 test/transport.go: move to httputil, so we can use it without linking testing
Change-Id: I889d167a78ef0fba5ef2e9e3e884f594e151e940
2014-07-14 18:33:10 +02:00
mpl a8f8fd57cf httputil: allow full error serving when with devcam
When hacking on e.g. importers, I sometimes want to use devcam with
-hostname, to test with a non local context. In which case, when hitting
an error httputil.ServeError would not print the full error, which
makes it harder to debug.

Change-Id: I2fb8c97748d4f077404fbf12010addd6e7af3572
2014-04-23 15:19:55 +02:00
Brad Fitzpatrick c179081ab7 httputil: add IsWebsocketUpgrade, fix BadRequestError inconsistency
Change-Id: I85e34fe4e87c2402fe6ab002bcd374ef0fd41141
2014-02-03 20:24:45 -05:00
Brad Fitzpatrick 2fde1c3632 httputil: make CloseBody more robust
Change-Id: If46c60a5088f57ada45b8aa91ae29c8f512604c7
2014-01-29 11:31:52 +01:00
Brad Fitzpatrick 7c3afd2711 httputil: add test for CloseBody
Change-Id: I6049de69279aef7bb0c013935ae4d37b8eaff853
2014-01-28 23:14:02 -08:00
Brad Fitzpatrick eb27581da7 httputil: add DecodeJSON and CloseBody helpers, then use them everywhere
httputil.CloseBody closes a http.Response.Body nicely, such that it's
more likely to be re-used.

httputil.DecodeJSON parses JSON from a Response.Body and closes the
body. It also keeps a temporary copy of the input, for error messages.

Then use them everywhere, to improve HTTP TCP connection re-use.

And misc other clean-ups.

Change-Id: I5dc44be0165b13659459ed29ce5ab44c17d9739f
2014-01-28 22:00:52 -08:00
Brad Fitzpatrick 60685a1194 auth: warn when local connection uid doesn't match
Change-Id: Icdcef55c4831b4f77f7df34e58c87a6985401a04
2013-12-16 17:19:31 -08:00
Mathieu Lonjaret f8348c5875 Merge "buildbot/master: add Basic Auth support." 2013-11-22 15:29:06 +00:00
Bill Thiede 738783ebf4 buildbot/master: add Basic Auth support.
Moved BasicAuth parsing and localhost detection code from pkg/auth ->
pkg/httputil for use by buildbot master.

Added user config file for remote access.  The file's name is
"masterbot-config.json" and is located in osutil.CamliConfigDir(),
which on Unix will resolve to $XDG_CONFIG_HOME/camlistore/, if
XDG_CONFIG_HOME set, or ~/.config/camlistore/.  On Windows it will be
under %APPDATA%\Camlistore\.  The expected format is a json object
with usernames as the keys and sha1 sums of the password as the
values, i.e.:

    {
        "user1": "1234567890abcdef12341234567890abcdef1234",
        "user2": "1234abcdef12345678901234abcdef1234567890"
    }

This file is polled at a 1 minute interval and reparsed if the file's
modification time is more recent then the previous parse attempt.  It
is ok for the file to go missing, it will zero out the remote user
list. A malformed file will result in the master exiting.

New commandline flags, -tlsCertFile & -tlsKeyFile, added.  Specifying
both will enable TLS on the listener specified by -host.  The go
source contains generate_cert.go in crypto/tls that can be used to
generate self-signed cert.pem and key.pem for testing.

Added -skiptlscheck commandline option to builder.  This allows the
builder to report to https:// addresses with self-signed certs as we
don't currently have a way to specify the cert chains to be used
for TLS verification.  This is a stop-gap solution.

When launching a master that listens for secure connections, we
currently need tell the builders to skip certificate validation. Add
'-builderopts="-skiptlscheck"' to the master's commandline to skip
cerfication verification.

Change-Id: I0750b5c9fa8f4def67fc05a841087b50abded2f7
2013-11-21 21:22:12 -08:00
Robert Obryk 68d6af1e82 httputil: set the headers before writing them
Change-Id: Ibb7db1ca06744b64ec29664059364fe7a1e8e837
2013-11-08 02:06:27 +01:00
Brad Fitzpatrick 128f56887b Handle GET and HEAD the same in more (all?) places.
Change-Id: I486a95f0a6fc127d17dcf0abeb94eab292980156
2013-09-15 20:12:26 +01:00
Brad Fitzpatrick 0bdf20884b all: delete pkg/blobref; convert all from *blobref.BlobRef to new blob.Ref
Change-Id: Id2dfb7f19452bedf4f3c9310b36227fd8117b225
2013-08-03 19:54:30 -07:00
Brad Fitzpatrick 4a404a3c60 search: clean up describe handler and DescribeRequest
Change-Id: I4e6184765b6139de3bdc153d27a12d3cb205a55e
2013-07-11 10:35:06 +10:00
Brad Fitzpatrick cf0d9aca6e More docs
Change-Id: I5c21f240c85bcf91fb67487cc172bf3faeb49fff
2013-07-07 18:52:14 -07:00
Brad Fitzpatrick d832b6a06f Document PrefixHandler and clean up users with accessor functions.
Change-Id: Ib0b23ff3cc548a35c35eae1cdc59a8673dc19638
2013-06-12 11:17:30 +02:00
Brad Fitzpatrick 4c7e985c14 uploadhelper: use a type for JSON response, not maps.
Change-Id: I642af4daf97829a813bf7dad61cc7683ec780e69
2013-02-11 20:33:53 -08:00
Brad Fitzpatrick 7300184066 httputil, search: more HTTP util usage, deleting code.
Change-Id: I90e51a3e8b3fdd868b26a40446f9715aaa7f89c0
2013-02-08 22:21:09 -08:00
Brad Fitzpatrick e624e24c46 httputil/handler: start of more formal approach to input validation and HTTP error handling
Change-Id: I219a6263e1ba2d0e64819c01a885078dbf1c8b4c
2013-02-08 19:04:24 -08:00
Brad Fitzpatrick c408e19ec4 httputil: more Transport timing stats
Change-Id: I536d8f549e6cdbc0778c44943e770e4f0665c868
2013-01-06 09:03:47 -08:00
Brad Fitzpatrick 056d1efb26 httputil: more Transport debugging
Change-Id: If3ea30237abf8614d41bb340c24a65f7725234c2
2013-01-05 17:17:56 -08:00
Brad Fitzpatrick 9f6776dddb camget, httputil: more verbose HTTP debugging
Change-Id: I2fc1a1fe0fff99202245d391a8a76e9fab07adc2
2013-01-05 16:44:34 -08:00
mpl 39218e3cd5 Merge "Changed auth to take into account not only the credentials, but the requested operation/action too." 2013-01-04 21:09:37 +00:00
mpl 12213c058e Changed auth to take into account not only the credentials,
but the requested operation/action too.

This allows to restrict vivify credentials to only upload
(as well as get and stat, because they're needed) to the
blobserver.

Change-Id: Idaed60d1f0d679cb9795ba9a11f094f964774335
2013-01-04 22:17:12 +01:00
Brad Fitzpatrick fe13c7c712 move camput's statsTransport to httputil.StatsTransport, use it in camget too.
Change-Id: Iab48e32ccb8a99b8ec6b058061cff9a5b99e8712
2013-01-02 20:32:13 -08:00
Maxime Lavigne 3ec58472db Modifications allowing /setup to be view on Windows
When you go to /setup on Windows, it currently checks
the "localhostAuthorized" function that checks your
UserID with os.Getuid() and than check with the local
and remote address to see if you are allow. In Windows
os.Getuid will always equals -1 and the function default
to unallowed. On darwin, it does check for uid but
afterward only cares if the local and remote addresses
are loopback addresses.

So, I changed this function so that when the os.Getuid
is not avalaible on the platform and returns -1, it does
the same check as in darwin.

I also modified the "isLocalhost" function to use the
helper function "net.IsLoopback" instead of string matching.

Since, I already had parsed addresses and had to call
AddPairUserId, I checked if it wouldn't be simpler to
pass net.Addr directly instead of strings and reconvert
them afterward. It seemed after looking at all the code
that called this function that it simplified calls quite
a bit to do so.

Finally, I modified "netutil.Localhost" for it to return
the first IP it finds instead of the string representation
of the first IP enclosed in square brackets. Since the square
brackets around the IP are only necessary in a TCPAddr, it
would be simplier and more robust for the user to directly
print TCPAddr instead of appending this string to a post.

Change-Id: Id79de6bebd6380f877074211c0d260782058765f
2012-11-18 20:20:25 -05:00
Brad Fitzpatrick 546e2fc14d Send Content-Length on JSON responses
Change-Id: I7e8034b0aea8031011607527f237e24d63b88502
2012-11-04 13:50:35 +01:00
mpl 844551d0b2 If baseURL was not user defined, this change makes sure it is
dynamically defined from incoming requests.
The listen flag has moved from pkg/webserver to server/camlistored
and is not using runsit anymore.
httputil.ServerError now only outputs the full error if we're
in internal/debug mode.
http://code.google.com/p/camlistore/issues/detail?id=53

Change-Id: I55a1c6d43a549d9ee2022742a83aa142dc19e02a
2012-10-15 16:26:50 +02:00
Brad Fitzpatrick 8c293e34b6 Per Go initialism style, rename Json to JSON 2012-07-28 15:42:56 -07:00
Brad Fitzpatrick 0714a463c9 Update from r60 to [almost] Go 1.
A lot is still broken, but most stuff at least compiles now.

The directory tree has been rearranged now too.  Go libraries are now
under "pkg".  Fully qualified, they are e.g. "camlistore.org/pkg/jsonsign".

The go tool cannot yet fetch from arbitrary domains, but discussion is
happening now on which mechanism to use to allow that.

For now, put the camlistore root under $GOPATH/src.  Typically $GOPATH
is $HOME, so Camlistore should be at $HOME/src/camlistore.org.

Then you can:

$ go build ./server/camlistored

... etc

The build.pl script is currently disabled.  It'll be resurrected at
some point, but with a very different role (helping create a fake
GOPATH and running the go build command, if things are installed at
the wrong place, and/or running fileembed generators).

Many things are certainly broken.

Many things are disabled.  (MySQL, all indexing, etc).

Many things need to be moved into
camlistore.org/third_party/{code.google.com,github.com} and updated
from their r60 to Go 1 versions, where applicable.

The GoMySQL stuff should be updated to use database/sql and the ziutek
library implementing database/sql/driver.

Help wanted.

Change-Id: If71217dc5c8f0e70dbe46e9504ca5131c6eeacde
2012-02-18 21:53:06 -08:00