No more dynamic upload URL, which trips up half our new users behind
reverse proxies when the camlistored process doesn't know its
forward-facing URL.
The original camlistore stat + upload protocol was influenced by App
Engine's limitations at the time, and some of our indecision about
where the Camlistore design is going. We understand the Camlistore
design now, and App Engine's former limitations are gone. Time to
clean things up.
More REST-y now too.
See http://camlistore.org/issue/123
Change-Id: I92c6552f830b925cef379c204a982a2213bf2f4b
Moved BasicAuth parsing and localhost detection code from pkg/auth ->
pkg/httputil for use by buildbot master.
Added user config file for remote access. The file's name is
"masterbot-config.json" and is located in osutil.CamliConfigDir(),
which on Unix will resolve to $XDG_CONFIG_HOME/camlistore/, if
XDG_CONFIG_HOME set, or ~/.config/camlistore/. On Windows it will be
under %APPDATA%\Camlistore\. The expected format is a json object
with usernames as the keys and sha1 sums of the password as the
values, i.e.:
{
"user1": "1234567890abcdef12341234567890abcdef1234",
"user2": "1234abcdef12345678901234abcdef1234567890"
}
This file is polled at a 1 minute interval and reparsed if the file's
modification time is more recent then the previous parse attempt. It
is ok for the file to go missing, it will zero out the remote user
list. A malformed file will result in the master exiting.
New commandline flags, -tlsCertFile & -tlsKeyFile, added. Specifying
both will enable TLS on the listener specified by -host. The go
source contains generate_cert.go in crypto/tls that can be used to
generate self-signed cert.pem and key.pem for testing.
Added -skiptlscheck commandline option to builder. This allows the
builder to report to https:// addresses with self-signed certs as we
don't currently have a way to specify the cert chains to be used
for TLS verification. This is a stop-gap solution.
When launching a master that listens for secure connections, we
currently need tell the builders to skip certificate validation. Add
'-builderopts="-skiptlscheck"' to the master's commandline to skip
cerfication verification.
Change-Id: I0750b5c9fa8f4def67fc05a841087b50abded2f7
On FreeBSD DevAuth was showing up as consuming ~10% of the time when doing big
camputs. It disappears with this change. FreeBSD and Mac exec external
programs for localhost auth, so I imagine they'll both benefit.
On linux, checking the password in memory is (probably) still faster than
reading from /proc (which I'm assuming turns into a roundtrip
userland->kernel->userland which is nice to avoid).
Making the change to UserPass under the assumption the performance improvement
would be similar. Untested/unprofiled though.
Change-Id: Idb7e888df6e9a36db0be671a44911e018eb7986e
but the requested operation/action too.
This allows to restrict vivify credentials to only upload
(as well as get and stat, because they're needed) to the
blobserver.
Change-Id: Idaed60d1f0d679cb9795ba9a11f094f964774335
When you go to /setup on Windows, it currently checks
the "localhostAuthorized" function that checks your
UserID with os.Getuid() and than check with the local
and remote address to see if you are allow. In Windows
os.Getuid will always equals -1 and the function default
to unallowed. On darwin, it does check for uid but
afterward only cares if the local and remote addresses
are loopback addresses.
So, I changed this function so that when the os.Getuid
is not avalaible on the platform and returns -1, it does
the same check as in darwin.
I also modified the "isLocalhost" function to use the
helper function "net.IsLoopback" instead of string matching.
Since, I already had parsed addresses and had to call
AddPairUserId, I checked if it wouldn't be simpler to
pass net.Addr directly instead of strings and reconvert
them afterward. It seemed after looking at all the code
that called this function that it simplified calls quite
a bit to do so.
Finally, I modified "netutil.Localhost" for it to return
the first IP it finds instead of the string representation
of the first IP enclosed in square brackets. Since the square
brackets around the IP are only necessary in a TCPAddr, it
would be simplier and more robust for the user to directly
print TCPAddr instead of appending this string to a post.
Change-Id: Id79de6bebd6380f877074211c0d260782058765f
A lot is still broken, but most stuff at least compiles now.
The directory tree has been rearranged now too. Go libraries are now
under "pkg". Fully qualified, they are e.g. "camlistore.org/pkg/jsonsign".
The go tool cannot yet fetch from arbitrary domains, but discussion is
happening now on which mechanism to use to allow that.
For now, put the camlistore root under $GOPATH/src. Typically $GOPATH
is $HOME, so Camlistore should be at $HOME/src/camlistore.org.
Then you can:
$ go build ./server/camlistored
... etc
The build.pl script is currently disabled. It'll be resurrected at
some point, but with a very different role (helping create a fake
GOPATH and running the go build command, if things are installed at
the wrong place, and/or running fileembed generators).
Many things are certainly broken.
Many things are disabled. (MySQL, all indexing, etc).
Many things need to be moved into
camlistore.org/third_party/{code.google.com,github.com} and updated
from their r60 to Go 1 versions, where applicable.
The GoMySQL stuff should be updated to use database/sql and the ziutek
library implementing database/sql/driver.
Help wanted.
Change-Id: If71217dc5c8f0e70dbe46e9504ca5131c6eeacde
Brad Fitzpatrick