StreamingFetcher is now just Fetcher, and its FetchStreaming is now
just Fetch.
SeekFetcher is gone. Blobs are max 16 MB anyway, so we can slurp to
memory when needed. The main thing that cared about SeekFetcher
was the GET handler, ServeBlobref, because http.ServeContent needed
one for range requests. That's rewritten in an earlier commit, using
the FakeSeeker from another earlier commit.
Lot of code got simpler as a result.
Change-Id: Ib819413e48a8f9b8d97f596d0fbf771dab211f11
Add blobserver.EnumerateAllFrom, make S3 client API match the underlying S3 protocol
with marker instead of after, and then push after->marker conversion logic up
into the s3 enumerate code that's using the S3 client.
Change-Id: I034a7c1c8af441881ebba74bcb523bd690cd16d3
Not just in blob.SizedRef, but in blobserver.Fetch and
blobserver.FetchStreaming, too.
Blobs have a max size of 10-32 MB anyway, and the index.Corpus is now using
uint32 to save memory.
Change-Id: I1172445c2f9463fdaee55bfe0f1218d44be4aa53
Will eventually be plumbed through lots of APIs, especially those requiring or benefiting from
cancelation notification and/or those needing access to the HTTP context (e.g. App Engine).
Change-Id: I591496725d620126e09d49eb07cade7707c7fc64
Move up a layer to the HTTP. Also, start to remove ContextWrapper
stuff. We've done it differently for App Engine instead, and will do
it differently yet moving forward.
Also add blobserver.Receive and use it in most places, moving checksum
verification up a layer.
Bunch of other cleanup and TODO fixing too.
Much simpler and cleaner.
Change-Id: I12e56c5d4e53bfcf82bdd8fb0b6d57c248ff605c
A lot is still broken, but most stuff at least compiles now.
The directory tree has been rearranged now too. Go libraries are now
under "pkg". Fully qualified, they are e.g. "camlistore.org/pkg/jsonsign".
The go tool cannot yet fetch from arbitrary domains, but discussion is
happening now on which mechanism to use to allow that.
For now, put the camlistore root under $GOPATH/src. Typically $GOPATH
is $HOME, so Camlistore should be at $HOME/src/camlistore.org.
Then you can:
$ go build ./server/camlistored
... etc
The build.pl script is currently disabled. It'll be resurrected at
some point, but with a very different role (helping create a fake
GOPATH and running the go build command, if things are installed at
the wrong place, and/or running fileembed generators).
Many things are certainly broken.
Many things are disabled. (MySQL, all indexing, etc).
Many things need to be moved into
camlistore.org/third_party/{code.google.com,github.com} and updated
from their r60 to Go 1 versions, where applicable.
The GoMySQL stuff should be updated to use database/sql and the ziutek
library implementing database/sql/driver.
Help wanted.
Change-Id: If71217dc5c8f0e70dbe46e9504ca5131c6eeacde
mpl