When fetching shared blobs, we rely on the share chain to verify if a
blob can be reached. This chain is updated whenever we fetch an
additional link of the chain, by updating the Client.via map. However,
when some blobs of the chain are already cached in camget's DiskCache,
because we get them from the cache, we don't fetch them with
Client.FetchVia, which means the Client.via map isn't updated. And thus
the chain is broken.
This change adds Client.UpdateShareChain, and sets it as a hook to be
called by the CachingFetcher in the event of a cache hit. That way, we
ensure that the share chain is updated even when we get blobs from the
cache (instead of from the Client).
We also add a mutex to guard Client.via, because it is accessed by
concurrent smartFetch calls in case of a static-set.
As FetchVia was undocumented and not used by anyone, I made it
unexported. We can always export it again later when needed.
Fixes#856
Change-Id: I767cbec4b6f382cbccc25c0b97782b2a7472deb8
The import path was added to the go file that included the package
documentation if one existed. Otherwise, I used what seemed to be the
primary file for the package.
Fixes#689
Change-Id: If51be0e86529fd6f179e80af6781e639f8550fd2
Most of it replaced with vendor/go4.org/types and
vendor/go4.org/readerutil
u32 went where needed in pkg/blobserver/*
invertedBool went in pkg/types/serverconfig
atomics64 went in pkg/fs
Change-Id: I230426cda35be4b45ed67e869f14e6fdae89be22
Also add a flag to camtool discovery:
$ camtool discovery -httpversion
HTTP/2.0
Fixescamlistore/camlistore#661
Change-Id: I6db392e01bd176a2757ebdb4a0e881eebaec9b03
The conflict was introduced in cea5e332ce
by linking pkg/cmdmain in, which introduced its public flags.
Change-Id: I6374c2fb1657f6fd534c1342a5947ef6db16a3e2
At rev c3c16077b235db4c6b2a2a6121c2d429707f4ed4
And use it in new package pkg/camlegal
And remove pkg/legal
Change-Id: I5e398aa5f145a02a04d33719276fd2f8b82ab12b
Also fix the other bug that was making the first bug only visible when
verbose :-)
http://camlistore.org/issue/526
Change-Id: Ia33bdc73a0eab4995f79abd7e9a37f08d3391f06
This patch:
1. Introduces a JSON schema for unix domain sockets (socket files).
2. Adds support for the new schema to pkg/schema.
3. Adds support for uploading and restoring sockets to camput and
camget respectively.
Depends on CL 2706 (https://camlistore-review.googlesource.com/2706/)
Change-Id: Ib0d6e9f9ebd348f55efee5a08cd774f3969d0760
This patch:
1. Introduces a JSON schema for FIFOs (named pipes).
2. Adds support for uploading and restoring FIFOs to camput and camget
respectively.
It includes tests for the basic schema support in pkg/schema and
integration tests for camput and camget.
This CL does not add FIFO support to the HTTP server and fs (cammount
support) code. Those will be addressed in a subsequent patch.
Change-Id: Iab926073c36e653d25e693a559178bea9bf19b9a
camget runs out of FDs when fetching the contents of a directory with
many entries. Close the connections to the server more aggressively
to fix this.
Change-Id: Iabef20481d3ec12cc5f15e75edc01d0e1e883486
StreamingFetcher is now just Fetcher, and its FetchStreaming is now
just Fetch.
SeekFetcher is gone. Blobs are max 16 MB anyway, so we can slurp to
memory when needed. The main thing that cared about SeekFetcher
was the GET handler, ServeBlobref, because http.ServeContent needed
one for range requests. That's rewritten in an earlier commit, using
the FakeSeeker from another earlier commit.
Lot of code got simpler as a result.
Change-Id: Ib819413e48a8f9b8d97f596d0fbf771dab211f11
This patch adds support for restoring symbolic links when camget is
called with the -o option.
This fixes an issue which caused `camget -o' to write an incomplete
snapshot to the target directory when it encountered a symlink.
Note that no attempt is made to restore the symlink metadata since: a)
A symlink's permissions do not matter and are fixed to 1777 on some
systems b) Go's os.Chtimes() always acts on the symlink target c)
os.Chown() is not currently called for any file in the current code.
Tested on Linux.
Change-Id: I7099592dcd5cc513c2685671be6f6b165f8f8053
-added some doc.go for each command to make /cmd/*
work nicely.
-hacked a bit of style, mostly borrowed from golang too.
Change-Id: I011d70bca2c0151a29dfd53c78a896821730d5fb
a simple localhost test with only 4.4 MB and 4 files drops from 350 HTTP requests to 76.
more work to be done, but this is better.
Change-Id: I66ed67fe95ddd30aeefe742395874b2eabc237c7
A lot is still broken, but most stuff at least compiles now.
The directory tree has been rearranged now too. Go libraries are now
under "pkg". Fully qualified, they are e.g. "camlistore.org/pkg/jsonsign".
The go tool cannot yet fetch from arbitrary domains, but discussion is
happening now on which mechanism to use to allow that.
For now, put the camlistore root under $GOPATH/src. Typically $GOPATH
is $HOME, so Camlistore should be at $HOME/src/camlistore.org.
Then you can:
$ go build ./server/camlistored
... etc
The build.pl script is currently disabled. It'll be resurrected at
some point, but with a very different role (helping create a fake
GOPATH and running the go build command, if things are installed at
the wrong place, and/or running fileembed generators).
Many things are certainly broken.
Many things are disabled. (MySQL, all indexing, etc).
Many things need to be moved into
camlistore.org/third_party/{code.google.com,github.com} and updated
from their r60 to Go 1 versions, where applicable.
The GoMySQL stuff should be updated to use database/sql and the ziutek
library implementing database/sql/driver.
Help wanted.
Change-Id: If71217dc5c8f0e70dbe46e9504ca5131c6eeacde
mpl