StreamingFetcher is now just Fetcher, and its FetchStreaming is now
just Fetch.
SeekFetcher is gone. Blobs are max 16 MB anyway, so we can slurp to
memory when needed. The main thing that cared about SeekFetcher
was the GET handler, ServeBlobref, because http.ServeContent needed
one for range requests. That's rewritten in an earlier commit, using
the FakeSeeker from another earlier commit.
Lot of code got simpler as a result.
Change-Id: Ib819413e48a8f9b8d97f596d0fbf771dab211f11
NewFakeSeeker returns a ReadSeeker that can pretend to Seek (based
on the provided total size of the reader's content), but any reads
will fail if the fake seek position doesn't match reality.
Change-Id: I4e63aa78c80fdbcd87b2133018009ed72bedb9f3
Allows efficiently finding parents of permanodes, or permanodes of
file contents, etc.
Adds Corpus.ForeachClaimBackLocked.
Change-Id: I6dffb32eab4a959cd3c5922a7f47d11fdcea5f3d
By default it will use cznic/kv. But if you want to use something else
(like MySQL, or leveldb, or Postgres, or Mongo), you can.
Change-Id: I8ce3571a701717ffde3b80856c72a9e3223ab439
This patch adds support for restoring symbolic links when camget is
called with the -o option.
This fixes an issue which caused `camget -o' to write an incomplete
snapshot to the target directory when it encountered a symlink.
Note that no attempt is made to restore the symlink metadata since: a)
A symlink's permissions do not matter and are fixed to 1777 on some
systems b) Go's os.Chtimes() always acts on the symlink target c)
os.Chown() is not currently called for any file in the current code.
Tested on Linux.
Change-Id: I7099592dcd5cc513c2685671be6f6b165f8f8053
Currently controlled by an environment variable, but will become
a config option + on-demand button in UI in later commits.
Change-Id: I25fa878c9b30cdd713e2859585210eb722092f7b
Fix deadlock, much better status page, show per-blob status & errors,
clear errors when they've resolved themselves, fix known data race.
Change-Id: I968de0de4f308ff0a410adceb181a0712800d401
Add blobserver.EnumerateAllFrom, make S3 client API match the underlying S3 protocol
with marker instead of after, and then push after->marker conversion logic up
into the s3 enumerate code that's using the S3 client.
Change-Id: I034a7c1c8af441881ebba74bcb523bd690cd16d3
We still can't find their width/height or thumbnail the, but at least
we don't wedge the UI now.
Fixes camlistore.org/issue/389
Change-Id: I8af6cefa5d84e7f6e26f3920e4dd6e1d5eb8b192
Because of Go's rules regaring closures (function literals) and bound
variables, in the case in which there were multiple patterns in the
ignoredFiles configuration variable, the entire slice (vector) of
registered matching functions was being called with the
last-configured ignoredFiles pattern as the first variable. This meant
that ignoredFiles did not work if it contained more than one pattern
and in certain circumstances, such a configuration would lead to a
crash.
Change-Id: I0b991aa74d079d4ce27fc0ba373e2ee2cf9eb772
The handler was adding 1 to things to see if it was at the end, but that was causing
HTTP requests to Amazon like: limit 1000, limit 1, limit 1000, limit 1, etc.
This makes it twice as fast.
Change-Id: Ibb7e3f6ae7229a21c87817c7438324d36e7b491a
Brad Fitzpatrick