Commit Graph

6 Commits

Author SHA1 Message Date
Brad Fitzpatrick 66db09453f blobserver: add context to BlobRemover
Updates #733

Change-Id: I2fffb5cad59aa994441ee82ac5d940270113ee5a
2018-01-19 09:54:46 -08:00
Brad Fitzpatrick 194d4f9443 blobserver, all: add contexts to ReceiveBlob, Fetch & million resulting deps
I had intended for this to be a small change.

I was going to just add context.Context to the BlobReceiver interface,
but then I saw blob.Fetcher could also use one, so I decided to do two
in one CL.

And then it got a bit infectious and ended up touching everything.

I ended up doing SubFetch in the process by necessity.

At a certain point I finally started using context.TODO() in a few
spots, but not too many. But removing context.TODO() will come in the
future. There are more blob storage interfaces lacking context, too,
like RemoveBlobs.

Updates #733

Change-Id: Idf273180b3f8e397ac5929c6d7f520ccc5cdce08
2018-01-18 16:22:16 -08:00
Brad Fitzpatrick 0e8980b54b blobserver: change BlobStatter interface, simplify proxycache
This addresses a long-standing TODO in the BlobStatter interface to
clean it up. Just like all new Go programmers, I misused channels in
APIs. I should've cleaned this up years ago.

While here, I also added a context.

The rest should get contexts later.

This also cleans up a few things here & there.

The pkg/client statting no longer does batching, which added a lot of
complexity. There was a comment saying something like "once we have
SPDY, we can delete this". Well, we have HTTP/2 now, so seems
deletable.

All tests pass.

Change-Id: I034ce07d9b70e5cc9e5482213368993e638d4bc8
2018-01-08 16:54:52 -08:00
Brad Fitzpatrick 11e9c5567c Move some packages from perkeep.org/pkg to perkeep.org/internal
Notably: pkg/misc all moves.

And pkg/googlestorage is deleted, since it's not used. Only the
x/net/http2/h2demo code used to use it, but that ended in
https://go-review.googlesource.com/33230 (our vendored code is old).
So just nuke that dir for now. When it's refreshed, it'll either be
gone (dep prune) or new enough to not need googlestorage.

Also move pkg/pools, pkg/leak, and pkg/geocode to internal.

More remains.

Change-Id: I2640c4d18424062fdb8461ba451f1ce26719ae9d
2018-01-01 20:54:48 -08:00
Brad Fitzpatrick d6a0b05df0 Rename import paths from camlistore.org to perkeep.org.
Part of the project renaming, issue #981.

After this, users will need to mv their $GOPATH/src/camlistore.org to
$GOPATH/src/perkeep.org. Sorry.

This doesn't yet rename the tools like camlistored, camput, camget,
camtool, etc.

Also, this only moves the lru package to internal. More will move to
internal later.

Also, this doesn't yet remove the "/pkg/" directory. That'll likely
happen later.

This updates some docs, but not all.

devcam test now passes again, even with Go 1.10 (which requires vet
checks are clean too). So a bunch of vet tests are fixed in this CL
too, and a bunch of other broken tests are now fixed (introduced from
the past week of merging the CL backlog).

Change-Id: If580db1691b5b99f8ed6195070789b1f44877dd4
2018-01-01 16:03:34 -08:00
Govert Versluis 8548962dbe Add Azure blobserver support.
Fixes #425

Change-Id: I02bb29e6503bfef0894cbfde0c2a3304cf70c932
2017-12-29 12:39:49 -08:00