Commit Graph

34 Commits

Author SHA1 Message Date
Brad Fitzpatrick 194d4f9443 blobserver, all: add contexts to ReceiveBlob, Fetch & million resulting deps
I had intended for this to be a small change.

I was going to just add context.Context to the BlobReceiver interface,
but then I saw blob.Fetcher could also use one, so I decided to do two
in one CL.

And then it got a bit infectious and ended up touching everything.

I ended up doing SubFetch in the process by necessity.

At a certain point I finally started using context.TODO() in a few
spots, but not too many. But removing context.TODO() will come in the
future. There are more blob storage interfaces lacking context, too,
like RemoveBlobs.

Updates 

Change-Id: Idf273180b3f8e397ac5929c6d7f520ccc5cdce08
2018-01-18 16:22:16 -08:00
Brad Fitzpatrick 38f10a7bd0 all, testhooks: use sha224 by default, add hook for some tests to use sha-1
Remove the blob.SHA{1,224}From{Bytes,String} constructors too. No
longer used. This adds blob.RefFromBytes which was missing. We had
blob.RefFromString. Now everything uses blob.RefFrom* instead of
specifying a hash function.

Some tests set a flag to force use of SHA-1 because there was too much
golden data to update. We can remove those one-by-one over time as we
fix up tests.

Updates 

Change-Id: Ibe6428089a6221594c2b751f53f98b03b5a28dc2
2018-01-09 20:03:38 -08:00
Brad Fitzpatrick 57648c6b83 all: update copyright holder from Google Inc to The Perkeep Authors
The AUTHORS file is the list of copyright holders.
2018-01-03 16:52:49 -08:00
Brad Fitzpatrick c3d05cdce9 Move more packages out of pkg/ and into internal/
Moved hashutil, httputil, osutil, netutil,
images, media, magic, video, and rollsum.
2018-01-02 21:03:30 -08:00
Brad Fitzpatrick e093206a2a doc: add jsonsign protocol docs
And misc tweaks.

Updates 
Updates 

Change-Id: Ibb750c8cc69763c0a59b72f42bc53f68e3ee120f
2018-01-02 19:24:58 -08:00
Brad Fitzpatrick d6a0b05df0 Rename import paths from camlistore.org to perkeep.org.
Part of the project renaming, issue .

After this, users will need to mv their $GOPATH/src/camlistore.org to
$GOPATH/src/perkeep.org. Sorry.

This doesn't yet rename the tools like camlistored, camput, camget,
camtool, etc.

Also, this only moves the lru package to internal. More will move to
internal later.

Also, this doesn't yet remove the "/pkg/" directory. That'll likely
happen later.

This updates some docs, but not all.

devcam test now passes again, even with Go 1.10 (which requires vet
checks are clean too). So a bunch of vet tests are fixed in this CL
too, and a bunch of other broken tests are now fixed (introduced from
the past week of merging the CL backlog).

Change-Id: If580db1691b5b99f8ed6195070789b1f44877dd4
2018-01-01 16:03:34 -08:00
Paul Lindner e97eae91dc all: lint fixes for "don't use leading k in Go names"
Change-Id: I8647c1dcc0535b1fc8dc0e1b1ca4623c23d6ac7c
2017-12-13 11:31:25 -08:00
mpl 3d048ca304 vendor: rm code.google.com/p/go.crypto/openpgp
And add golang.org/x/crypto/openpgp instead, at rev
ede567c8e044a5913dad1d1af3696d9da953104c
As well as golang.org/x/crypto/cast5 , as a dep.

Change-Id: I385fb0cd0eef438b43fa1dcd2a36e6a1442fd3b1
2016-11-28 19:34:32 +01:00
Mathieu Lonjaret 7332aad3aa Merge "server/camlistored: upload gpg key blob on startup" 2016-04-25 22:43:38 +00:00
mpl 36944ca818 server/camlistored: upload gpg key blob on startup
Fixes Issue 

Change-Id: Ica92c321666220043e9c157de38e67a043250afe
2016-04-25 15:42:04 -07:00
Brad Fitzpatrick 75d60962f6 Move remaining stuff in third_party/* to vendor/*
Change-Id: Ifbcc02817083cba68d8c1acec3e6ec50e8f61149
2016-04-20 16:49:15 -07:00
Will Norris 77ed42edf8 add canonical import paths
The import path was added to the go file that included the package
documentation if one existed.  Otherwise, I used what seemed to be the
primary file for the package.

Fixes 

Change-Id: If51be0e86529fd6f179e80af6781e639f8550fd2
2016-03-13 19:57:14 -07:00
mpl 40be4d8da2 vendor: go4.org/jsonconfig (and errorutil)
Previously pkg/jsonconfig and pkg/errorutil

Copied from go4.org at rev d1b8a2fb2de6160036e4801aa5e4d855571078b8

Change-Id: I673ed55b0825baa2607289b6082f205100261d7a
2015-12-01 17:21:49 +01:00
Mathieu Lonjaret 6570d9ea94 Merge "pkg/server: decouple sign handler from UI, replace JSON maps with structs" 2015-05-25 16:52:40 +00:00
Fabian Reinartz 76a445a956 pkg/server: decouple sign handler from UI, replace JSON maps with structs
Signhandler was moved from the ui to the root handler and genconfig
adjusted accordingly.
New discovery struct types to replace the JSON maps are defined in
pkg/types/camtypes. The changes are related to Issue .

Change-Id: Ib3f12feeaf6fa81107de2d468cab491723e590ae
2015-05-25 09:28:07 +02:00
Fabian Reinartz a624d4271e pkg/jsonsign: replace JSON map with struct in verify response
This change is related to Issue .

Change-Id: I40feb09dd8a4270edd0daf6c267eed60f13280e7
2015-05-17 11:12:10 +02:00
mpl 5036d6b456 blobserver: add memory storage, mainly for tests.
Mostly code moved from pkg/test/fetcher.go

Also got rid of blob.MemoryStorage

http://camlistore.org/issue/416

Change-Id: I8d17b80942e2fde1c869f71738bfb8a67ed8ab31
2014-08-31 01:17:52 +02:00
Brad Fitzpatrick f86f9a1292 signhandler: minor style cleanups made while debugging
Change-Id: I3ea1537d0a6b277c49065053fc366e8f93e5e105
2014-08-17 13:56:34 -07:00
mpl cdc4ed5ae2 camput init: ring/key related fixes
1) Removed exec call to gpg, because it automatically looks in .gnupg/,
which we don't use anymore as a default.
2) Now taking into account global --secret-keyring flag.
This flag is now in osutil.
3) New or modified funcs in osutil
4) Made sure --gpgkey works too.
5) Cleaned up error messages and hints.

Context: http://camlistore.org/issue/364
         http://camlistore.org/issue/368

Change-Id: I2e51032ed0597da656db100d72f5588b37308e1a
2014-03-28 19:55:18 +01:00
Brad Fitzpatrick bf94a73859 Get rid of SeekFetcher vs StreamingFetcher distinction and complexity.
StreamingFetcher is now just Fetcher, and its FetchStreaming is now
just Fetch.

SeekFetcher is gone. Blobs are max 16 MB anyway, so we can slurp to
memory when needed. The main thing that cared about SeekFetcher
was the GET handler, ServeBlobref, because http.ServeContent needed
one for range requests. That's rewritten in an earlier commit, using
the FakeSeeker from another earlier commit.

Lot of code got simpler as a result.

Change-Id: Ib819413e48a8f9b8d97f596d0fbf771dab211f11
2014-03-14 12:29:13 -07:00
mpl 5a29733d3c jsonsign: clean up remnants of old secret ring default
http://camlistore.org/issue/368

Change-Id: I2708a97fdffd218f15ed9a1d482712ed7ed07197
2014-02-09 00:15:37 +01:00
Brad Fitzpatrick 0f7b7951db jsonsign: don't upload public key until the first thing is signed
Change-Id: Ibe1f69bc82caae7abf98ff9fac0def944686378d
2014-01-13 20:11:14 -08:00
Brad Fitzpatrick 0324143892 importer: make the schema.Signer available to the importer
Change-Id: Iad1a2f3bfe381b31a381957edb85d6d4aa21b38c
2013-10-24 22:45:28 -07:00
Brad Fitzpatrick 8319411ab4 Convert more ReceiveBlob into blobserver.Receive or blobserver.ReceiveNoHash
Change-Id: I9199555324b617167a6062a8b55ed09b449bae4f
2013-09-16 15:57:14 +01:00
Brad Fitzpatrick 128f56887b Handle GET and HEAD the same in more (all?) places.
Change-Id: I486a95f0a6fc127d17dcf0abeb94eab292980156
2013-09-15 20:12:26 +01:00
Brad Fitzpatrick ae20a0f7de Ditch App Engine ContextWrapper noise; add test that App Engine always compiles.
I had broken App Engine in my previous commit, since it wasn't part of 'make presubmit'

Change-Id: I32c25efbc7c348ff516128c87b6e817ae0685386
2013-08-21 18:51:10 -05:00
Brad Fitzpatrick 0bdf20884b all: delete pkg/blobref; convert all from *blobref.BlobRef to new blob.Ref
Change-Id: Id2dfb7f19452bedf4f3c9310b36227fd8117b225
2013-08-03 19:54:30 -07:00
Brad Fitzpatrick 9468e5ba70 More docs. Every package is documented now.
misc.CountingReader moves into readerutil.

pkg/atomics is folded into pkg/types.

pkg/test/testdep is folded into pkg/test, with better name/docs.

Old cruft from pkg/webserver is deleted.

Change-Id: I3f72d8b29804254ef944995fb085837c878f79f5
2013-07-07 21:12:30 -07:00
Brad Fitzpatrick ca58d8e2e0 Remove noisy var _ = log.Printf lines.
Change-Id: Ia58b8ef5f271f542ae4fe61c7fb1497322770322
2013-06-14 12:55:55 -07:00
Brad Fitzpatrick d832b6a06f Document PrefixHandler and clean up users with accessor functions.
Change-Id: Ib0b23ff3cc548a35c35eae1cdc59a8673dc19638
2013-06-12 11:17:30 +02:00
mpl f21cea5131 share: refactored
-pkg/server/share.go handles share(d) blobs.
-pkg/blobserver/gethandler no longer deals with share blobs
and has been rewritten like the other handlers (no more auth
in there).
-client and signhandler changed accordingly.
-serverconfig: auth moved up in makeCamliHandler
-share parameter added to the user level and low level config.
-share URL is now http(s)://host[:port]/[prefix/]share/sha1-xxxx.

Addresses http://camlistore.org/issue/107

Change-Id: I0a2c07ac1a1e435b141702e0ff06dc8182721d65
2013-03-06 19:11:00 +01:00
mpl 83fa932f3c signhandler: inject the claimdate (if present) in the signrequest
This allows to set a signature time (instead of using time.Now()),
so that identical files will result in identical permanodes.

Change-Id: I315c06bd30e8ac5e4d36b0e3f758483c22a31063
2013-02-02 02:48:00 +01:00
Brad Fitzpatrick 1bf01d7315 schema: lot of Blob/Builder work, and kill schema.Map.
Next up: unexport schema.Superset.

Change-Id: Ia4bb34790abba75bdb4d4101df7a4f15875f4237
2013-01-21 20:56:12 -08:00
mpl ed20da7592 vivify: support in camput and upload handler.
no special credentials/auth yet.
pkg/blobserver/handlers/get.go moved to
pkg/blobserver/gethandler/get.go to avoid
a dependency loop with the json sign helper.
pkg/server/sig.go was moved to pkg/jsonsign/signhandler
because it seemed inapproriate to import in
pkg/blobserver/handlers/upload.go something from
pkg/server

Change-Id: Ifeb14512e182e8a101d4fced6d6d4184e2b9cb99
2012-12-30 23:39:22 +01:00