The os/user package nowadays says:
When cgo is available, and the required routines are implemented
in libc for a particular platform, cgo-based (libc-backed) code
is used. This can be overridden by using osusergo build tag,
which enforces the pure Go implementation.
So we don't need to work around it anymore.
Signed-off-by: Brad Fitzpatrick <brad@danga.com>
AppEngine lets you do just about anything these days. No need to cater
to the ancient limited version.
Signed-off-by: Brad Fitzpatrick <brad@danga.com>
* propagate more context cancelations from pkg/client
* convert ErrCancel to fuse.EINTR to enable clients to handle interrupts
* improve interrupt handling in fs_test.go
* fix race in mutDir between rename and first populate
The directory created by `T.TempDir()` and is automatically removed when
the test and all its subtests complete.
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
`signer.KeyId()` was only used in a counter-productive way
by comparing it with half of a long ID.
Also use this opportunity to return the full fingerprint from
ParseArmoredPublicKey. NewSigner was the only caller of this
function.
A working importer for the Instapaper service. The importer imports
bookmarks, full text articles, and highlights. A blob item handler for
Instapaper highlight permanodes is included to show summarized highlight
content from the server UI.
Also, this CL updates github.com/garyburd.com/go-auth to rev
bca2e7f09a178fd36b034107a00e2323bca6a82e in order to get support for
XAuth requests.
Fixes#1208
Change-Id: I72e0c40b245c7eec4a44bb475fcaa96a0ee9a1c5
This will allow consumers of pkg/schema to do things like verify the
signature on claims.
Fixes#1167
Change-Id: Iec76e382e18d91f82694412bc855ac5be82df825
The current maximum size for a schema blob is 1MB. For a large enough
directory (~20000 children), the resulting static-set JSON schema is
over that maximum size.
We could increase that maximum, but we would eventually hit the maximum
blob size (16MB), which would only allow for ~300000 children. Even if
that is an uncommon size, it is technically possible to have such large
directories, so I don't think it would be reasonable to restrict users
to such a limit. So it does not seems like enough of a solution.
The solution proposed in this CL is to spread the children of a
directory (when they are more numerous than a given maximum, here set to
10000) onto several static-sets, recursively if needed. These
static-sets (subsets of the whole lot of children) are stored in the new
"mergeSets" field of their parent static-set schema. The actual fileRefs
or dirRefs, are still stored in the "members" field of the subset they were
spread in. The "mergeSets" and "members" field of a static-set are therefore
mutually exclusive.
Fixes#924
Change-Id: Ibe47b50795d5288fe904d3cce0cc7f780d313408
- correct logging that logged functions instead of their value
- use ID vs Id naming
- use correct function names in comments
Change-Id: I61562cef7ebac7337ec6c85312cdf7915cb1a84b
I had intended for this to be a small change.
I was going to just add context.Context to the BlobReceiver interface,
but then I saw blob.Fetcher could also use one, so I decided to do two
in one CL.
And then it got a bit infectious and ended up touching everything.
I ended up doing SubFetch in the process by necessity.
At a certain point I finally started using context.TODO() in a few
spots, but not too many. But removing context.TODO() will come in the
future. There are more blob storage interfaces lacking context, too,
like RemoveBlobs.
Updates #733
Change-Id: Idf273180b3f8e397ac5929c6d7f520ccc5cdce08
Remove the blob.SHA{1,224}From{Bytes,String} constructors too. No
longer used. This adds blob.RefFromBytes which was missing. We had
blob.RefFromString. Now everything uses blob.RefFrom* instead of
specifying a hash function.
Some tests set a flag to force use of SHA-1 because there was too much
golden data to update. We can remove those one-by-one over time as we
fix up tests.
Updates #537
Change-Id: Ibe6428089a6221594c2b751f53f98b03b5a28dc2
Part of the project renaming, issue #981.
After this, users will need to mv their $GOPATH/src/camlistore.org to
$GOPATH/src/perkeep.org. Sorry.
This doesn't yet rename the tools like camlistored, camput, camget,
camtool, etc.
Also, this only moves the lru package to internal. More will move to
internal later.
Also, this doesn't yet remove the "/pkg/" directory. That'll likely
happen later.
This updates some docs, but not all.
devcam test now passes again, even with Go 1.10 (which requires vet
checks are clean too). So a bunch of vet tests are fixed in this CL
too, and a bunch of other broken tests are now fixed (introduced from
the past week of merging the CL backlog).
Change-Id: If580db1691b5b99f8ed6195070789b1f44877dd4
The DownloadHandler only accepted file schemas as input for building a
zip archive so far.
It can now zip directories and their contents as well.
Non-regular files (socket, fifo, symlink) are now handled too.
As previously, no compression is applied when zipping.
When the DownloadHandler's Fetcher is a caching fetcher, all the files
that are supposed to be included in the archive are read, so we can
report reading errors even before starting to create the archive. We now
also take advantage of this optimization to build a
blobRef->filepath mapping when checking the files. Then, when the zip
archive is actually being built, the file path can be looked up in the
map, instead of having to assemble it again with recursive concatenation
of directory names.
Change-Id: I853c495798a9a43e12f3386603a70560ff46a237
Uses the Google Drive API to import photos from the Google Photos folder
in Google Drive.
Still a work in progress, so not yet added to allimporters.
Issues #874#896
Change-Id: I9b9bffe071434b2a55be667579c3699a3addb8ed
It seems that signing a claim schema does not automatically set the
claim date to the signature time, which I believe should be the case.
This change adds a test to demonstrate the issue, and the fix that makes
the test pass.
Fixes#917
Change-Id: I55d6c75cbe3c3c4c1d6e5b69f6d26277d0d12728
WARNING: this app is still experimental, and even its data schema might
change. Do not use in production.
This change adds a Camlistore-based port of the scanning cabinet app
originally created by Brad Fitzpatrick:
https://github.com/bradfitz/scanningcabinet
Some of it is inspired from the App Engine Go port of Patrick Borgeest:
https://bitbucket.org/pborgeest/nometicland
The data schema is roughly as follows:
-a scan is a permanode, with the node type: "scanningcabinet:scan".
-a scan's camliContent attribute is set to the actual image file.
-a scan also holds the "dateCreated" attribute, as well as the
"document" attribute, which references the document this scan is a part
of (if any).
-a document is a permanode, with the node type: "scanningcabinet:doc".
-a document page, is modeled by the "camliPath:sha1-xxx" = "pageNumber"
relation, where sha1-xxx is the blobRef of a scan.
-a document can also hold the following attributes: "dateCreated",
"tag", "locationText", "title", "startDate", and "paymentDueDate".
Known caveats, in decreasing order of concern:
-the data schema might still change.
-the scancab tool, to actually create and upload the files from physical
documents, is practically untested (since I do not own a scanner).
-some parts, in particular related to searches, are probably
sub-optimized.
-the usual unavoidable bugs.
Change-Id: If6afc509e13f7c21164a3abd276fec075a3813bb
And add golang.org/x/crypto/openpgp instead, at rev
ede567c8e044a5913dad1d1af3696d9da953104c
As well as golang.org/x/crypto/cast5 , as a dep.
Change-Id: I385fb0cd0eef438b43fa1dcd2a36e6a1442fd3b1
Done with gopherjs and jquery.
Some build tagging added in pkg/schema and pkg/netutil because
gopherjs does not support cgo (so no os/user).
Issue #798
Change-Id: Ib1e1e94185f75cdf696aa2dd31c57fa9e3af84a1
The comment was right: a directory with too many entries depleted all my
file descriptors. I want my file descriptors back.
Change-Id: Iacdd95f06a7f5b2a2e7d4676b17eb33f86060ad2
The import path was added to the go file that included the package
documentation if one existed. Otherwise, I used what seemed to be the
primary file for the package.
Fixes#689
Change-Id: If51be0e86529fd6f179e80af6781e639f8550fd2
Most of it replaced with vendor/go4.org/types and
vendor/go4.org/readerutil
u32 went where needed in pkg/blobserver/*
invertedBool went in pkg/types/serverconfig
atomics64 went in pkg/fs
Change-Id: I230426cda35be4b45ed67e869f14e6fdae89be22
This change is in anticipation of moving pkg/images to go4.org, where it
should not depend on packages in third_party.
So:
third_party/github.com/nf/cr2 -> vendor/github.com/nf/cr2
third_party/github.com/rwcarlsen/goexif -> vendor/github.com/rwcarlsen/goexif
third_party/golang.org/x/image/tiff -> vendor/golang.org/x/image/tiff
Note that third_party/go/pkg/image/jpeg was also a dependency of
pkg/images. We had vendored image/jpeg from tip at the time because it
offered advantages over the version from Go1.3
(https://github.com/camlistore/camlistore/issues/463).
Since we now depend on Go1.5, we can go back to depend on the stdlib
version, so we simply remove third_party/go/pkg/image/jpeg and adjust
the imports accordingly.
Change-Id: Ifc8ffae0551102e644a0a0c67f3ff89e04df15c7
Previously pkg/singleflight
Imported from github.com/camlistore/go4/ at rev
039bcc4ca7e407e5611cbdf15a1f208be0bceda0
Also reimport vendor/go4.org/syncutil because some of it got split into
github.com/camlistore/go4/syncutil/syncdebug
Change-Id: Iaf48de71928d17e0410442ea6f5bef7262ba60d2
Brad Fitzpatrick