mirror of https://github.com/perkeep/perkeep.git
db2604f981
The current maximum size for a schema blob is 1MB. For a large enough directory (~20000 children), the resulting static-set JSON schema is over that maximum size. We could increase that maximum, but we would eventually hit the maximum blob size (16MB), which would only allow for ~300000 children. Even if that is an uncommon size, it is technically possible to have such large directories, so I don't think it would be reasonable to restrict users to such a limit. So it does not seems like enough of a solution. The solution proposed in this CL is to spread the children of a directory (when they are more numerous than a given maximum, here set to 10000) onto several static-sets, recursively if needed. These static-sets (subsets of the whole lot of children) are stored in the new "mergeSets" field of their parent static-set schema. The actual fileRefs or dirRefs, are still stored in the "members" field of the subset they were spread in. The "mergeSets" and "members" field of a static-set are therefore mutually exclusive. Fixes #924 Change-Id: Ibe47b50795d5288fe904d3cce0cc7f780d313408 |
||
|---|---|---|
| .. | ||
| nodeattr | ||
| testdata | ||
| .gitignore | ||
| blob.go | ||
| dirreader.go | ||
| fileread_test.go | ||
| filereader.go | ||
| filewriter.go | ||
| filewriter_test.go | ||
| lookup.go | ||
| lookup_cgo.go | ||
| lookup_nocgo.go | ||
| schema.go | ||
| schema_darwin.go | ||
| schema_linux.go | ||
| schema_posix.go | ||
| schema_public_test.go | ||
| schema_test.go | ||
| sign.go | ||
| sign_test.go | ||