They were internal packages (under pkg), which we are now moving to
go4.org, so we in turn need to vendor them in now.
Change-Id: I92224f731404d0bd4ca1c57492bed37cb3367ed4
Allows to upload only the contents of a file, without any of its
attributes, i.e. the file JSON schema on top only has its camlitype,
and the file's parts, set.
Issue #42
Change-Id: I1d0ab8d7cff0f0350439553a37b618b941067c93
I can recreate the file, set the modification time, but can't forge creation
time, thus cannot recreate the exact same uploaded file twice.
This change targets this problem.
As this is a very rarely used flag, this is enabled only if the CAMLI_DEBUG
environment variable is true - such as the other "gross" options
(statcache, sqlitecache...).
Change-Id: I99c7805898ed97883043de4e7dcec23a472e462c
The upload was constrained to have only 5 workers goroutines at a
time, but those goroutines could have been uploading directories and
blocked waiting for their dependent children, which never got to
upload if all 5 goroutines were uploading directories.
Now, allow an unlimited number of upload goroutines (see commit
058434449), but add a gate around things that allocate file
descriptors within the upload function.
This means that now we could have a ton of goroutines existing but
blocked on the gate, wasting ~4KB of memory each rather than the
relatively small amount of memory we used for the *node on the
container/list before.
Future CLs can reduce the memory usage and/or rewrite this.
Or maybe we forget to clean this up, but get smaller goroutine memory
usage in Go 1.3 as is being discussed. It _is_ convenient to just
launch tons of goroutines and manage dependencies with channel
operations rather than managing the dependencies state and goroutine
scheduling in your own code, just to save some memory.
Change-Id: I43d9d7795e5df4d790d6086d18e4b4b99eaacbc8
creation time from potential file metadata.
camput: add the -exiftime option that allows
the use of the above for -filenodes and -vivify.
Fixes: http://code.google.com/p/camlistore/issues/detail?id=77
Change-Id: I05d1af3d875bb984e47e06775fd149e5ed0ed7b1
We might tweak the parameters yet, and I want to add a bunch more
tests, but this is a major relief for me. I've been reluctant to
put too much data into my personal Camlistore instance until I
figured out how I wanted the file chunks to split.
This also simplifies (deletes) a fair bit of code.
Change-Id: I35c38cc5d39a8a43e3f62445434f8d7fd5de5d17
This change introduces 2 features, illustrated by those examples:
1) camput file -filenodes *.jpg
This was not previously possible because -filenodes option only worked
with directories.
2) camput file -filenodes -tag 'nice pics' /media/foo/DCIM/
This was not previously possible because the -tag option could only be
used with the -permanode option.
Change-Id: I80ed95fdc819d359cacfe97ba3b7849b96e406f4
A planned permanode involves setting the contents of the permanode
(instead of a random string) as well as the OpenPGP signing time so
the resultant bytes of the blob (and thus its blobref) is deterministic.
This allows multiple independent devices (my laptops) to create the
same permanodes for the same files (photos backed up from my phone)
when offline (airplane) and then when they sync later, still only have
one permanode per unique file. This means that tagging and other
metadata applied to permanodes on one laptop merge cleanly with
metadata from the other.
Brad Fitzpatrick