Commit Graph

40 Commits

Author SHA1 Message Date
Brad Fitzpatrick 8e44c62047 camget: finish --shared fetching support
Change-Id: I32edf63e01068a0e96f3255ba4d1313682cf03c4
2013-01-02 14:50:52 -08:00
Brad Fitzpatrick 827feaa3ac camget, client, schema: start of camget --shared support. see flag docs.
Change-Id: I5dd43129cb0032821a5913a8f20da0ddb38c63da
2013-01-02 12:55:12 -08:00
Brad Fitzpatrick 898e522126 Close FileReaders. Hunting an fd leak, but this isn't it,
since FileReader.Close is back to doing nothing.

Change-Id: I65e906d75cf2825b9476ed5008ce042f44582113
2012-12-31 18:02:13 -08:00
mpl ed20da7592 vivify: support in camput and upload handler.
no special credentials/auth yet.
pkg/blobserver/handlers/get.go moved to
pkg/blobserver/gethandler/get.go to avoid
a dependency loop with the json sign helper.
pkg/server/sig.go was moved to pkg/jsonsign/signhandler
because it seemed inapproriate to import in
pkg/blobserver/handlers/upload.go something from
pkg/server

Change-Id: Ifeb14512e182e8a101d4fced6d6d4184e2b9cb99
2012-12-30 23:39:22 +01:00
Brad Fitzpatrick 08f09ebf2b schema: fix FileReader bugs, simplify, add exhaustive tests
Change-Id: I5d098d7b12892d278190391dd5abd7272ed997c2
2012-12-25 17:27:52 -08:00
Brad Fitzpatrick 0a0cbe72c1 schema: start of more stressful FileReader tests, chasing a bug
Change-Id: I2aa2b815c785d37674c309a9d6b7c26c67ec3b6e
2012-12-25 12:17:45 -08:00
Brad Fitzpatrick 0b022b433a camput, schema, client: query server for dups before uploading, like the JavaScript UI.
Change-Id: Id12f78cf2a095adcfad90ca4c6416bb80b6e9b14
2012-12-22 22:48:21 -08:00
mpl 6cb7ecd644 camput: Preparation for vivify.
Refactor writeFileMapRolling so we can write the chunks,
then the top level contents, separately.

Change-Id: Ib56cd4b2aa295516fafbe7072ad5b352d1aaaa89
2012-12-23 01:15:29 +01:00
Brad Fitzpatrick 22495f8bfd schema: adjust rollsum splitting algorithm, make it the default
We might tweak the parameters yet, and I want to add a bunch more
tests, but this is a major relief for me.  I've been reluctant to
put too much data into my personal Camlistore instance until I
figured out how I wanted the file chunks to split.

This also simplifies (deletes) a fair bit of code.

Change-Id: I35c38cc5d39a8a43e3f62445434f8d7fd5de5d17
2012-10-29 02:03:05 +01:00
Brad Fitzpatrick 044194b746 schema: document PopulateParts, add a missing error check
Change-Id: Ic9d65fe3e56c4233f481254c6f730a8b382fb4ba
2012-10-29 00:10:31 +01:00
mpl 83ce85b59f old TODOs
Change-Id: Ifa8877f594e65e081536b97f536106301f5c9836
2012-10-19 23:37:14 +02:00
Brad Fitzpatrick e557066f1d schema: more cleanup, making FileReader a ReaderAt.
testing TODOs and further cleanup TODOs remain.

Change-Id: I997153c66805cfe3220d3d735322be14b68b75dc
2012-08-23 20:09:31 -07:00
Brad Fitzpatrick faf3c90d6f schema: more docs and some FileReader work
Change-Id: Id9e754e3b6358a651a826a4cdeb00efb330e3c3a
2012-08-23 18:44:03 -07:00
mpl b355a29150 just one missing map[string]interface{} -> Map conversion
Change-Id: I616ef7260dac9187ea5157f8d47c1b05815ad723
2012-08-22 15:09:42 +02:00
Brad Fitzpatrick 1a51f44196 schema: fix test
Change-Id: Iaacc42167e40233dcb9f5f03bfcfcfe86655a2b1
2012-08-22 04:38:51 +10:00
Brad Fitzpatrick 228c0ca56f schema: change PopulateSymlinkMap to be Map.SetSymlinkTarget
Change-Id: I072ce65331048b54c90c32e951f5034060c1764c
2012-08-22 04:38:25 +10:00
Brad Fitzpatrick a8629ec5dd schema: unexport NewBytes
Change-Id: Ia6ec7c9aec33319bf277b93a189b4fa43d0540b9
2012-08-22 04:33:06 +10:00
Brad Fitzpatrick 439db110ec schema: more cleanup; ditch the Claimer type
Change-Id: I2a52564c1a99c96a2cd8bc84a791784f0a4e4b2e
2012-08-22 04:31:48 +10:00
Brad Fitzpatrick b67d753364 schema: docs and cleanup
Change-Id: Ib720b7f711d71bdeaaf33374f376f14b30a5a97c
2012-08-22 04:14:47 +10:00
Brad Fitzpatrick 11eac8721e Convert more code to use schema.Map instead of map[string]interface{}
Change-Id: I285fd1b0e730ebd32069688a71cb992c43770ade
2012-08-22 03:47:38 +10:00
Brad Fitzpatrick 3cd7f75f89 schema: introduce type Map
Change-Id: I8209930d8505a4cb74b8ea29b5b57f2cd6bd14d2
2012-08-22 03:36:36 +10:00
Brad Fitzpatrick 0a0b012eba camput,schema: cleanup, docs
Change-Id: Ifa94cb1e9e086f85222692508ab12616f4cc3211
2012-08-22 03:31:44 +10:00
Brad Fitzpatrick 8fec29c974 camput: make file permanodes consistent between runs (fix claim times)
Change-Id: Idd2bbda2372802772703eaeaf6fe2e74f9bcdd83
2012-08-21 20:43:38 +10:00
Brad Fitzpatrick b815a2b758 camput: add -filenodes option for per-file content-based planned permanodes
Change-Id: Ic03b990654d860e03b4cc710de92f7d40c392df8
2012-08-21 16:11:10 +10:00
Brad Fitzpatrick 2db390d58e Add schema.NewPlannedPermanode 2012-07-28 15:41:19 -07:00
Brad Fitzpatrick 6fec55a384 simplify legacy time code
Change-Id: I643b722adddaa3bfecab6044de2c5a373e5d7dd1
2012-05-13 13:27:11 -07:00
Brad Fitzpatrick ad6047abf0 initialism
Change-Id: I3ae54133a56b48f614938ac3082dd28ae6197980
2012-05-13 13:06:50 -07:00
Brad Fitzpatrick add0f80721 fs: implement fuse owner/groups
Change-Id: I3cb8fea6aa7dc572e1a77bc2bf872b8c584d1cbf
2012-04-29 10:03:07 +10:00
Brad Fitzpatrick e67e7fef50 schema: add ModTime accessor, and fix unixPermission logic error
Change-Id: Id0637f1a4ab7e9efefe1de0decb28b7965ea50eb
2012-04-28 01:34:53 -07:00
Brad Fitzpatrick 26ebfa7e3a schema: fix/optimize uid/gid lookups
Change-Id: Icc6f9b4912e0ac640af2cba9b39fd803184ceab2
2012-04-19 17:59:25 -07:00
Brad Fitzpatrick 1c19140c7a schema: fix build breakage on darwin; syscall ctim vs ctimespec naming difference
Change-Id: Icbd844d585d331ebe46e9a097ca0e155eb543844
2012-04-15 11:33:59 -07:00
Brad Fitzpatrick 18e53cd72f rename MapToCamliJson to Go style initialism MapToCamliJSON
Change-Id: I674944c9fafa6b78c495d22c525b9ffd0b75dbf9
2012-04-15 07:45:10 -07:00
Brad Fitzpatrick 7961187a0f schema: use filepath.Base instead of by hand, fixes bug with trailing slash
Change-Id: I0216ce084d6a68bdb1c92d85304db030d22f05fb
2012-04-14 15:26:51 -07:00
Brad Fitzpatrick 6aca4d1aee remove debug
Change-Id: I7ced1f4d6376444b8c7491e038addaf9ef35f8c4
2012-04-14 15:22:41 -07:00
Brad Fitzpatrick eaa9c427ac schema: include ctime, if it differs from mtime
Change-Id: Ifc511cc9abb00cee870e5d12fae86d34dd00e054
2012-04-14 15:22:07 -07:00
Brad Fitzpatrick 83727904a5 schema: port some pre-Go 1 schema stuff
Change-Id: Ib5081f38ba2b8997d8c0f58dfbad8e0234f18f8c
2012-04-13 18:40:59 -07:00
Brad Fitzpatrick 40712671ee pkg/auth and pkg/schema: fix tests.
Change-Id: I54f391241c77902ad301edea92d39660f7cf0008
2012-03-29 16:15:59 -07:00
Brad Fitzpatrick a229abeebd camget: fix build. update dev-camget.
Change-Id: I5289801de7b50caedeabd9cc2513d768aafd424b
2012-03-05 08:04:42 -08:00
Brad Fitzpatrick 3ab1618f0b schema: generate unique permanodes.
it was using second time granularity for both the random and the signing,
leading to dup permanode creation on fast machines.

Change-Id: Id7ea50f8ce8c4c3da6964e98bb3a09dc7992cec7
2012-03-03 12:37:54 -08:00
Brad Fitzpatrick 0714a463c9 Update from r60 to [almost] Go 1.
A lot is still broken, but most stuff at least compiles now.

The directory tree has been rearranged now too.  Go libraries are now
under "pkg".  Fully qualified, they are e.g. "camlistore.org/pkg/jsonsign".

The go tool cannot yet fetch from arbitrary domains, but discussion is
happening now on which mechanism to use to allow that.

For now, put the camlistore root under $GOPATH/src.  Typically $GOPATH
is $HOME, so Camlistore should be at $HOME/src/camlistore.org.

Then you can:

$ go build ./server/camlistored

... etc

The build.pl script is currently disabled.  It'll be resurrected at
some point, but with a very different role (helping create a fake
GOPATH and running the go build command, if things are installed at
the wrong place, and/or running fileembed generators).

Many things are certainly broken.

Many things are disabled.  (MySQL, all indexing, etc).

Many things need to be moved into
camlistore.org/third_party/{code.google.com,github.com} and updated
from their r60 to Go 1 versions, where applicable.

The GoMySQL stuff should be updated to use database/sql and the ziutek
library implementing database/sql/driver.

Help wanted.

Change-Id: If71217dc5c8f0e70dbe46e9504ca5131c6eeacde
2012-02-18 21:53:06 -08:00