Commit Graph

209 Commits

Author SHA1 Message Date
mpl a534e02c2d camput: client becomes StatReceiver.
also added ReaderSize function to
guess before reading the size of
a source.

Change-Id: I659e174821696b3c58759b132b82710a49583018
2012-10-11 11:51:12 +02:00
Brad Fitzpatrick 228c0ca56f schema: change PopulateSymlinkMap to be Map.SetSymlinkTarget
Change-Id: I072ce65331048b54c90c32e951f5034060c1764c
2012-08-22 04:38:25 +10:00
Brad Fitzpatrick 439db110ec schema: more cleanup; ditch the Claimer type
Change-Id: I2a52564c1a99c96a2cd8bc84a791784f0a4e4b2e
2012-08-22 04:31:48 +10:00
Brad Fitzpatrick 11eac8721e Convert more code to use schema.Map instead of map[string]interface{}
Change-Id: I285fd1b0e730ebd32069688a71cb992c43770ade
2012-08-22 03:47:38 +10:00
Brad Fitzpatrick 3cd7f75f89 schema: introduce type Map
Change-Id: I8209930d8505a4cb74b8ea29b5b57f2cd6bd14d2
2012-08-22 03:36:36 +10:00
Brad Fitzpatrick 0a0b012eba camput,schema: cleanup, docs
Change-Id: Ifa94cb1e9e086f85222692508ab12616f4cc3211
2012-08-22 03:31:44 +10:00
Brad Fitzpatrick dc684dfa6a camput: updated comments, TODO
Change-Id: I8df73d4c1c44c71424bcfaf7d3cb124c1738c791
2012-08-22 02:39:36 +10:00
Brad Fitzpatrick 8fec29c974 camput: make file permanodes consistent between runs (fix claim times)
Change-Id: Idd2bbda2372802772703eaeaf6fe2e74f9bcdd83
2012-08-21 20:43:38 +10:00
Brad Fitzpatrick b815a2b758 camput: add -filenodes option for per-file content-based planned permanodes
Change-Id: Ic03b990654d860e03b4cc710de92f7d40c392df8
2012-08-21 16:11:10 +10:00
Brad Fitzpatrick 51e88cac65 Support for "planned permanodes" in pkg/jsonsign, pkg/schema and cmd/camput.
A planned permanode involves setting the contents of the permanode
(instead of a random string) as well as the OpenPGP signing time so
the resultant bytes of the blob (and thus its blobref) is deterministic.

This allows multiple independent devices (my laptops) to create the
same permanodes for the same files (photos backed up from my phone)
when offline (airplane) and then when they sync later, still only have
one permanode per unique file.  This means that tagging and other
metadata applied to permanodes on one laptop merge cleanly with
metadata from the other.
2012-07-28 16:32:31 -07:00
Brad Fitzpatrick 8c293e34b6 Per Go initialism style, rename Json to JSON 2012-07-28 15:42:56 -07:00
mpl 326d62aea4 camput: actually catch errors when trying to upload,
also added startupcheck for remote blobserver

Change-Id: Id9f5383c6fd3fdf697f2b75052f3e63b0972205d
2012-07-23 11:05:57 +02:00
mpl e3c0d11011 camput share, argument bugfix
Change-Id: Ic1bc9889b50dc69509810a619f34b4896e68587b
2012-05-13 15:55:12 +02:00
Brad Fitzpatrick 4b8df05859 fs,fuse: new 'welcome' root node, where you can navigate/search anywhere.
Change-Id: I66f076a85d8e474bb5d93ad8743fc8181de7502f
2012-04-29 14:29:51 +10:00
Brad Fitzpatrick f19c1f9026 cammount: fix fmt pattern
Change-Id: I8bb3df8a14cd3d3b44c899f3e2754b50b35b0028
2012-04-26 19:34:29 -07:00
Brad Fitzpatrick 5315bbf232 camput: re-do the stat/have flatcaches.
still waiting on leveldb-go, but this works for now.

Change-Id: I630d7bd2757a02d3a711c48075486c5cc6339118
2012-04-22 17:18:19 -07:00
Brad Fitzpatrick b4bff6a7d9 camput: check decode errors, exposes bugs in flat cache
Change-Id: I5fd812300c0c92892447a57d1da49cceedca9e3f
2012-04-21 08:54:26 -07:00
Brad Fitzpatrick df339b446a camput: show blob stats during upload too
Change-Id: I988e5e12cfee355dce0170165c42cb4d162dc259
2012-04-21 08:30:04 -07:00
Brad Fitzpatrick da42bdf523 camput: more code movement
Change-Id: I775096c39b5c9671a3c054b7d02562cfa787f0a1
2012-04-21 07:22:32 -07:00
Brad Fitzpatrick e6e26f2017 camput: move common uploader stuff to uploader.go
Change-Id: I6137a945f89f4a0603189a5fc04b49a570a3403b
2012-04-21 07:14:57 -07:00
Brad Fitzpatrick cd79828912 camput: move nodeworker into its own file
Change-Id: Ic61d35955fe4dd59ada65a314f0c94cc3f7b0f87
2012-04-21 07:07:07 -07:00
Brad Fitzpatrick 2274bd97b1 camput: bunch of cleanups and bugs and usability stuff.
Change-Id: I029cc38d5b10a2d322e26148eb4971b3c838c889
2012-04-20 14:43:23 -07:00
Brad Fitzpatrick 0d989a6ae6 camput: fix up statcache to work with Go 1. crashed before with gob error on os.FileInfo.
Change-Id: I9b88435e00352988e5f89e64f45ea4cb3d2c76ab
2012-04-20 14:27:52 -07:00
Brad Fitzpatrick 9168eb84f0 camput: fix logging bug
Change-Id: I8781b523013dca814b81f99fed85ba59a489862c
2012-04-20 14:03:09 -07:00
Brad Fitzpatrick 5005dcf751 camput: fix statcache/havecache flag mix-up.
Change-Id: Ie7caea7841d055ed47774d0279095f218740ab5a
2012-04-20 13:05:34 -07:00
Brad Fitzpatrick 0baa073b44 camput: switch to new tree upload code.
Change-Id: Iae653e4442bab63979d584caecdce58df942385d
2012-04-19 17:59:41 -07:00
Brad Fitzpatrick 0f3ce52c54 camput: memoize SumBytes
Change-Id: Ie0ea05a895f97de2b069c088d4195b533b70ea8e
2012-04-18 07:24:45 -07:00
Brad Fitzpatrick 7e94879e86 camput: in du mode, only run 1 upload worker.
Change-Id: I65b642095ca9f162f14db512f34bb7073ef7a13b
2012-04-17 17:46:05 -07:00
Brad Fitzpatrick 715c5278e0 camput: working diskusage mode.
Change-Id: I7c3df20fed7720e3535b5e618d20842d1ddaadcc
2012-04-15 17:21:19 -07:00
Brad Fitzpatrick 4f0f53a90e camput: more TreeUpload work; adds NodeWorker helper.
Change-Id: Ie5919faa34fd2f6eb1b61669598e0a06e1d8ba07
2012-04-15 17:11:25 -07:00
Brad Fitzpatrick 69ce6654e8 camput: fix bug so default switch case can be hit now.
otherwise all other files were hitting the not-directory case.

Change-Id: I03e6021e947eee301e21db3f20c2da3c2f92ab2d
2012-04-15 14:25:41 -07:00
Brad Fitzpatrick 92b10a39a4 camput: remove dead flag
Change-Id: Idf6ec9416b5878379de19b53a93b86982125f7cb
2012-04-15 14:21:38 -07:00
Brad Fitzpatrick d05a336cb0 camput: move more blob-only stuff to blobs.go
Change-Id: I967e9dc975db18088b88c00b2672353a5a72f458
2012-04-15 14:21:00 -07:00
Brad Fitzpatrick 90c254038c camput: remove dead code
Change-Id: Ifab0247b6c23d050abd822b479558a0d75930b8b
2012-04-15 14:10:08 -07:00
Brad Fitzpatrick 77fc9e8abb camput: only support '-' meaning stdin for the 'blob' subcommand, not 'file'
Change-Id: I9a8923792385e232dc42c7707c1b748f82a5ac37
2012-04-15 14:01:39 -07:00
Brad Fitzpatrick 88cff46f28 camput: more TreeUpload work
Change-Id: I8638ccdc342642447661e68918259878e04ef756
2012-04-15 13:54:21 -07:00
Brad Fitzpatrick 61a895b1f8 camput: some more TreeUpload work
Change-Id: I2b4e74f7892720bbf6b544a7abb9867d5ad3415b
2012-04-15 11:26:36 -07:00
Brad Fitzpatrick f366dfb850 some TreeUpload TODO notes
Change-Id: If26ddcb6ee8ebc1b35f569ecbf835da5f666c3a7
2012-04-15 08:55:11 -07:00
Brad Fitzpatrick 72ac3fc8b1 camput: TODOs for VFS
Change-Id: I9b403ec5e3e66f150da4277eabe99e079c61d2b8
2012-04-15 08:28:18 -07:00
Brad Fitzpatrick 683e762c79 camput: go through virtual filesystem. will be good for filtering skipDirs & testing.
Change-Id: I2fa56615b6b199ab054de388dde652749049d5e1
2012-04-15 08:11:37 -07:00
Brad Fitzpatrick 147cdb05c9 camput: unindent
Change-Id: I151b136a97a129a75f3f83fca8f1be841f7bec2b
2012-04-15 07:47:38 -07:00
Brad Fitzpatrick 18e53cd72f rename MapToCamliJson to Go style initialism MapToCamliJSON
Change-Id: I674944c9fafa6b78c495d22c525b9ffd0b75dbf9
2012-04-15 07:45:10 -07:00
Brad Fitzpatrick a8687d112c camput: move rollSplits to Uploader field, not passed along.
Change-Id: I8e78fe88e179235a03117927bd7b36217242e055
2012-04-15 07:18:40 -07:00
Brad Fitzpatrick 91501cdbd1 remove unused code
Change-Id: I4e35ce6dbb34f3db17024f49dd98bed7d3f9b4f0
2012-04-15 07:10:10 -07:00
Brad Fitzpatrick 77f9544936 gofmt
Change-Id: If71fa78e10464bc93fc7ec748e73d935d3c555ba
2012-04-14 22:04:00 -07:00
Brad Fitzpatrick 5b379561a7 Fix bogus r60->go1 gofix.
Change-Id: Ieb229a6767dbb955b898006d10926d8715bbe307
2012-04-14 22:03:54 -07:00
Brad Fitzpatrick 6f3826f6af camput: flag for diskUsage mode
Change-Id: I9a8bdd907ddcb9debb8381c11691a7485ecba6c2
2012-04-14 22:01:19 -07:00
Brad Fitzpatrick f8f95554d5 camput: fix panic on errors.
I think maybe I should just change handleResult to panic with its own error type and catch it above.

Change-Id: Ib0622321e29e7926173c4957dc0e56f3b514e7d2
2012-04-12 22:18:38 -07:00
Brad Fitzpatrick e4a17bc8ac camput: fix init and some config stuff.
Change-Id: I121bd9c2dfac3c076db0db75d8f939b5ec571ca4
2012-04-12 19:36:00 -07:00
Brad Fitzpatrick c6e6f7a92d camdb: gofmt and change panic to log.Fatalf
Change-Id: I8c8ced3cb5f74a7742a2a9f86874276b3c1fd0c2
2012-04-11 19:13:39 -07:00
Brad Fitzpatrick 9447a8cb78 camput: reorder file type switch; fixes symlink bug
Change-Id: Ic4d0f6c2be02d153f933d72c1a02e72926c36786
2012-03-27 10:41:34 -07:00
Brad Fitzpatrick 7f90db0291 rename mysqlindexer to mysql to match import
Change-Id: I23cd9cfc40a0e01cb0086fc5c9a3ad108f3705a8
2012-03-27 09:37:32 -07:00
Brad Fitzpatrick 69328c751f fix camdbinit; use correct mysqlindexer import
Change-Id: I227e219ec32f81a11cc6192cd60de8b89878bedc
2012-03-27 09:17:26 -07:00
Brad Fitzpatrick 10cf23c700 Get the MySQL indexer working again.
It now uses the generic indexing layer, rather than its own schema.
Also, move it under pkg/index/, like mongo.
Also, link in mongo to the main binary.
Not using the tests yet (leaving for mpl :))

Change-Id: I47a2203ca6c0b5244cd458aedec46227c4363c62
2012-03-26 13:57:53 -07:00
Brad Fitzpatrick 899f8a0e7a move camdbinit to cmd/
Change-Id: I21657e1cc91bda7a7b18b609b1bfac37680256cb
2012-03-24 19:46:38 -07:00
Brad Fitzpatrick 374c1f5360 cammount: move to cmd/ directory; update dev-cammount script.
Change-Id: Ifa0c4d586c30ebb807a8094594e4872dcd5039ef
2012-03-18 23:58:36 -07:00
Brad Fitzpatrick a229abeebd camget: fix build. update dev-camget.
Change-Id: I5289801de7b50caedeabd9cc2513d768aafd424b
2012-03-05 08:04:42 -08:00
Brad Fitzpatrick c825acde06 move camdebug to cmd; it compiles as Go 1 already.
Change-Id: Ic9aa1112a504da96738b7ec3fc224df54b92cdc8
2012-02-21 02:14:34 -08:00
Brad Fitzpatrick 0714a463c9 Update from r60 to [almost] Go 1.
A lot is still broken, but most stuff at least compiles now.

The directory tree has been rearranged now too.  Go libraries are now
under "pkg".  Fully qualified, they are e.g. "camlistore.org/pkg/jsonsign".

The go tool cannot yet fetch from arbitrary domains, but discussion is
happening now on which mechanism to use to allow that.

For now, put the camlistore root under $GOPATH/src.  Typically $GOPATH
is $HOME, so Camlistore should be at $HOME/src/camlistore.org.

Then you can:

$ go build ./server/camlistored

... etc

The build.pl script is currently disabled.  It'll be resurrected at
some point, but with a very different role (helping create a fake
GOPATH and running the go build command, if things are installed at
the wrong place, and/or running fileembed generators).

Many things are certainly broken.

Many things are disabled.  (MySQL, all indexing, etc).

Many things need to be moved into
camlistore.org/third_party/{code.google.com,github.com} and updated
from their r60 to Go 1 versions, where applicable.

The GoMySQL stuff should be updated to use database/sql and the ziutek
library implementing database/sql/driver.

Help wanted.

Change-Id: If71217dc5c8f0e70dbe46e9504ca5131c6eeacde
2012-02-18 21:53:06 -08:00