perkeep/TODO

There are two TODO lists. This file (good for airplanes) and the online bug tracker:

     https://code.google.com/p/camlistore/issues/list

Offline list:

-- unexport more stuff from pkg/server. Cache, PublishRoots, etc.

-- make UI handler's scaledImage cache take a jsonconfig constructor
   of a sorted.KeyValue-registered type. Always slap a memory LRU in front
   of it.

-- In ImageHandler.cache, write the thumbnail out as one large blob
   instead of using schema.WriteFileFromReader if the thumbnail
   is smaller than blobserver.MaxBlobSize (16MB).

-- blobhub.go NotifyBlobReceived should do the sync hooks before firing off
   goroutines to notify that things were good. they might not be good yet
   if the hook fails. also, no need for the slice. can just fire off goroutines
   directly.

-- look into garbage from openpgp signing

-- make leveldb memdb's iterator struct only 8 bytes, pointing to a recycled
   object, and just nil out that pointer at EOF.

-- genconfig needs to be updated to route incoming blobs to sync handlers;
   or should sync handlers subscribe to their source's blobhub?
   No, see latest email to Bill Thiede: sync handlers already subscribe
   with a blocking hook. Related: index should too.

-- bring in the google glog package to third_party and use it in
   places that want selective logging (e.g. pkg/index/receive.go)

-- camput's havecache initialization should move up a layer to
   to be global to all commands, not specific to "camput file".
   (the statcache can stay in file).  otherwise, we have no haveCache
   on the pkg/client.*Client and the UploadAndSignBlob call (for
   creating permanodes and other claims) can't efficiently upload its
   public key if the server doesn't already have it.  currently it
   has to try to upload it (doing a remote HTTP stat) each time
   when using "camput permanode" or "camput attr".

-- use 'uint32' instead of 'int64' for blob sizes everywhere (notably
   blob.SizedRef).  blobs have a max size of 10-32 MB anyway, and the
   index.Corpus is now using uint32 to save memory.

-- verify all ReceiveBlob calls and see which should be
   blobserver.Receive instead, or ReceiveNoHash.  git grep -E
   "\.ReceiveBlob\(" And maybe ReceiveNoHash should go away and be
   replaced with a "ReceiveString" method which combines the
   blobref-from-string and ReceiveNoHash at once.

-- union storage target. sharder can be thought of a specialization
   of union. sharder already unions, but has a hard-coded policy
   of where to put new blobs. union could a library (used by sharder)
   with a pluggable policy on that.

-- support for running cammount under camlistored. especially for OS X,
   where the lifetime of the background daemon will be the same as the
   user's login session.

-- website: remove the "Installation" heading for /cmd/*, since
   they're misleading and people should use "go run make.go" in the
   general case.

-- website: add godoc for /server/camlistored (also without a "go get"
   line)

-- tests for all cmd/* stuff, perhaps as part of some integration
   tests.

-- move most of camput into a library, not a package main.

-- server cron support: full syncs, camput file backups, integrity
   checks.

-- status in top right of UI: sync, crons. (in-progress, un-acked
   problems)

-- finish metadata compaction on the encryption blobserver.Storage wrapper.

-- get security review on encryption wrapper. (agl?)

-- peer-to-peer server and blobserver target to store encrypted blobs
   on stranger's hardrives.  server will be open source so groups of
   friends/family can run their own for small circles, or some company
   could run a huge instance.  spray encrypted backup chunks across
   friends' machines, and have central server(s) present challenges to
   the replicas to have them verify what they have and how big, and
   also occasionally say what the SHA-1("challenge" + blob-data) is.

-- sharing: make camget work with permanode sets too, not just
   "directory" and "file" things.

-- sharing: when hitting e.g. http://myserver/share/sha1-xxxxx, if
   a web browser and not a smart client (Accept header? User-Agent?)
   then redirect or render a cutesy gallery or file browser instead,
   still with machine-readable data for slurping.

-- rethink the directory schema so it can a) represent directories
   with millions of files (without making a >1MB or >16MB schema blob),
   probably forming a tree, similar to files. but rather than rolling checksum,
   just split lexically when nodes get too big.

-- delete mostly-obsolete camsigd.  see big TODO in camsigd.go.

-- we used to be able live-edit js/css files in server/camlistored/ui when
   running under the App Engine dev_appserver.py.  That's now broken with my
   latest efforts to revive it.  The place to start looking is:
        server/camlistored/ui/fileembed_appengine.go

-- should a "share" claim be not a claim but its own permanode, so it
   can be rescinded?  right now you can't really unshare a "haveref"
   claim.  or rather, TODO: verify we support "delete" claims to
   delete any claim, and verify the share system and indexer all
   support it.  I think the indexer might, but not the share system.
   Also TODO: "camput delete" or "rescind" subcommand.
   Also TODO: document share claims in doc/schema/ and on website.

-- make the -transitive flag for "camput share -transitive" be a tri-state:
   unset, true, false, and unset should then mean default to true for "file"
   and "directory" schema blobs, and "false" for other things.

-- index: static directory recursive sizes: search: ask to see biggest directories?

-- index: index dates in filenames ("yyyy-mm-dd-Foo-Trip", "yyyy-mm blah", etc).

-- get webdav server working again, for mounting on Windows.  This worked before Go 1
   but bitrot when we moved pkg/fs to use the rsc/fuse.

-- work on runsit more, so I can start using this more often.  runsit should
   be able to reload itself, and also watch for binaries changing and restart
   when binaries change.  (or symlinks to binaries)

-- add gofmt check to "make presubmit". also add "make fmt" target.

-- BUG: osutil paths.go on OS X: should use Library everywhere instead of mix of
   Library and ~/.camlistore?

OLD:

-- add CROS support?
   http://hacks.mozilla.org/2009/07/cross-site-xmlhttprequest-with-cors/

-- brackup integration, perhaps sans GPG? (requires Perl client?)

-- blobserver test suite: flesh it out.  (bs-test.pl ... it's pretty good
   so far, but not done)

-- blobserver: clean up channel-closing consistency in blobserver interface
   (most close, one doesn't.  all should probably close)

Android:

[ ] Fix wake locks in UploadThread.  need to hold CPU + WiFi whenever
    something's enqueued at all and we're running.  Move out of the Thread
    that's uploading itself.
[ ] GPG signing of blobs (brad)
    http://code.google.com/p/android-privacy-guard/
    http://www.thialfihar.org/projects/apg/
    (supports signing in code, but not an Intent?)
    http://code.google.com/p/android-privacy-guard/wiki/UsingApgForDevelopment
    ... mailed the author.

Client libraries:

[X] Go
[X] JavaScript
[/] Python (Brett); but see https://github.com/tsileo/camlipy
[ ] Perl
[ ] Ruby
[ ] PHP