perkeep/TODO

There are two TODO lists. This file (good for airplanes) and the online bug tracker:

     https://code.google.com/p/camlistore/issues/list

Offline list:

-- add camliRoot & closureRoot to pkg/genconfig high-level config &
   the docs

-- website: remove the "Installation" heading for /cmd/*, since
   they're misleading and people should use "go run make.go" in the
   general case.

-- website: add godoc for /server/camlistored (also without a "go get"
   line)

-- make fileembed & zembed only happen as part of "make.go", but
   remove the z*.go files from git (and make them all .gitignore'd).
   Users' options will be either a) use make.go to get a fully static
   binary (with all of Closure zembed-ed), or b) use "go install" and
   get a binary that requires configuration to tell it where files on
   disk are. On start-up, if the UI is configured, the UI should fail
   and say "You didn't configure the path to the UI & Closure
   directories".  make.go could further have a flag.Bool to do the
   zembed, defaulting to true.

   Despite the full Closure source being 10 MB, it's only 2 MB if we
   compress it and embed a zip or .tar.gz in the static binary:

   $ find tmp/closure-lib/closure/ -name '*.js' | xargs cat  | wc
   327098 1214807 10449092
   $ find tmp/closure-lib/closure/ -name '*.js' | xargs cat  | gzip -c | wc
   7436   58545 1932707

-- tests for all cmd/* stuff, perhaps as part of some integration
   tests.

-- move most of camput into a library, not a package main.

-- server cron support: full syncs, camput file backups, integrity
   checks.

-- status in top right of UI: sync, crons. (in-progress, un-acked
   problems)

-- encryption blobserver.Storage wrapper (like cond & replicate)
   putting encrypted metadata in one blobserver.Storage and encrypted
   blobs in another blobserver.Storage.  the metadata one will
   occasionally be scanned and small (single record) items be combined
   into larger metadata blobs with the small ones removed.
   enumeration comes from opening all metadata blobs and merge-sorting
   them all.  the metadata blobs are themselves blobs (content-addressed and all),
   containing encrypted data of:
   (plaintext-blobref plaintext-size encrypted-blobref that-enc-blob-IV)
   Using AES and crypto/cipher (re-consult agl on specifics)

-- peer-to-peer server and blobserver target to store encrypted blobs
   on stranger's hardrives.  server will be open source so groups of
   friends/family can run their own for small circles, or some company
   could run a huge instance.  spray encrypted backup chunks across
   friends' machines, and have central server(s) present challenges to
   the replicas to have them verify what they have and how big, and
   also occasionally say what the SHA-1("challenge" + blob-data) is.

-- newui: add a Makefile rule (like "make embeds") to actually run the
   Closure Compiler on the newui code, and show errors/warnings, and
   generate new z*_embed.js file(s). (we can do rolling checksum splits
   of it later. TODO also: figure out if we can compile in non-minifying
   mode for now, to increase chances of rolling checksums splitting
   predictably)

-- sharing: make camget work with permanode sets too, not just
   "directory" and "file" things.

-- sharing: when hitting e.g. http://myserver/share/sha1-xxxxx, if
   a web browser and not a smart client (Accept header? User-Agent?)
   then redirect or render a cutesy gallery or file browser instead,
   still with machine-readable data for slurping.

-- rethink the directory schema so it can a) represent directories
   with millions of files (without making a >1MB or >16MB schema blob),
   probably forming a tree, similar to files. but rather than rolling checksum,
   just split lexically when nodes get too big.

-- delete mostly-obsolete camsigd.  see big TODO in camsigd.go.

-- we used to be able live-edit js/css files in server/camlistored/ui when
   running under the App Engine dev_appserver.py.  That's now broken with my
   latest efforts to revive it.  The place to start looking is:
        server/camlistored/ui/fileembed_appengine.go

-- should a "share" claim be not a claim but its own permanode, so it
   can be rescinded?  right now you can't really unshare a "haveref"
   claim.  or rather, TODO: verify we support "delete" claims to
   delete any claim, and verify the share system and indexer all
   support it.  I think the indexer might, but not the share system.
   Also TODO: "camput delete" or "rescind" subcommand.

-- make the -transitive flag for "camput share -transitive" be a tri-state:
   unset, true, false, and unset should then mean default to true for "file"
   and "directory" schema blobs, and "false" for other things.

-- index: static directory recursive sizes: search: ask to see biggest directories?

-- index: index dates in filenames ("yyyy-mm-dd-Foo-Trip", "yyyy-mm blah", etc).

-- get webdav server working again, for mounting on Windows.  This worked before Go 1
   but bitrot when we moved pkg/fs to use the rsc/fuse.

-- read/write fuse.  using search / camliMember, permanode per inode
   (https://github.com/hanwen/go-fuse + Go camlistore client == easy?)

-- work on runsit more, so I can start using this more often.  runsit should
   be able to reload itself, and also watch for binaries changing and restart
   when binaries change.  (or symlinks to binaries)

-- BUG: ./dev-server and hitting http://localhost:3179/ shouldn't mention
   the /setup handler.

-- add gofmt check to "make presubmit". also add "make fmt" target.

-- BUG: osutil paths.go on OS X: should use Library everywhere instead of mix of
   Library and ~/.camlistore?

-- camput: support for skipping common cache/temp files (most of the support is
   there now, just need to modify cmd/camput/files.go)

OLD:

-- add CROS support?
   http://hacks.mozilla.org/2009/07/cross-site-xmlhttprequest-with-cors/

-- brackup integration, perhaps sans GPG? (requires Perl client?)

-- blobserver test suite: flesh it out.  (bs-test.pl ... it's pretty good
   so far, but not done)

-- blobserver: clean up channel-closing consistency in blobserver interface
   (most close, one doesn't.  all should probably close)

Android:

[ ] Fix wake locks in UploadThread.  need to hold CPU + WiFi whenever
    something's enqueued at all and we're running.  Move out of the Thread
    that's uploading itself.
[ ] GPG signing of blobs (brad)
    http://code.google.com/p/android-privacy-guard/
    http://www.thialfihar.org/projects/apg/
    (supports signing in code, but not an Intent?)
    http://code.google.com/p/android-privacy-guard/wiki/UsingApgForDevelopment
    ... mailed the author.

Client libraries:

[X] Go
[X] JavaScript
[/] Python (Brett)
[ ] Perl
[ ] Ruby
[ ] PHP