Commit Graph

3 Commits

Author SHA1 Message Date
Brad Fitzpatrick 70475701d1 Get rid of QueueCreator and all its associated complexity.
Previous TODO entry was:

-- Get rid of QueueCreator entirely. Plan:
     -- sync handler still has a source and dest (one pair) but
        instead of calling CreateQueue on the source, it instead
        has an index.Storage (configured via a RequiredObject
        so it can be a kvfile, leveldb, mysql, postgres etc)
     -- make all the index.Storage types be instantiable
        from a jsonconfig Object, perhaps with constructors keyed
        on a "type" field.
     -- make sync handler support blobserver.Receiver (or StatReceiver)
        like indexes, so it can receive blobs.  but all it needs to
        do to acknowledge the ReceiveBlob is write and flush to its
        index.Storage. the syncing is async by default. (otherwise callers
        could just use "replica" if they wanted sync replication).
        But maybe for ease of configuration switching, we could also
        support a sync mode.  when it needs to replicate a blob,
        it uses the source.
     -- future option: sync mirror to an alternate path on ReceiveBlob
        that can delete. e.g. you're uploading to s3 and google,
        but don't want to upload to both at once, so you use the localdisk
        as a buffer to spread out your upstream bandwidth.
     -- end result: no more hardlinks or queue creator.

Change-Id: I6244fc4f3a655f08470ae3160502659399f468ed
2013-11-22 14:33:31 -08:00
Tamás Gulácsi a624b6d3b2 Add generation support for diskpacked.
Refactor the localdisk, diskpacked common code to pkg/blobserver/local
(only StorageGeneration, ResetStorageGeneration in this CL)

Change-Id: Ib04125805d5a1960bd29a474d3fc7ca985708d8d
2013-10-10 21:35:44 +02:00
Brad Fitzpatrick 71d7cf9e88 Implement storage generations, init and discovery 2012-11-07 21:14:56 +01:00