Commit Graph

94 Commits

Author SHA1 Message Date
Brad Fitzpatrick 2086406552 Client fixes and cleanups:
* Fix TODO about moving camput file's havecache up a layer into
  camput itself. So now it's used for all commands (camput permanode,
  camput attr, etc). Reduces HTTP requests.

* Fix camlistore.org/issue/343 -- don't do stats before uploads
  in cases where it's useless. Adds a field to UploadHandle.

* Fix camlistore.org/issue/344: upload the public key whenever
  uploading a signed blob. Usually this is a no-op because of the
  have cache.

* Use zero (or <=0 rather), not -1, to mean unknown on UploadHandle
  Size.

* More docs on public stuff.
2014-01-15 18:32:02 -08:00
Brad Fitzpatrick a3e6f4c845 TODO: remove some items
Change-Id: I1a57ea29222a0bf86beeff5d6b130fab753d52ee
2014-01-14 17:52:37 -08:00
Brad Fitzpatrick 533600daff pkg/server: Run synchronous blob hooks first in blobhub.
Change-Id: I4cc04b10d97bb9aefea4f0e2ae61117b8ef15c17
2014-01-14 17:51:31 -08:00
Brad Fitzpatrick e14a849c35 Remove TODO. Done with env CAMLI_DISABLE_THUMB_CACHE now.
Change-Id: I84835c565eda7bcb0562a4748357c9e05188eca3
2014-01-14 17:43:44 -08:00
Brad Fitzpatrick ab635c6ab3 misc UI TODOs
Change-Id: Ic874bd8fac550d35268f006e1e1beb3bd88ba3b5
2013-12-15 20:58:53 -08:00
Brad Fitzpatrick 4edcdd5098 Use singleflight when generating thumbnails.
Change-Id: I3691f912748a85db09f96029584129e0897191ef
2013-12-14 12:05:51 -08:00
Brad Fitzpatrick 5c5666d037 Thumbnail cleanups and let the meta map persist on disk with a sorted.KeyValue
Previously, every time you restarted the server, it forgot about all
thumbnails previously generated.  (in practice it didn't/doesn't
matter with a single user, though, since they're still cached in the
browser and we always reply to If-Modified-Since immediately without
checking the cache)  But it'll matter more with the Publish handler.

Also, rename some stuff, clean up some stuff, drop an unused interface.

And then necessarily change the serverconfig low-level generator to use
a kvfile for the thumbmeta map when using local disk for blobs.

--

Change-Id: I4dcfcb21429a440aa118794c03f7abf7bd69c33b
2013-12-14 09:37:56 -08:00
Brad Fitzpatrick eb483eb9f6 Limit number of image resizes happening at once.
Attempt to ease memory spikes.

Change-Id: I4df7177c063b284ac841335d8f18b1a6da5289d0
2013-12-14 15:25:39 +01:00
Brad Fitzpatrick a159d20b41 server: unexport some image stuff, add docs, TODOs, clean up
Change-Id: I4914de33406da08a4bbf806d9d0386a285e4a8d7
2013-12-14 15:16:29 +01:00
Brad Fitzpatrick 1ade80b0f0 add blobhub TODO
Change-Id: I8d3b69d0264610cfdb35addfefeaad11be43c557
2013-12-11 15:53:17 +04:00
Brad Fitzpatrick c7a0c5842c little TODO
Change-Id: Icda704f63db4ad39ea4bfe13bb9b7d62b227d662
2013-12-04 00:39:09 -08:00
Brad Fitzpatrick 749c5aed3d leveldb TODO
Change-Id: I20f5b0491d3adcc7eca89d819345efd31f11adde
2013-12-03 21:36:58 -08:00
Brad Fitzpatrick 0c73a65f78 TODO: use uint32 for SizedRef, not int
Change-Id: I6f8bdfd483c1be4e3ec933f2bf05686163ed2bd7
2013-12-02 12:01:42 -08:00
mpl 1ee5fd20c5 search: deletions are not modifications
1) pkg/search: documented that deletions times do not
qualify as modtimes

2) pkg/index: got rid of DeletedAt, and keyDeletes

http://camlistore.org/issue/191

Change-Id: I39578913345454d36af4599e29e7053f46577846
2013-11-29 00:29:57 +01:00
Brad Fitzpatrick c733835684 more TODO
Change-Id: Ib43348a234a1902fa94d3d13b7e10aecb49da0e2
2013-11-26 19:27:03 -08:00
Brad Fitzpatrick ab19715dc6 docs and TODOs
Change-Id: I434c4d00a4dd63d338646376a563f69b122a3c53
2013-11-23 09:09:40 -08:00
Brad Fitzpatrick 90c1e48afe Rename index.Storage to sorted.KeyValue and move it into a new package.
Having index.Index and index.Storage both in the same package led to
confusing discussions about "an index". Better names now, and smaller
packages.
2013-11-22 23:24:54 -08:00
Brad Fitzpatrick 70475701d1 Get rid of QueueCreator and all its associated complexity.
Previous TODO entry was:

-- Get rid of QueueCreator entirely. Plan:
     -- sync handler still has a source and dest (one pair) but
        instead of calling CreateQueue on the source, it instead
        has an index.Storage (configured via a RequiredObject
        so it can be a kvfile, leveldb, mysql, postgres etc)
     -- make all the index.Storage types be instantiable
        from a jsonconfig Object, perhaps with constructors keyed
        on a "type" field.
     -- make sync handler support blobserver.Receiver (or StatReceiver)
        like indexes, so it can receive blobs.  but all it needs to
        do to acknowledge the ReceiveBlob is write and flush to its
        index.Storage. the syncing is async by default. (otherwise callers
        could just use "replica" if they wanted sync replication).
        But maybe for ease of configuration switching, we could also
        support a sync mode.  when it needs to replicate a blob,
        it uses the source.
     -- future option: sync mirror to an alternate path on ReceiveBlob
        that can delete. e.g. you're uploading to s3 and google,
        but don't want to upload to both at once, so you use the localdisk
        as a buffer to spread out your upstream bandwidth.
     -- end result: no more hardlinks or queue creator.

Change-Id: I6244fc4f3a655f08470ae3160502659399f468ed
2013-11-22 14:33:31 -08:00
Brad Fitzpatrick e190090cbd add TODO
Change-Id: I3e4c33d1bafcdf4f6afde3440995c591c9457787
2013-11-22 12:40:33 -08:00
Brad Fitzpatrick 27b19787e0 TODO: new plan to kill QueueCreator
Change-Id: I098307063f7645889a39dedafc4d3f862a3f798c
2013-11-18 18:33:59 -08:00
Brad Fitzpatrick 6a4c0ec09c TODO: use glog
Change-Id: I94ce97b93c73bee47f90a966f5cb1e860b23233a
2013-11-09 19:55:49 -05:00
Brad Fitzpatrick 63b4f390cb client: upload public key if necessary whenever uploading a signed blob
Change-Id: I2e61da4fa2eeae2332c67902f8dd5f064b027835
2013-09-22 19:38:42 +01:00
Brad Fitzpatrick e8f89f609b website: document the 0.5 release, Castletownbere
Change-Id: I825bd08fdc60d84137e7b68663058ba7fa2fc42d
2013-09-21 18:27:42 +01:00
Brad Fitzpatrick 8319411ab4 Convert more ReceiveBlob into blobserver.Receive or blobserver.ReceiveNoHash
Change-Id: I9199555324b617167a6062a8b55ed09b449bae4f
2013-09-16 15:57:14 +01:00
Brad Fitzpatrick 13691c699f make.go: build camput, etc faster, ignoring embedded UI resources
Change-Id: I1f7f60f4d5de97f69ddaf8c370889320ca8d5efe
2013-09-10 22:42:24 -07:00
Brad Fitzpatrick e731931edd Some TODO items.
Change-Id: I4236764cb1dc4a1aea003d86b43687576c1fe3c5
2013-09-10 20:02:26 -07:00
Brad Fitzpatrick 2963597d1e add a TODO
Change-Id: I653b5cea785ba08455e84e06f01734a6d384f240
2013-08-29 16:12:18 -07:00
Brad Fitzpatrick dbb93fd888 Don't link to /setup handler for dev server.
Change-Id: Ic0c27718b3a06c2a6dff21a24744b1dce2944617
2013-08-26 11:25:11 -05:00
Brad Fitzpatrick d293b8fd34 Use pkg/gate in a couple places. Do s3 parallel state. Update TODO.
Change-Id: I23006baa7b454ceca3f07c8fa7789ad54c524849
2013-08-21 19:05:52 -05:00
Brad Fitzpatrick b24cad68dd Cleanup: remove BlobHub and time.Duration waits from storage interface
Move up a layer to the HTTP.  Also, start to remove ContextWrapper
stuff.  We've done it differently for App Engine instead, and will do
it differently yet moving forward.

Also add blobserver.Receive and use it in most places, moving checksum
verification up a layer.

Bunch of other cleanup and TODO fixing too.

Much simpler and cleaner.

Change-Id: I12e56c5d4e53bfcf82bdd8fb0b6d57c248ff605c
2013-08-21 13:57:28 -07:00
Brad Fitzpatrick 7364fd6110 link to camlipy
Change-Id: I54b16e8f0d6c8216957e9a0655edbd596e3f3036
2013-08-15 10:07:58 -07:00
Brad Fitzpatrick 0bdf20884b all: delete pkg/blobref; convert all from *blobref.BlobRef to new blob.Ref
Change-Id: Id2dfb7f19452bedf4f3c9310b36227fd8117b225
2013-08-03 19:54:30 -07:00
Brad Fitzpatrick cf0d9aca6e More docs
Change-Id: I5c21f240c85bcf91fb67487cc172bf3faeb49fff
2013-07-07 18:52:14 -07:00
Brad Fitzpatrick 953240b870 TODO about cammount
Change-Id: I3f78dbbd7bce1198bde1b6b0c883633dad8d47c2
2013-07-07 15:30:55 -07:00
Brad Fitzpatrick 53c7027160 more TODO
Change-Id: Iade5be2c9f2759790d6c31e62d6a1ec9c1911d9f
2013-07-07 12:23:24 -07:00
Brad Fitzpatrick 26e8b857ce TODO about blobhub notification stuff
Change-Id: I21b6b7baf88ac1972ae03871acd09110c29d8b39
2013-07-07 12:19:26 -07:00
Brad Fitzpatrick de00851493 TODO: blobref representation
Change-Id: Ide46b799d3a134952f6037ad501f2e55dbb361e1
2013-07-07 11:29:10 -07:00
Brad Fitzpatrick c4145082e9 TODO: one added, one removed
Change-Id: I3675048493ff130cdfac22c72a72cad00a0bb7c8
2013-07-07 01:56:44 -07:00
Brad Fitzpatrick 8af0cbd01a add TODO
Change-Id: Ib28e073efdbf4b8f50a729405ae36c9d73007bb8
2013-07-06 14:36:16 -07:00
Brad Fitzpatrick 10d000d900 ui: serve Closure from embedded zip data when available
Change-Id: I9bb6bb4f13f69b293fd98441d82068c0677ffbd5
2013-06-18 23:14:36 -07:00
mpl b198c00cde TODO: "revert" non-typo
Change-Id: Ibbeb4458970f66b5b8aeba3d769d065a43e8cdd9
2013-06-13 01:59:27 +02:00
mpl c31c97186a TODO: typo
Change-Id: Ic9146c8adce1e4a7871dd75fbbd25f051a6980e2
2013-06-13 01:46:09 +02:00
Brad Fitzpatrick 5761b57d69 Work on implementing new static resource plan, as documented in the big TODO item.
Change-Id: I79d7665a24633630c39cceb5e1ab49cbb0300150
2013-06-12 10:10:24 -07:00
Brad Fitzpatrick 5cd09010a3 more TODO
Change-Id: Ie3bf8dc8aa5c19465e85052686693335d66127e4
2013-06-12 09:46:44 -07:00
Brad Fitzpatrick 5b2b2ca267 TODO: file embed notes
Change-Id: Ia0d35ca91f9d21724d90e90788108a21419562bc
2013-06-12 07:27:12 -07:00
Brad Fitzpatrick 8d0a3b57c4 TODO: update make.go status
Change-Id: I2c7ab846a0294db871d170d1c39884cc5fec6c3c
2013-06-12 03:58:42 -07:00
Brad Fitzpatrick f7d7d48cdd Cleanup
Change-Id: Ic0cdae094108bd3b6dca5f7db3005f1e17199de3
2013-06-11 13:27:04 +02:00
Brad Fitzpatrick ebbc943502 Fix environment of dev-camtool.
Change-Id: I671e18cbc07ed9f28afdd3ac7368cd691f44a992
2013-06-09 11:03:48 +02:00
Brad Fitzpatrick 248a30ee9f more TODO
Change-Id: I9a8ffab14e51fc92d6fa4b67b4b28bddc82ea7bd
2013-06-09 10:58:06 +02:00
Brad Fitzpatrick 6a773f903e more TODOs
Change-Id: Ibeb698d87ce474ef7f729ec741e3a89ed3e3ffc0
2013-06-09 10:11:31 +02:00