perkeep/doc/protocol/blob-upload-protocol.txt

Uploading a blob is done in two parts:

1) a pre-upload "stat" request to both check what blobs are necessary
   to upload in the first place, as well as to get an "upload URL".
   This is mostly due to a broken App Engine limitation, which we want
   to support, but it could also theoretically used for some lame load
   balancing.  In any case, the first reason (checking which blobs the
   server already has) is still valid, especially because the
   alternative (a bunch of HTTP-pipelined HEAD requests) wouldn't work
   in practice as few HTTP servers support pipelining well enough.

2) the actual "upload" (HTTP POST multipart/form-data) containing one
   or more blobs to be uploaded.

============================================================================
Preupload request:
============================================================================

(see blob-stat-protocol.txt)

============================================================================
Upload request:
============================================================================

Things to note about the request:

   * You MUST provide a "name" parameter in each multipart part's
     Content-Disposition value.  The part's name matters and is the
     blobref ("digest-hexhexhexhex") of your blob.  The bytes MUST
     match the blobref and the server MUST reject it if they don't
     match.

   * You (currently) MUST provide a Content-Type for each multipart
     part.  It doesn't matter what it is (it's thrown away), but it's
     necessary to satisfy various HTTP libraries.  Easiest is to just
     set it to "application/octet-stream" Server implementions SHOULD
     fail if you clients forget it, to encourage clients to remember
     it for compatibility with all blob servers.

   * You (currently) MUST provide a "filename" parameter in each
     multipart's Content-Disposition value, unique per blob, but it
     will also be thrown away and exists purely to satisfy various
     HTTP libraries (mostly App Engine).  It's recommended to either
     set this to an increasing number (e.g. "blob1", "blob2") or just
     repeat the blobref value here.

Some of these requirements may be relaxed in the future.

Example:

POST /some/server-chosen/url HTTP/1.1
Host: upload-server.example.com
Content-Type: multipart/form-data; boundary=randomboundaryXYZ

--randomboundaryXYZ
Content-Disposition: form-data; name="sha1-9b03f7aca1ac60d40b5e570c34f79a3e07c918e8"; filename="blob1"
Content-Type: application/octet-stream

(binary or text blob data)
--randomboundaryXYZ
Content-Disposition: form-data; name="sha1-deadbeefdeadbeefdeadbeefdeadbeefdeadbeef"; filename="blob2"
Content-Type: application/octet-stream

(binary or text blob data)
--randomboundaryXYZ--

-----------------------------------------------------
Response (status may be a 200 or a 303 to this data)
-----------------------------------------------------

HTTP/1.1 200 OK
Content-Type: text/plain

{
   "received": [
      {"blobRef": "sha1-9b03f7aca1ac60d40b5e570c34f79a3e07c918e8",
       "size": 12312},
      {"blobRef": "sha1-deadbeefdeadbeefdeadbeefdeadbeefdeadbeef",
       "size": 29384933}
   ],
   "maxUploadSize": 1048576,
   "uploadUrl": "http://example.com/TheNextUploadUrlRandomString",
   "uploadUrlExpirationSeconds": 7200,
}

Response keys:

   received         required   Array of {"blobRef": BLOBREF, "size": INT_bytes}
                               for blobs that were successfully saved. Empty
                               list in the case nothing was received.
   maxUploadSize    required   Integer of max byte size for whole request
                               payload, which may be one or more blobs.
   uploadUrl        required   Next URL to use to upload any more blobs.
   uploadUrlExpirationSeconds
                    required   How long the upload URL will be valid for.
   errorText        optional   String error message for protocol errors
                               not relating to a particular blob.
                               Mostly for debugging clients.  

If connection drops during a POST to an upload URL, you should re-do a
stat request to verify which objects were received by the server
and which were not.  Also, the URL you received from stat before
might no longer work, so stat is required to a get a valid upload
URL.

For information on resuming truncated uploads, read blob-upload-resume.txt