perkeep/pkg/blobserver/s3
Euan Kemp 90042fe296 pkg/blobserver/s3: use official aws client
vendor: add github.com/aws/aws-sdk-go at rev bc3f534c19ffdf835e524e11f0f825b3eaf541c3

This changes all uses of the s3 client to be the official aws-sdk-go. As
such, it also deletes the previous client maintained in internal.

There are a few notable changes that come with this:

1. An 'aws_region' key is added to the low-level config. The
   'get-bucket-location' call is used to identify the correct region
   rather than using the redirect returned to determine it.
2. Deleting multiple objects at once is much faster (batches of up to
   1000 vs 1 at a time).
3. Retriable errors are retried automatically (see #1184).
4. Newer APIs are used for some operations (v2 / batch apis).

An unfortunate result of 4 is that this is technically a backwards
incompatible change. An IAM policy which used to be sufficient for
perkeep may no longer permit some of the new api calls (even if they're
not doing anything different really).

Fixes #1184, #911

Change-Id: I077fe88def18b7e9a14267820773245e7003634c
2018-07-31 15:51:31 -07:00
..
enumerate.go pkg/blobserver/s3: use official aws client 2018-07-31 15:51:31 -07:00
fetch.go pkg/blobserver/s3: use official aws client 2018-07-31 15:51:31 -07:00
receive.go pkg/blobserver/s3: use official aws client 2018-07-31 15:51:31 -07:00
remove.go pkg/blobserver/s3: use official aws client 2018-07-31 15:51:31 -07:00
s3.go pkg/blobserver/s3: use official aws client 2018-07-31 15:51:31 -07:00
s3_preflight.go pkg/blobserver/s3: use official aws client 2018-07-31 15:51:31 -07:00
s3_test.go pkg/blobserver/s3: use official aws client 2018-07-31 15:51:31 -07:00
stat.go pkg/blobserver/s3: use official aws client 2018-07-31 15:51:31 -07:00