A working importer for the Instapaper service. The importer imports
bookmarks, full text articles, and highlights. A blob item handler for
Instapaper highlight permanodes is included to show summarized highlight
content from the server UI.
Also, this CL updates github.com/garyburd.com/go-auth to rev
bca2e7f09a178fd36b034107a00e2323bca6a82e in order to get support for
XAuth requests.
Fixes#1208
Change-Id: I72e0c40b245c7eec4a44bb475fcaa96a0ee9a1c5
vendor: add github.com/aws/aws-sdk-go at rev bc3f534c19ffdf835e524e11f0f825b3eaf541c3
This changes all uses of the s3 client to be the official aws-sdk-go. As
such, it also deletes the previous client maintained in internal.
There are a few notable changes that come with this:
1. An 'aws_region' key is added to the low-level config. The
'get-bucket-location' call is used to identify the correct region
rather than using the redirect returned to determine it.
2. Deleting multiple objects at once is much faster (batches of up to
1000 vs 1 at a time).
3. Retriable errors are retried automatically (see #1184).
4. Newer APIs are used for some operations (v2 / batch apis).
An unfortunate result of 4 is that this is technically a backwards
incompatible change. An IAM policy which used to be sufficient for
perkeep may no longer permit some of the new api calls (even if they're
not doing anything different really).
Fixes#1184, #911
Change-Id: I077fe88def18b7e9a14267820773245e7003634c
A new "put" mode is added to the pk command, so that the "pk put"
command can be used to create and upload blobs.
What this command does is actually just call the previously named
"camput" executable, which is renamed to "pk-put" in this change.
This involves adding a new way to register a mode in cmdmain, when such
a mode is just meant to call an external binary. To emphasize the
distinction, the existing func (to register a sub-command, or a mode) is
renamed from RegisterCommand to RegisterMode, and RegisterCommand is now
the name of the new func/way.
Updates #981
Updates #1056
Change-Id: Ief954c17aa88a376f551df7de4b4e9fe41ad96d1
DBNames is supposed to provide configuration for the various databases
names. However,
1) I contend that nobody needs or wants to configure them as long as we
provide sane defaults.
2) it seems the only obvious user we have for this is to set up some of
the names on GCE.
3) having another external source for names complicates the code
further, especially when we already have the distinction between
database names for DBMS and file names for file-based databases.
4) writing a correct documentation for it is awkward.
Therefore, in this CL, I propose that we remove DBNames. Instead,
genconfig.go now sets some consistent default names for the various
queues and indexes set up on a DBMS (MySQL, PostGres, Mongo). To that
end, we introduce the new, but optional, DBUnique configuration
parameter, that is used as a part of all the database names, in order to
be able to run several Perkeep instances on the same DBMS, without name
conflicts.
In addition, the queue for the bs->index synchandler is now set up on
the same DBMS that is already in use for the index itself, instead of
using a file-base database.
And i think we could proceed likewise for the other queues.
Fixes#951
Change-Id: Ib6a638f088a563d881e3957e4042e932382b44f4
These improvements on the server app handler should help writing
and running stand-alone apps.
The two main goals are:
1) "simple" configurations should work automatically; the parameters for
the app are derived from the Listen and BaseURL of the Camlistore
server.
2) More advanced configurations, such as being behind a proxy, should be
easily configurable through the app's Listen, BackendURL, and ApiHost
parameters.
I had worked on them while doing the scanning cabinet app, and I am
backporting them now since we haven't landed the scanning cabinet yet,
and people have been having trouble setting up the publisher.
pkg/app/app_test.go is gone because app.ListenAddress is now dumb. The
hard work is done in pkg/server/app instead.
Fixes#818
Change-Id: Ice2610d6bac611b209cc3a928e67fa6093a41d3e
GitHub treats the CONTRIBUTING file special, in that it is referenced
any time someone files a new issue or starts to create a pull request.
Also make minor updates to file to use markdown and current hacking
instructions.
Change-Id: I68f0d7a69397851ee6c65671ae692cbfb2b93946
Ian Denhardt