Commit Graph

33 Commits

Author SHA1 Message Date
Brad Fitzpatrick 8fbd54c09b picasa: set url property on photos and albums
Change-Id: If89ee61138e4fb80e68f87130c5fe189b7d82582
2014-07-31 14:34:52 -07:00
Brad Fitzpatrick 1ff2918602 importers: add SupportsIncremental accessor
If an importer is efficient at running regularly, it returns true.

If it's not true, the UI won't allow turning on automatic runs.

Flickr doesn't set it (yet).
2014-07-31 11:34:31 -07:00
Brad Fitzpatrick 4ea387545a picasa: ignore shard number when comparing URLs for equality.
Change-Id: I9e31756f3f8122f7bc563ba2a2f1e19aee553f65
2014-07-30 17:26:46 -07:00
Brad Fitzpatrick d4f22274cd picasa: document and simplify Photo fields in picago.
Be smarter about title vs description. Don't use caption as an attribute name.

Change-Id: I746fc6d88e27b6b85a8505456e6eb423443d7840
2014-07-30 16:24:56 -07:00
Brad Fitzpatrick 95492813b1 picasa: clean up album attributes, add tests, fix keywords with spaces
Change-Id: I32faae16334e3999731f3980502eb6a1a6d38595
2014-07-30 15:50:34 -07:00
Brad Fitzpatrick 606acb5915 picasa: re-use existing permanode if it doesn't conflict 2014-07-29 14:54:43 -07:00
Brad Fitzpatrick 29c63cc2ab picasa, picago: support video, change struct definitions, add start of more tests 2014-07-29 11:44:44 -07:00
Brad Fitzpatrick 1763efbefe picasa: TODO about broken video support
Change-Id: Ie44e1129b1965f379adb8599ffe7f7693fb02eab
2014-07-28 23:15:15 -07:00
Brad Fitzpatrick b4c4aabbd4 picasa: more work towards permanode reuse
Change-Id: Icf590e3593036c5a1a2764a800062f40eada09c1
2014-07-28 23:14:20 -07:00
Brad Fitzpatrick e61c7b949f picasa: start of reusing existing permanode when importing photo
Change-Id: Id43c60f2d5cfea27fb57adb0ca81b621360039b1
2014-07-28 21:35:58 -07:00
Brad Fitzpatrick f5f02768b3 picasa: give a title to importer root permanode
Change-Id: I0e259db52060518dfe33f31f0222b169a8c9e0ef
2014-07-28 19:16:17 -07:00
Brad Fitzpatrick ea6785569d picasa: simplify photo import code, only redownload photo when necessary, do 3 at once.
Change-Id: I99551ad90359ced164b47b12f121ef207b797430
2014-07-28 16:37:19 -07:00
Brad Fitzpatrick 2695422f2e picasa: incremental album import, and key album off its ID, not name.
Change-Id: I748e01950fa22eec024bb08ae22060b6939513dd
2014-07-28 13:58:24 -07:00
Brad Fitzpatrick b8756b73a5 picasa: logging and TODO
Change-Id: Icf010ad8b781dbbf66f03160d49089d8986853a0
2014-07-27 13:13:01 -07:00
Brad Fitzpatrick e8816c072d picasa: cleanups, missing body.Close, start of incremental support, fix SummarizeAccount
Change-Id: I497338743c7d85dea9f374215662543f4a9ad6a9
2014-07-27 12:25:18 -07:00
Brad Fitzpatrick db09f2fe1a Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2014-07-25 16:03:53 -07:00
Brad Fitzpatrick 4021f812c6 importer, picasa: move ExtendedOAuth2 out of pkg importer, fix picasa
The ExtendedOAuth2 code tried to be generic, but wasn't (had Picasa
comments?) and was only used by Picasa anyway.  We can move it later
if there are actually 2 users and it works for both.

At the same time, make Picasa actually work by writing a correct
IsAccountReady function.
2014-07-25 16:02:43 -07:00
Brad Fitzpatrick d0fbfd9751 nodeattr: rename Content to CamliContent, since Content also exists.
Change-Id: I305ca3cecf9b223a125fb9207966995c9dbf83b8
2014-07-25 14:41:17 -07:00
Brad Fitzpatrick 8af06fa3ff picasa: work on making the instructions accurate & work 2014-07-25 13:37:47 -07:00
Brad Fitzpatrick fad3a60a77 picasa: some importer attribute cleanup
Use nodeattr constants. Don't set lat/long when unknown.
2014-07-25 13:24:46 -07:00
mpl 7e86255cac importer/flickr: set DateCreated attribute
Also cleaned up a few vars.
Moved some attributes to nodeattr pkg.

Change-Id: Iea496d50b12e67132fe88f371a89534c2fe32761
2014-07-25 00:57:10 +02:00
mpl 2e939b0b35 importer/picasa: make test data for devcam
http://camlistore.org/issue/417

Change-Id: If1a8771a863e1f7d0797ec5a8e50cd3573718c95
2014-07-23 19:47:45 +02:00
Tamás Gulácsi 925f9d3f86 picasa importer: remove Run-private HTTPClient
A more complete version of Aaron Boodman's
https://camlistore-review.googlesource.com/#/c/3066/
in removing Transports. As Aaron pointed out,
(https://groups.google.com/d/msg/camlistore/9OwQKH6z7rY/840oQrRgsGIJ) it
is illegal for oauth.Transport.Transport to ever be another instance of
oauth.Transport.

So this commit tries hard to use Context's HTTPClient
everywhere and make sure to have its Transport an *oauth.Transport, but
that's Transport is not an *oauth.Transport, but the underlying
Transport.

To eliminate "403: Token not found" errors, we set
oauth.Config.AccessType, oauth.Config.ApprovalPrompt to
"offline", "force".
This rewards us with a non-empty Token.RefreshToken,
and store the full Token in AcctAttrOAuthToken.
This enables us to import continuously without user intervention.

Change-Id: I4defb346abfcf3f6c17597fc2dc23a6e536eda37
2014-07-17 08:12:11 +02:00
mpl 1fd526b999 test/transport.go: move to httputil, so we can use it without linking testing
Change-Id: I889d167a78ef0fba5ef2e9e3e884f594e151e940
2014-07-14 18:33:10 +02:00
Brad Fitzpatrick bafc53703d context: don't allow mutating an existing context's HTTP client
Instead, require specifying it when constructing a new context.

Prevents misuse and bugs / data races.

Change-Id: I775e998bf69b9e238ec726c469494c1a65f8f9de
2014-07-02 14:34:24 -07:00
Brad Fitzpatrick adf538ba7c picasa: fix compilation failure 2014-07-01 11:48:35 -07:00
Brad Fitzpatrick 6fd1550371 picasa: remove useless uplodateDate property
No use storing time.Now().
2014-07-01 11:47:19 -07:00
mpl 77c59e97bb importer/picasa: a couple schema changes
-album: removed Name attribute. It seems like a valid entry from the feed
(picago indeed gets value), but it is not referenced anywhere in
https://developers.google.com/picasa-web/docs/2.0/reference and it seems
to have the same value as Title anyway.

-photo: removed broken (and redundant with camliContent if not broken)
camliPath.

-s/ID/picasaId/

Change-Id: I5389d825eae83b7618f282698a43c94be6ca8df3
2014-06-27 19:21:54 +02:00
mpl 357e405660 importer/picasa: use test.FileResponder, fix test on devcam
We needed to include xml files in make.go

Change-Id: Ibf89a276ebef74d4b03f24934c36c337060226c9
2014-05-29 17:44:48 +02:00
mpl 731a362e27 importers: provide authenticated transport for picasa
And revert context-related changes to importer.go so the other
importers work again.

The problem is picago calls take an *http.Client, so we need to provide
one which already has a transport set up correctly wrt to authentication.
In particular, it needs to have the access token that was obtained
during oauth.

Change-Id: I7b83cd80910111a9abf1c192cc59841f6d818fb7
2014-05-22 15:25:34 +02:00
Tamás Gulácsi cbb777ffb0 Rewrite Picasa importer for the new Importer API
Reorganizes common OAuth2 things into importer/oa2_importers.go
Rewrites CallbackURLParameters to return url.Values.

A controversial workaround for nil context has been left out,
and this may cause panic and will need a proper fix.

Change-Id: I0789f4732412aa559078ec2c6b6797925ab8cd05
2014-05-21 22:37:50 +02:00
Brad Fitzpatrick bf692123bf importer: use context.Context instead of Interrupt type
Change-Id: I79f93c6a14934320022b90500d926453e614d001
2014-03-21 12:27:06 -07:00
Tamás Gulácsi 65fa5d6d3e Picasa: add importer.
Imports all images from all albums, and puts each image in its album.
Each album is linked under "Picasa (<username>)".
Tries to skip already imported files (check based on album/filename).
See Issue #391.

Change-Id: If00751671a429891c05bbe41f59c1421b2ff2de8
2014-03-17 10:52:51 +01:00