Rooba
/
perkeep
mirror of https://github.com/perkeep/perkeep.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
4,399 Commits 44 Branches 11 Tags 59 MiB
Commit Graph

50 Commits

Author SHA1 Message Date
mpl 8d307672b2 importer/twitter: follow up on little fixes 
Change-Id: I6b7cfac3cf5893226016e819441f4cbbe458b4bf
 2014-04-23 19:20:31 +02:00
Brad Fitzpatrick bf618fca6c importer/twitter: little fixes 
Change-Id: I11ee5c6c5779c5e479120653f4131c0b4edfb2c8
 2014-04-23 08:54:15 -07:00
Brad Fitzpatrick cea6b15013 Merge "twitter importer: fixed to new API" 2014-04-23 15:48:24 +00:00
mpl ef5acd75e3 twitter importer: fixed to new API 
Change-Id: I490001a1f780f1074baa78a4daa563e55096242c
 2014-04-23 16:53:49 +02:00
Brad Fitzpatrick bfd97aefb3 importer/foursquare: few little tweaks, mostly logging. 
Change-Id: Iee200c348ee89bf5ef37b4394d6320bcdcff3842
 2014-04-22 22:38:13 -07:00
mpl 33ba1ad928 importer/foursquare: remove unused oauthConfig 
Change-Id: Ic663ad97ab41515d52e330544a546d58e8cbe583
 2014-04-22 16:43:38 +02:00
Brad Fitzpatrick 317465375a Merge. 
Change-Id: I05f6694a16935d9ef9b293c1be462308b60bfcb7
 2014-04-19 13:02:42 -07:00
mpl 98e895de45 importer/foursquare: remove unused initRoot function 
Change-Id: I74537a28ee31bf5062322aea113453030950a077
 2014-04-18 22:30:09 +02:00
mpl 09a80dff99 importer: doc typo 
Change-Id: Ib8d2a7d1daf318ed2ec2228269942ff480331c2f
 2014-04-18 21:56:31 +02:00
Aaron Boodman a4a41f89fd FS importer: fix filenames for category images and photos. 
Change-Id: I16a1298c077abeadba52b563a336779fa9dce26a
 2014-04-12 16:47:42 -07:00
Aaron Boodman 18eed8251a Custom rendering for Foursquare checkins. 
Change-Id: I12e0bc49810deef1c5c14fe223ecd50c5949ac7c
 2014-04-11 22:21:54 -07:00
Brad Fitzpatrick bfb13b62b9 dummy importer: only register if in dev mode 
Change-Id: I5db990f1c3822e755d7968ddd71939930525e2e9
 2014-04-01 15:11:05 -07:00
Brad Fitzpatrick bff0eefefb importer: update the dummy importer with lots of docs 
Change-Id: I6e7edc778e4bd14ce724ce445dc59ff8d0e0796d
 2014-04-01 11:47:03 -07:00
Brad Fitzpatrick bf2a7b60a3 Importer overhaul. Only Foursquare is currently working. 
The rest are currently stubbed out and will need updating
to the new APIs.

Change-Id: I9d70302b3ac1026192413bf9dcd3c8f1eb420349
 2014-03-31 10:56:30 -07:00
Brad Fitzpatrick a90879bac0 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2014-03-21 13:57:07 -07:00
Aaron Boodman 2802cfb95c Minor cleanup to importer README files. 
Change-Id: Id37ed68b5b8eefe9bc0011452a0e41f70466d87b
 2014-03-21 13:41:16 -07:00
Brad Fitzpatrick bf692123bf importer: use context.Context instead of Interrupt type 
Change-Id: I79f93c6a14934320022b90500d926453e614d001
 2014-03-21 12:27:06 -07:00
Nick O'Neill 2d70c95aef Tweet importing with some basic info 
Change-Id: I73455eef50dcdc495ddc042d266b3a4a8020c877
 2014-03-20 18:57:43 -07:00
Tamás Gulácsi 65fa5d6d3e Picasa: add importer. 
Imports all images from all albums, and puts each image in its album.
Each album is linked under "Picasa (<username>)".
Tries to skip already imported files (check based on album/filename).
See Issue #391.

Change-Id: If00751671a429891c05bbe41f59c1421b2ff2de8
 2014-03-17 10:52:51 +01:00
Nick O'Neill f4f4368a40 Added some icons for checkins and venues. 
Change-Id: I4bfc5b7a8e099643c159ee6cb2532c71b0e62159
 2014-03-11 18:23:13 -07:00
Nick O'Neill 729d5c5851 Import places and checkins from foursquare. 
Change-Id: Ida2e411821f48a88212856398f5fe94caf7acd75
 2014-03-11 17:31:17 -07:00
Brad Fitzpatrick eb27581da7 httputil: add DecodeJSON and CloseBody helpers, then use them everywhere 
httputil.CloseBody closes a http.Response.Body nicely, such that it's
more likely to be re-used.

httputil.DecodeJSON parses JSON from a Response.Body and closes the
body. It also keeps a temporary copy of the input, for error messages.

Then use them everywhere, to improve HTTP TCP connection re-use.

And misc other clean-ups.

Change-Id: I5dc44be0165b13659459ed29ce5ab44c17d9739f
 2014-01-28 22:00:52 -08:00
Brad Fitzpatrick d1f2ae227f Start of Foursquare importer. 
Change-Id: I56b0e71a3aad697b82710ebea6fc941436a3c119
 2014-01-14 20:45:23 -08:00
Aaron Boodman 88973d4871 Clarify confusing log message in Flickr importer 
Change-Id: I9201bd12e7afb31c08b34b4bc4cd7c39567de296
 2013-12-27 00:02:33 -08:00
Dan Pupius bdcf858a0c Flickr importer paginates on sets and photostream 
Change-Id: I921400958e97c97467f6ef4d3eec642b81a8e614
 2013-12-16 21:16:18 -08:00
Brad Fitzpatrick 22101b61d8 flickr: clarify in README that the flag is a devcam flag, not server flag 
Change-Id: Iaab951c326ff90f09b6751f8fe4d439743a5e430
 2013-12-12 15:39:44 +04:00
Aaron Boodman 6a5a2d6d7c Add a README for the flickr importer package 
Change-Id: I018c1574e231cae4119fb55ceb7b63d976db0074
 2013-12-10 21:35:58 -08:00
Aaron Boodman 9855ab281d Flickr importer: import sets and populate with links to photos. 
Change-Id: I6c53dade3e96373fb3fb9ea4b172c53d5e22d988
 2013-11-22 12:21:03 -08:00
Aaron Boodman 08ea4ed13e Flickr importer: Don't refetch or import unmodified photos. 
Change-Id: I2ade0aa70e3caca8c3c1153ffee448d09d819da6
 2013-11-22 10:13:21 -08:00
Aaron Boodman 376fb91747 Merge "Flickr importer: store credentials in Camlistore, not weird JSON file." 2013-11-22 17:50:49 +00:00
Aaron Boodman 93b0bbbee5 Flickr importer: store credentials in Camlistore, not weird JSON file. 
Change-Id: I07ed4e9043c9fa448425144886dc3b7047c4e85b
 2013-11-22 09:50:03 -08:00
Brad Fitzpatrick ed889d38be dummy importer: upload a file and make a node. 2013-11-21 17:51:13 -08:00
Aaron Boodman 7c3a7bd30a Ugh. Make importer.Object.SetAttr() actually ignore duplicate values. 
I'm going to spend some time writing unit tests for this next I think.

Change-Id: Id19f24882f33adea4060ff852a986626ea8de171
 2013-11-20 23:57:11 -08:00
Aaron Boodman 2a747ccd56 Add ability to specify flickr config for devcam. 
Change-Id: Ic1139730d969558e2d6375dd86c6741e3dfd2b9b
 2013-11-18 21:16:09 -08:00
Aaron Boodman e3d71c34d1 Actually import the first 100 photos from Flickr. 
Currently this just imports the title, description, and image data.

Change-Id: I95de17cddb0ac18eec05f8ee288764b04cd7f406
 2013-11-17 21:58:06 -08:00
Brad Fitzpatrick bd11fd5cb9 importer: some TODO notes 
Change-Id: Ia92be507092722d5751f42ec3c21e18cfc4cb7f0
 2013-11-17 15:00:46 -08:00
Aaron Boodman de9e5103e7 Grab the userID from the flickr oauth handshake. 
Remove the mutex from auth.go because I believe the
mutex in importer.go should guaranteeing only one
flickr importer is running at a time.

Change-Id: Ice971c6c041cf21fdb2bc93cb15f861dd1c8c1d7
 2013-11-17 14:26:42 -08:00
Aaron Boodman fde2936507 Remove some unnecessary code from dummy importer. 
Change-Id: Ifd1cb70f6f5e11261353e8fe16d17c348ea95f12
 2013-11-16 22:43:17 -08:00
Aaron Boodman 9ae23e52b0 Rearrange the Flickr importer to use OAuth instead of Flickr's custom importer. 
Change-Id: I8284431fa6572a08d3c7ff0d0e655297627a4eda
 2013-11-16 22:30:21 -08:00
Brad Fitzpatrick f38f40c453 importer: flesh out most of the Host/Object interface 
Change-Id: Ia7e6ac3aa5e86bfa1fe950ad0c3944989c1d3f8a
 2013-10-25 00:23:29 -07:00
Brad Fitzpatrick 0324143892 importer: make the schema.Signer available to the importer 
Change-Id: Iad1a2f3bfe381b31a381957edb85d6d4aa21b38c
 2013-10-24 22:45:28 -07:00
Brad Fitzpatrick 353c43c02a wip 
Change-Id: I7cdafbd53b59fc53dafa4c19aa391f51f6982061
 2013-10-21 20:17:57 -07:00
Brad Fitzpatrick 8695ac5ca4 importer: partial API sketch. 
Change-Id: I05adf61164e4aabca35a1bb0ff1f0c8c0fd34ef1
 2013-10-20 20:54:43 -07:00
Brad Fitzpatrick 8a56f3744d importer: add accessors for Target and Search handler 
Change-Id: I49cd2a1fc08547eada21cee1506f190fe8f6d6de
 2013-10-20 12:24:20 -07:00
Brad Fitzpatrick c512e3d1be Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-10-20 12:00:22 -07:00
Brad Fitzpatrick 35e77c45ad importer: initialize Host's search handler and blobserver target 
Change-Id: I482e6f192620c254648b19b3044eedbd44de288c
 2013-10-20 12:00:15 -07:00
Aaron Boodman e4f25a2444 First little bit of Flickr importer. 
Change-Id: Ia42b07a81b01904f5722d06653c4bf9de2e81c64
 2013-10-20 11:18:53 -07:00
Brad Fitzpatrick e84d5865cf importer: rudimentary manual start/stop of importers 
Change-Id: I753f3d6e4a241c803f79446889169c4a25f1bdf3
 2013-10-19 16:59:46 -07:00
Brad Fitzpatrick cf116c6da9 importer: work on infrastructure for third-party import from other sites 
Change-Id: Ib4e5e4d543a1b2a913b99fa0a4d92f1a5b94c5dc
 2013-10-19 15:48:05 -07:00
Brad Fitzpatrick 6866b68024 importer: start of a third-party importing system. 
As discussed with Ryan Barrett.

Change-Id: I15773ee9cf3566214afbce947104be1584f6e600
 2013-01-20 16:17:32 -08:00