Commit Graph

4759 Commits

Author SHA1 Message Date
Brad Fitzpatrick 7bd5e287d4 Merge "Fixed calls to formatting functions" 2014-07-21 17:34:27 +00:00
mpl e68742473d importer/foursquare: minor typo
Change-Id: Ic3c0d2f01ea0e6dc191500ed2856810e8c9581d6
2014-07-22 00:12:51 +02:00
tnt 6868a882b9 Fixed calls to formatting functions
Change-Id: I3ec2af85364674ac6f1a88cd12b6cb369d8e50e1
2014-07-21 22:06:53 +00:00
Brad Fitzpatrick 6b65adbbdd Merge "picasa importer: remove Run-private HTTPClient" 2014-07-18 15:21:41 +00:00
mpl c46d66c3eb Merge "dockertest: skip test when DB in container not ready" 2014-07-17 20:53:10 +00:00
mpl 955cd34404 dockertest: skip test when DB in container not ready
http://camlistore.org/issue/474

Change-Id: Id65b3c35a2f38466c4d7370bad4c5f73a8cb22db
2014-07-17 18:09:02 +02:00
Tamás Gulácsi 925f9d3f86 picasa importer: remove Run-private HTTPClient
A more complete version of Aaron Boodman's
https://camlistore-review.googlesource.com/#/c/3066/
in removing Transports. As Aaron pointed out,
(https://groups.google.com/d/msg/camlistore/9OwQKH6z7rY/840oQrRgsGIJ) it
is illegal for oauth.Transport.Transport to ever be another instance of
oauth.Transport.

So this commit tries hard to use Context's HTTPClient
everywhere and make sure to have its Transport an *oauth.Transport, but
that's Transport is not an *oauth.Transport, but the underlying
Transport.

To eliminate "403: Token not found" errors, we set
oauth.Config.AccessType, oauth.Config.ApprovalPrompt to
"offline", "force".
This rewards us with a non-empty Token.RefreshToken,
and store the full Token in AcctAttrOAuthToken.
This enables us to import continuously without user intervention.

Change-Id: I4defb346abfcf3f6c17597fc2dc23a6e536eda37
2014-07-17 08:12:11 +02:00
Brad Fitzpatrick b062430d99 Merge "fs: handle directory permanode titles" 2014-07-16 18:19:25 +00:00
Brad Fitzpatrick e9de15d596 Merge "make: run 'devcam test' for presubmit target" 2014-07-16 05:14:41 +00:00
Bill Thiede c360867970 make: run 'devcam test' for presubmit target
The previous behavior (now build target oldpresubmit) was slightly different
than devcam test; it didn't copy files to a clean directory and test in
isolation.  The buildbot uses devcam test, and for consistency 'make
presubmit' should too.

This runs devcam test with -short, mirroring -short from the previous
presubmit.  The buildbot runs devcam without -short.

Change-Id: I71e308bd16104004a8cd790901e88edebedab2a9
2014-07-15 22:17:09 -07:00
Piotr S. Staszewski cac9aa4483 fs: handle directory permanode titles
Set corresponding permanode title on new root,
new directory and directory rename.

Change-Id: I5432e14ef57d6c0c5555b11e895e8b8ad6d8efcf
2014-07-15 08:41:05 +02:00
Andrew Gerrand 83a4b39f10 Merge "pkg/sorted/buffer: flesh out, add test" 2014-07-14 22:24:48 +00:00
Andrew Gerrand c15b4c05a7 pkg/sorted/buffer: flesh out, add test
Change-Id: I18f57425ad2e9a4d692d4efe865fea81e8f143e3
2014-07-15 08:24:54 +10:00
Brad Fitzpatrick be651687ba Merge "context: minor typos in documentation." 2014-07-14 20:44:50 +00:00
mpl aa3cda0762 importer/twitter: add TestDataMaker to generate imported tweets
Also fix first user_timeline call (no maxId).

http://camlistore.org/issue/417

Change-Id: I1157331bd876a6400e16d0fd18712d8c459fdc20
2014-07-14 19:26:13 +02:00
mpl e4a31ea4ba Merge "test/transport.go: move to httputil, so we can use it without linking testing" 2014-07-14 17:10:15 +00:00
Brad Fitzpatrick a68f8df380 images: fix broken test
Exif.DateTime now always returns a time.Local location when zone is unknown.

Change-Id: I898516398a88bc8dcd8daf26f87c801a24cc1d7a
2014-07-14 09:36:48 -07:00
mpl 1fd526b999 test/transport.go: move to httputil, so we can use it without linking testing
Change-Id: I889d167a78ef0fba5ef2e9e3e884f594e151e940
2014-07-14 18:33:10 +02:00
Brad Fitzpatrick 077763fcbb schema: use GPS location to find timezone from EXIF when UTC offset is unknown
Change-Id: Ia3590424db36508b491d8f19829738fe102e5c3d
2014-07-13 10:26:54 -07:00
Brad Fitzpatrick 286b53f119 index: use Exif.LatLong accessor
This code is now moved to the exif package.

Change-Id: Ifba2e0b6a96c076e75179528e8ea9a4c0641d843
2014-07-13 10:25:04 -07:00
Brad Fitzpatrick b6d3d1a103 exif: add Exif.LatLong accessor
Change-Id: I5d5d1d3d8124e665d6bb7e6cb7d06107aa39bda7
2014-07-13 10:23:30 -07:00
Brad Fitzpatrick a827380644 exif: make DateTime return time.Local location when unknown timezone
Change-Id: Id3ea677389e3e2e4e765116e85c0c60fb74fdff8
2014-07-13 10:22:45 -07:00
Brad Fitzpatrick 995a4ccb0e indextest: force timezone when testing receive
Otherwise JPEGs with EXIF can be interpretted differently.

Change-Id: Ia3fe64a92d7a8f0642363d758f2d1f402f1e4a1f
2014-07-13 10:04:03 -07:00
Brad Fitzpatrick 76f4ff618d Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2014-07-13 10:02:47 -07:00
Brad Fitzpatrick af4df6f8c5 schema: rewrite symlink test to not delete testdata, not use asserts
Change-Id: I238ee5d55203f5c53b294d82e2cc626c4ef78ef2
2014-07-13 10:02:30 -07:00
Brad Fitzpatrick 49b864a526 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2014-07-12 08:27:20 -07:00
Brad Fitzpatrick 80336a1a71 importer/foursquare: add a few more fields to the API struct.
Not used yet. The shout text at least we should import.

Change-Id: Ib8051e0956c6e7dbd2aee52e0a76d43b46340d20
2014-07-12 08:27:09 -07:00
Brad Fitzpatrick 06cd39ebb8 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2014-07-11 21:09:46 -07:00
Brad Fitzpatrick 3967119923 Import github.com/bradfitz/latlong to third_party.
Change-Id: Ia952a23995b91ac9865bd0ae912bf0bf88289146
2014-07-11 21:08:50 -07:00
Brad Fitzpatrick ab5be97995 Merge "feed importer: meaningful names for permanodes" 2014-07-11 17:08:59 +00:00
Brad Fitzpatrick cc418dac8f Merge "legal: package for compiled-in licenses" 2014-07-11 17:08:59 +00:00
mpl 73fdc7d47f importer/twitter: coordinates bugfix
Change-Id: I089f9125a231277bdabe7ccc29280f62ecd2cc51
2014-07-11 16:53:29 +02:00
Piotr S. Staszewski 5e61e2d84e legal: package for compiled-in licenses
Initial implementation, printing helper and 'wiring' for cam* tools.

Change-Id: Icf8baa77ee1f12495be8f1a57b7d575d6918cb5c
2014-07-10 23:01:54 +02:00
Bill Thiede 0455681bd4 diskpacked: bug fix walkPack and add test.
Replace reindex.go deleted blob.Ref detection loop with a regexp.
Replace diskpacked.go's isDeletedRef with same regexp.

Add a test that verifies the returned blobs match expectations and verifies
multipack diskpacked works..

Add new extension '.blobs' to devcam test's list of suffixes to copy.

Tests fail without the changes to reindex.go.

Change-Id: I0743c1b1ea295324ae3a39e70e6a07e2d6aa70e9
2014-07-09 21:55:04 -07:00
Bill Thiede 59050a893e Revert "diskpacked: bug fix walkPack and add test."
This reverts commit 83d2546d21.

Change-Id: Ia2d622601d68ca5795141de1314244f1e4cd14cb
2014-07-08 17:14:02 -07:00
Brad Fitzpatrick 05f4b0ad07 Merge "diskpacked: bug fix walkPack and add test." 2014-07-05 03:02:58 +00:00
Bill Thiede 83d2546d21 diskpacked: bug fix walkPack and add test.
Replace reindex.go deleted blob.Ref detection loop with a regexp.
Replace diskpacked.go's isDeletedRef with same regexp.

Add a test that verifies the returned blobs match expectations and verifies
multipack diskpacked works..

Tests fail without the changes to reindex.go.

Change-Id: I332c2c3c8c37ebf262ce95e1ec0628146ab5108e
2014-07-06 22:14:07 -07:00
Brad Fitzpatrick 4ac89ec630 Merge "Clarified use of camtool googinit" 2014-07-05 03:02:58 +00:00
Brad Fitzpatrick cc956829f3 Rename android.go files to androidx.go
Go 1.4 (Go tip currently) defines a new GOOS=android, and a GOOS.go file is ignored on
other operating systems. But these android.go aren't actually meant to be only run on
GOOS=android, since we just build Linux binaries on Android for now.
2014-07-08 11:18:40 -07:00
Robert Kroeger f40e2e6e61 Clarified use of camtool googinit
To avoid user confusion, force camtool googinit use to actually
specify if Google Cloud Storage or Google Drive auth setup is
desired.

Change-Id: I6541b4a589de2dda73b725fcaf7bcdadd3f60b65
2014-07-07 08:33:53 -06:00
Piotr S. Staszewski 24ec41b484 feed importer: meaningful names for permanodes
Instead of spamming with "", "Items" and "feed account" permanodes
use the feed's title to set meaningful titles.

Simplified version.

Change-Id: I2f008a30b48a1376e45cdca8d2dfa8b23bf49f40
2014-07-04 16:53:37 +02:00
Bill Thiede b2f5ff81c1 context: minor typos in documentation.
Learned that cancelled is appropriate for non-US English, but the function is
IsCanceled, so I went with consistency.

Change-Id: If63a8af75338780c2f8e81bf4404fc4b5f852d77
2014-07-03 22:05:16 -07:00
Bill Thiede bf6a89c9ad images: add support for fast jpeg down-sampling.
The new package pkg/images/fastjpeg uses djpeg(1) to quickly
down-sample images at load time as described here:

  http://jpegclub.org/djpeg/

Add benchmark comparing std lib jpeg.Decode and fastjpeg.DecodeDownsample
with a factor of 1, 2, 4, and 8.
Benchmark resizes a 4000x4000 pixel JPEG to 128x128.

Refactor in pkg/images to make images.Decode a little more readable.

Change-Id: I571db1f3c3068f99da4a01ec84fd246ef098a18c
2014-07-03 21:06:06 -07:00
Piotr S. Staszewski 98a73d966d twitter importer: meaningful names for permanodes
Make meaningful titles for created permanodes, simplified.

Change-Id: If0575b5c0ffec037feb8ce74e46f66556c582372
2014-07-03 19:27:44 +02:00
Bill Thiede eb7f66fe28 jpeg: enable images/jpeg imported from Go tip.
Addresses https://camlistore.org/issue/463

Change-Id: Ie7b8f937ded78d95875f4cd13b024d0429136981
2014-07-02 21:22:15 -07:00
Bill Thiede c919f12cf1 jpeg: import images/jpeg from go tip.
Imported @ version:

  parent: 20306:04c9a977b540 tip
   runtime: make runtime·usleep and runtime·osyield callable from cgo callback
  branch: default
  commit: (clean)
  update: (current)

Addresses https://camlistore.org/issue/463

Change-Id: I5a38144cb944f6474085d305f86bd2231d7fb0a2
2014-07-02 21:22:02 -07:00
Brad Fitzpatrick bafc53703d context: don't allow mutating an existing context's HTTP client
Instead, require specifying it when constructing a new context.

Prevents misuse and bugs / data races.

Change-Id: I775e998bf69b9e238ec726c469494c1a65f8f9de
2014-07-02 14:34:24 -07:00
Brad Fitzpatrick 9e20a76c99 Fix Makefile's forcefull target to avoid broken targets.
Change-Id: I43073fec6565d52375367dff108dee1ef25fa13a
2014-07-02 14:33:33 -07:00
Brad Fitzpatrick 208c28ce68 Merge "lock: Merge added FreeBSD support from upstream." 2014-07-02 19:49:24 +00:00
Bill Thiede 9d58809e8f lock: Merge added FreeBSD support from upstream.
Change-Id: I2cb6de1fe32069a675eed405744d632054d67a11
2014-07-04 19:21:02 -07:00