Commit Graph

7065 Commits

Author SHA1 Message Date
mpl 7247d11107 pkg/test/docker: force the MySQL version to 5
Since we don't support the latest MySQL version, our tests in Travis CI
are failing since they pull the latest docker image.

We should fix the issue
(https://github.com/perkeep/perkeep/issues/1114), but in the meantime,
we might as well restrict the tests to MySQL 5, so that CI tests keep on
being useful to alert us about other errors.

Change-Id: I32a2cc1dc5a4194c891183d96498dee14c759baf
2018-04-24 19:45:27 -07:00
Mathieu Lonjaret 03b0ab4601 Merge "website: add links to android app" 2018-04-25 01:53:10 +00:00
mpl e02c044e67 website: add links to android app
Change-Id: I90ed3cbe168b95dfaeaa674b330c1b704145f6e6
2018-04-24 18:51:58 -07:00
Mathieu Lonjaret 926a324800 Merge "clients/android: add steps to sign the apk" 2018-04-25 01:48:26 +00:00
mpl 1aeadf4a8a pkg/index: on slurping bogus gps coordinate, log instead of panicking
If a GPS coordinate is bogus, we should have caught it when receiving
the blob, and not write it out as an index row.
If we did not, we end up catching it when trying to slurp it to the
corpus. At this point there's no reason to penalize the user by
panicking, we can instead skip it and log about the error.

Change-Id: Ie40479befca78d8b5da343ef1a8420f703f07cb7
2018-04-24 17:53:33 -07:00
Brad Fitzpatrick 8a67582cf9 blobserver/{files,localdisk}: add VFS layer for use by localdisk
For now, no user-visible changes.

But this will permit an SFTP blobstorage layer in the future.

Next step will be moving 90% of the code from the localdisk package
into the files package.

Updates #1111

Change-Id: I62b924e3d69ca47e7c0fa83c78a77808a71ea33e
2018-04-24 16:30:08 -07:00
mpl e35db56405 clients/android: add steps to sign the apk
Change-Id: I776dd14efca4bd10263a7c7422e0ffd5c706a523
2018-04-24 16:10:05 -07:00
mpl 66e6766571 pkg/gpgchallenge: use long form when looking for key ID
We migrated to using the long form for key IDs sometime ago, but we
never updated the client of the gpgchallenge accordingly, so in
functions that take a keyID as an argument, it would fail to find the
given long keyID within a key ring.

However, for usability reasons, we want to keep the short form in the
hostname of the instance (and hence as the DNS entry), so we revert to
using the short form when perkeepd is setting its host name.

Change-Id: I8373016671fdead32780a04f6d64045e81cc9cd1
2018-04-23 21:44:25 -07:00
mpl 4a562043e0 pkg/sorted/sqlkv: make multiple close safe
Change-Id: I3d0f81e201a944b43bfd484e6393aed7cc52b7e3
2018-04-23 17:44:59 -07:00
mpl 2286b6442b website: move go code to pk-web package
Fixes #1109

Change-Id: Ibcba1addf25da414f4d9cb42f621d6b9184898da
2018-04-23 16:42:35 -07:00
mpl a2760b9767 pkg/deploy/gce: rename tarball name with perkeepd
Change-Id: I2dad2c783cdac7bd995f00e1ff22fc37d8bbb358
2018-04-23 14:06:48 -07:00
mpl 4ed26b3d04 misc/docker: fix expected output binaries location
Change-Id: I3f7f30580a513602e0091269163af9618926623a
2018-04-23 13:59:06 -07:00
mpl 3dc471f790 misc/docker: bump go image to 1.10.1
Change-Id: I993ff0606e0cbd6a6e816df35e5ca4eda3896cfc
2018-04-23 13:47:56 -07:00
Brad Fitzpatrick b5392b7d3b Merge "make.go: validate that make.go is run in GOPATH" 2018-04-23 03:57:55 +00:00
Mathieu Lonjaret 36bb53527f Merge "cmd: rename camget to pk-get, and make it a mode of pk" 2018-04-23 03:52:06 +00:00
mpl 7a27eb42eb cmd: rename camget to pk-get, and make it a mode of pk
A couple of related fixes and renames too.

Updates #981
Fixes #1056

Change-Id: Id47a933c77422edfb9db41d34c38ed9d0d7a1846
2018-04-22 20:50:54 -07:00
Brad Fitzpatrick 05684c3dc1 make.go: validate that make.go is run in GOPATH
It used to be a feature of make.go that it could run from anywhere and
users could ignore GOPATH. We would still like that, but the
shenanigans we did before to mirror the perkeep source tree to a temp
directory and set GOPATH to that temp directory were annoying. And
fortunately GOPATH is slowly dying, so we won't need to deal with this
forever. As of Go 1.8, there is always now a default GOPATH
(https://golang.org/doc/go1.8#gopath), so at least people don't need
to set environment variables. And in Go 1.12-ish, we'll probably ditch
GOPATH entirely.

So, for now require that users build in their GOPATH, which might be
the implicit GOPATH if they haven't set one. This only affects
developers. Regular users can use release binaries.

Change-Id: Ib4e3f05fa987baeaba5aa51349887e6c243f0092
2018-04-22 20:16:51 -07:00
Mathieu Lonjaret 78e60919f7 Merge "make.go: use first value of GOPATH" 2018-04-23 02:44:14 +00:00
Brad Fitzpatrick 01af2e3134 Merge "internal/osutil: move the config dir to perkeep" 2018-04-22 20:16:48 +00:00
Mathieu Lonjaret c87c026e7e Merge "pkg/test: do not expect binaries in ./bin" 2018-04-22 18:59:48 +00:00
mpl e778290117 pkg/test: do not expect binaries in ./bin
As make.go now does not install binaries in ./bin, but rather let them be
installed wherever go install does, the test World should now rely on
os/exec's LookPath to find the binaries it needs.

Fixes related failing tests

Change-Id: I256e79673ca994001e831aed9e7a3ab5ba485ead
2018-04-22 11:59:12 -07:00
Brad Fitzpatrick a062b701c9 all: more log spam & logging consistency cleanup
Change-Id: Ibc38c2eed86e75afb064ba25ba586494c813f56c
2018-04-22 11:30:02 -07:00
Brad Fitzpatrick ab797685bf importer: rename foursquare to swarm, add Properties to Importer interface
Foursquare was rebranded Swarm (for check-ins) some time ago, so
rename our implementation to match the upstream branding. But still
store "foursquare" as the permanode importer type, to not break old
users. That override is now added to the new Properties struct, and
all importers now return their properties.

More stuff will be added to Properties later: WIP flag, bug link,
proper title, icon?

Change-Id: I31cbe2feec3dbf9c6bdb0c866056d9c6966313e3
2018-04-22 11:29:50 -07:00
mpl b27ac3a71b make.go: use first value of GOPATH
Fixes #1101

Change-Id: Ia6993d81f93b8aa64582d1b265d094fc7b232fc7
2018-04-22 11:05:27 -07:00
Brad Fitzpatrick e5e29b004b dev/devcam: don't invoke make.go four times for four targets
Just once is fine.

Change-Id: I74b34f34ee5a193dcf7ac7d28acc858fd6263a68
2018-04-22 09:01:40 -07:00
Brad Fitzpatrick 38d0075c3a all: make log lines a bit more consistent
Change-Id: I06c5bbe072c9857ca3afbf97d14146b9cd96a49e
2018-04-22 08:51:43 -07:00
Mathieu Lonjaret 892dd7b0f3 Merge "pkg/search: allow empty expression" 2018-04-22 04:42:13 +00:00
Brad Fitzpatrick 915299b8b6 importer/twitter: remove some log spam, clean up a comment
Change-Id: I5709a27424a56fafe84f43358b1a735c496fedd1
2018-04-21 21:02:38 -07:00
mpl aaac9635d2 pkg/search: allow empty expression
We had checks in place to return an error if a search query had both
no constraint and no expression. However:
1) we accepted " " as a valid search expression (which is the
work-around that was used by the web UI for the index page)
2) the rest of the code seems to be coping well if we remove these
checks, which this CL does.

This allows us to remove the q=" " trick in the web UI, which does not
seem to be breaking anything, and has the side-effect of improving the
map aspect story (issue 1081). Because with this change, the index page
is not considered anymore as the result of a search query, which means
if one then clicks on an image, one then gets to the image permanode as
a "standalone" item, instead of as one item part of set.
As a consequence:
1) If one then clicks on the map aspect, one gets only this item on the
map (with the proper zoom level), instead of all the items from the
index page.
2) The other side of the coin is left/right navigation does not "work"
anymore in this context, since the item is not part of a set anymore.

In other words, this change makes the index page (i.e. the empty search)
a somewhat particular case, and makes it behave a little differently
from the result of any other (non-empty) search.

Updates issue #1081

Change-Id: Ibbb1711c67c7cf8adaf94ab5d2c68c3755e4983b
2018-04-21 17:44:43 -07:00
mpl 2bb666ccf6 all: rename remaining occurrences of camput
Also removed misc/buildbot while at it (which contained camput
references) since we don't use it anymore at all.

TODO: the OSX app seems to be relying on finding a binary in ../bin,
which we do not use anymore. This will probably need fixing.

Updates #981

Change-Id: I14220fbad2e81181330fca4bb2d2e5fe170e1bd6
2018-04-21 16:20:24 -07:00
Brad Fitzpatrick 9ce352249c internal/osutil: move the config dir to perkeep
Updates #981

Change-Id: I71ceea6395d31851e8502a2f397acb22056fe1b0
2018-04-21 12:50:24 -07:00
Mathieu Lonjaret 86ee09b012 Merge "make.go: fix cross compilation" 2018-04-21 19:45:35 +00:00
mpl 9035ba9f25 make.go: fix cross compilation
When installing, respect user defined GOBIN and/or GOPATH, or the
Go defaults.
But when cross-compiling, disable GOBIN, as it's not supported.

Fixes #1100

Change-Id: I69485722d6dbfcbf7721463077f828efc950d9d3
2018-04-21 21:44:05 +02:00
Brad Fitzpatrick 46f31ba425 server/perkeepd/ui: rename UI menu from Camli to Perkeep
Also disable the annoying mouse hover enter/exit behavior. Require
clicks. TODO: more focus events elsewhere should close the menu, but
this is better than before.

Also, rename the PIGGY variable to KEEPY for now, in prep for changing
the logo from a pig to the Keepy the Perkeep Parakeet mascot.

Updates #981

Change-Id: I48d9dd7267701b39122836aaad708726b1c105ac
2018-04-21 12:37:52 -07:00
Brad Fitzpatrick 614b3f4cee internal/images: use tmpfs for heiftojpeg's /tmp dir for ffmpeg scratch space
Change-Id: Ia1691e17fb8bd95dfb3333aab67bd180a4d6fe58
2018-04-21 12:06:05 -07:00
Brad Fitzpatrick 54578ea062 pkg/blobserver/blobpacked: log before starting integrity check
Fixes #1097

Change-Id: I3abea84a9cee090309098634e655721272386092
2018-04-21 12:05:41 -07:00
Brad Fitzpatrick ca76a40bbc Rename camlistored to perkeepd.
Updates #981

Change-Id: I8fe43c240c149074c23128a89ab426af9cbf94b4
2018-04-21 11:06:09 -07:00
mpl ce4658abfc cmd: rename camput to pk-put, and make "pk put" call it
A new "put" mode is added to the pk command, so that the "pk put"
command can be used to create and upload blobs.

What this command does is actually just call the previously named
"camput" executable, which is renamed to "pk-put" in this change.

This involves adding a new way to register a mode in cmdmain, when such
a mode is just meant to call an external binary. To emphasize the
distinction, the existing func (to register a sub-command, or a mode) is
renamed from RegisterCommand to RegisterMode, and RegisterCommand is now
the name of the new func/way.

Updates #981
Updates #1056

Change-Id: Ief954c17aa88a376f551df7de4b4e9fe41ad96d1
2018-04-21 10:26:55 -07:00
Brad Fitzpatrick a13abdeb8c cmd/camput: add flag to specify hash function for raw blobs
Also a bit more logging around indexing in debug mode.

Change-Id: I2eb67cfec12cff102ba64b17de0369bde38e416a
2018-04-20 21:02:43 -07:00
Brad Fitzpatrick 27abfcced5 cmd/pk: make "search" command highlight position of JSON syntax errors
Change-Id: Ie63b32f9dee403b9a985e45a646ae1ece5d2cc7c
2018-04-20 20:30:18 -07:00
Brad Fitzpatrick 61de9881da pkg/index: redo indexing of a blob when CAMLI_REDO_INDEX_ON_RECEIVE is set
So an upcoming HEIC reindexing tool.

Updates #969

Change-Id: If194904d770bb670fa581c4e5e09c303806bc784
2018-04-20 19:40:11 -07:00
Brad Fitzpatrick 6af54cc882 misc/docker/heiftojpeg: rearrange Dockerfile, add README
Change-Id: Ic5bdbeca75ede15cfdbd569c533f66c24810cdae
2018-04-20 14:06:15 -07:00
Brad Fitzpatrick a19c831ed6 misc/docker/heiftojpeg: switch to our fork with rotation support
Fixes #1094
Updates #969

Change-Id: I75f6b7433bbc2dac1ccc800696969e18164a514d
2018-04-20 13:55:34 -07:00
Brad Fitzpatrick 7b5b98237d Merge "search: support wildcards in filename search" 2018-04-20 20:43:32 +00:00
Brad Fitzpatrick 973df545ef search: support wildcards in filename search
Change-Id: Ic8a1b2cdad3215f7e850bcc47467e0b4f8e92c4f
2018-04-20 13:36:43 -07:00
Tilman Dilo 4eff07253a make.go: handle genfileembed stderr output again
Due to a regression introduced in f8ef262f, the stderr output of
genfileembed was not handled anymore. This caused the build to hang on
Windows systems.

This change introduces a buffer to contain all of genfileembed's stderr
output (usually a long list of files). The buffer's contents are printed
when the command fails to run successfully or when verbose build mode is
enabled. Running of the command is simplified by just calling Run()
instead of Start() and Wait(). The -output-files-stderr flag for
genfileembed, obsolete since f8ef262f, is removed.

Fixes #1099

Change-Id: Ibd8a124c73bf347b92acfc85fffd1186dfe4944b
2018-04-19 19:34:42 +02:00
mpl 951e9a45bf vendor: add golang.org/x/tools/go/buildutil
dep had computed it was needed, but apparently if I have it in my GOPATH
(and not in vendor), then building works (which seems weird?), which is
what made me miss it previously.

Change-Id: I979a107aea735e4f505882e4230a20c74eb5df04
2018-04-18 19:02:16 +02:00
Mathieu Lonjaret b1e655219c Merge "pkg/index: read EXIF bytes when HEIC file" 2018-04-18 15:40:49 +00:00
mpl 3014bd7413 pkg/index: read EXIF bytes when HEIC file
Updates #969

Change-Id: I11cc1668b853fe5bc2d076addda1a60c08be1dc5
2018-04-18 17:39:38 +02:00
Brad Fitzpatrick b489cfa6bf Merge "internal/images: reset err to nil on no EXIF found" 2018-04-18 01:51:09 +00:00