Commit Graph

164 Commits

Author SHA1 Message Date
Brad Fitzpatrick 98dff16008 clients/web/embed/closure: use modern go:embed instead of old bespoke make.go hack
Signed-off-by: Brad Fitzpatrick <brad@danga.com>
2023-12-31 12:15:42 -08:00
Brad Fitzpatrick bec657b7e2 Remove legacy +build comments; use go:build only
Signed-off-by: Brad Fitzpatrick <brad@danga.com>
2023-12-31 09:18:39 -08:00
Oleksandr Redko 001c417e73
all: replace deprecated io/ioutil with io and os (#1647) 2023-01-23 10:25:14 -08:00
Michael Hoffmann ad7454c304 misc: add race flag to devcam and fix fallout 2023-01-13 07:48:33 -08:00
Brad Fitzpatrick 921894783b all: gofmt for Go 1.19
Signed-off-by: Brad Fitzpatrick <brad@danga.com>
2022-12-31 22:45:40 -08:00
luz paz d605f86c82 Fix various typos
Found via `codespell -q 3 -S ./clients/web/embed,./clients/chrome -L ba,everytime,impres,keypair,msdos,pres,ro,te,ue`
2022-05-02 14:31:09 -07:00
Tamás Gulácsi c986ee3c62
Use modernc.org/sqlite (#1581)
* Use modernc.org/sqlite

This way it can be enabled by default, as it is a cgo-free, Go-only package.
No need for build tags, conditional compilation (whether libsqlite3-dev is installed).

* make.go: Remove unused -sqlite flag

* Remove use of -sqlite flag
2021-12-27 12:18:08 -08:00
Tamás Gulácsi 1c400bd5b3 Remove unused code
as `staticcheck -checks=U1000` suggested.
2021-08-09 17:34:14 +02:00
Alexandre Viau dd996d05f5
doc: document all environement variables (#1383)
envvardoc: repair the script so that it does not fail on very long
lines.

envvardoc: look for env vars with the `PERKEEP` prefix too.

envvardoc: adapt the `docVar` regex to the markdown format.

envvardoc: also look for vars in internal/.

CI: run envvardoc so that we notice when we add new undocumented
environement variables.

doc: List all remaining undocumented env variables. This "shitlist"
gives us a green build in the CI and allows us to ensure all new
environment variables are documented in the future.

doc: Link to the environment variables page.
2021-07-29 14:22:27 -04:00
Alexandre Viau dd14e9ea06
devcam hook: fix bad script path (#1381)
This would create the hook at `.git/hookspre-commit` instead of creating
it at `.git/hooks/pre-commit`.
2021-03-28 20:08:08 -04:00
Brad Fitzpatrick 41f5d4a190 Stop using Gerrit (googlesource.com), switch to just GitHub & PRs
Discussed at https://groups.google.com/d/msg/perkeep/terI3u9hfqg/-TWI0dOyCQAJ
2019-06-05 09:50:11 -07:00
Mathieu Lonjaret aa7ab23772 Merge "pkg/importer: add "instapaper" importer for Instapaper service" 2018-12-03 15:07:30 +00:00
Angel 3dbb01138f pkg/importer: add "instapaper" importer for Instapaper service
A working importer for the Instapaper service. The importer imports
bookmarks, full text articles, and highlights. A blob item handler for
Instapaper highlight permanodes is included to show summarized highlight
content from the server UI.

Also, this CL updates github.com/garyburd.com/go-auth to rev
bca2e7f09a178fd36b034107a00e2323bca6a82e in order to get support for
XAuth requests.

Fixes #1208

Change-Id: I72e0c40b245c7eec4a44bb475fcaa96a0ee9a1c5
2018-11-26 07:57:08 -05:00
mpl 5d12c9aa0d devcam: create hooks dir if it does not exist
Which can apparently happen with the way gitlab clones our repo.

Change-Id: I895c28647fd211ef7beea2e20ae51b866467f81c
2018-11-22 02:22:09 +01:00
mpl ca7f2e57e3 dev/devcam: use libsqlite3 tag when -sqlite=true
In 5894af5196 we chose not to vendor
sqlite3-binding.c and sqlite3-binding.h in, because we rely on the
system's libsqlite3 instead. But we never told devcam test to use the
corresponding necessary "libsqlite3" tag, which is now done in this CL.

Also, comment properly about it in .gitignore.

And make Travis CI use the devcam installed in $GOPATH/bin, instead of
specifically building one in tree.

Change-Id: I4b7bb1305669037f84063c7cf6c7f356c99f3edc
2018-09-14 22:16:23 +02:00
mpl 4e7b447f6e devcam mount: look for pk-mount in GOPATH
Change-Id: I6b06d76b02c4e66c64fbfe4fd54ce2e3a51ef494
2018-08-24 03:16:57 +02:00
mpl 0888d28260 all: fix misc bugs found with go test ./...
Change-Id: If5bda860c9cc6bdc14c2977646516c58d17d62de
2018-08-17 02:17:54 +02:00
Euan Kemp feef4b6b0d devcam hook: correct pathspec for older gits
Older git versions (prior to 2.13 / commit
859b7f1d0e)
required a more explicit pattern.

This also reorders the gofmt one to put the positive before the negative
match for consistency.

Fixes #1191

Change-Id: Ie74e3c3d434f667d76d5ebbb76811b4f304800c8
2018-06-18 18:23:28 -07:00
Euan Kemp f37c7ca7ac devcam hook: ignore vendor in whitespace check
While I was there, I also switched gofmt's vendor ignoring logic to use
git's builtin pathspec filtering.

Change-Id: Iab8093d795e7f967cceef351c6a593ba776d9790
2018-06-11 01:38:31 -07:00
mpl 64f77af5b6 devcam: review: fix origin in git config
Fixes #1176

Change-Id: I18222fd7b2d9ac47d4ab01740ef55b14d44b8b11
2018-05-26 02:16:32 +02:00
Brad Fitzpatrick a884a9181e Merge "devcam: add -branch option to review mode" 2018-05-19 02:44:41 +00:00
mpl f95bd2a593 devcam test: add internal and app/publisher to the list of tests
Fixes #1138

Change-Id: I71f46da09fb9d14c7a3648d6da901ac712466f72
2018-05-15 19:36:14 +02:00
mpl 94461e25ca devcam: add -branch option to review mode
Change-Id: I0c7d78bcb2ef357368db93751a8bb7a7977e528f
2018-05-02 15:25:29 -07:00
Brad Fitzpatrick b1abccd287 devcam, localhost, storagetest: fixes for Windows
Unset CGO_ENABLED if no sign of gcc.

The localdisk renaming stuff was fixed in Go ages ago in
golang/go#13673 and https://golang.org/cl/6140

And a defer in storagetest meant Windows couldn't delete files
because a file was still open.

Change-Id: I57aef85f24653b19ce10e3d1e18c778cee2d48f6
2018-05-01 10:46:19 -07:00
mpl a6ad4d88e9 dev/devcam: do not look for pk in ./bin dir
Change-Id: I1903b389fbed1bf836d4d3c91c5fb6d2db117b84
2018-04-25 22:51:41 -07:00
mpl b75523bbea dev/devcam: fix website go code location
also do not set GOBIN when building self

Change-Id: Ibea2ab4fcc7b60ef28dfcb6620a44b29541eedef
2018-04-25 12:19:10 -07:00
mpl 7a27eb42eb cmd: rename camget to pk-get, and make it a mode of pk
A couple of related fixes and renames too.

Updates #981
Fixes #1056

Change-Id: Id47a933c77422edfb9db41d34c38ed9d0d7a1846
2018-04-22 20:50:54 -07:00
Brad Fitzpatrick e5e29b004b dev/devcam: don't invoke make.go four times for four targets
Just once is fine.

Change-Id: I74b34f34ee5a193dcf7ac7d28acc858fd6263a68
2018-04-22 09:01:40 -07:00
mpl 9035ba9f25 make.go: fix cross compilation
When installing, respect user defined GOBIN and/or GOPATH, or the
Go defaults.
But when cross-compiling, disable GOBIN, as it's not supported.

Fixes #1100

Change-Id: I69485722d6dbfcbf7721463077f828efc950d9d3
2018-04-21 21:44:05 +02:00
Brad Fitzpatrick ca76a40bbc Rename camlistored to perkeepd.
Updates #981

Change-Id: I8fe43c240c149074c23128a89ab426af9cbf94b4
2018-04-21 11:06:09 -07:00
mpl ce4658abfc cmd: rename camput to pk-put, and make "pk put" call it
A new "put" mode is added to the pk command, so that the "pk put"
command can be used to create and upload blobs.

What this command does is actually just call the previously named
"camput" executable, which is renamed to "pk-put" in this change.

This involves adding a new way to register a mode in cmdmain, when such
a mode is just meant to call an external binary. To emphasize the
distinction, the existing func (to register a sub-command, or a mode) is
renamed from RegisterCommand to RegisterMode, and RegisterCommand is now
the name of the new func/way.

Updates #981
Updates #1056

Change-Id: Ief954c17aa88a376f551df7de4b4e9fe41ad96d1
2018-04-21 10:26:55 -07:00
Brad Fitzpatrick dd3516c6ee Remove some ancient Perl scripts.
Change-Id: If98021d39eb8560913bc1ba35f4cb7f29f25974c
2018-04-17 15:11:07 -07:00
mpl f8ef262ff9 make.go: clean up, notably remove the tmp source code mirroring
Fixes #1004

Change-Id: Iea01cfc601b8dd8f2a81fe5da40cdd272b164bb1
2018-04-17 19:39:46 +02:00
mpl 6dcbe2ee9c cmd/camtool: rename from camtool to pk
Updates #981
Updates #1056

Change-Id: Ib78c47a69fa9816d5ac6cc547586a2af321005e3
2018-03-06 16:20:59 +01:00
mpl b89bf256d1 pkg/search: start to make funcs use a GPG key ID instead of blobRef
Introduction of an Owner type  in pkg/index.
Related changes in pkg/index and pkg/serverinit.

Fixes #1047

Change-Id: Ifc316865833349d5202b5085e8f2c1235f3f2220
2018-02-07 00:54:38 +01:00
Mathieu Lonjaret 6e8a5930c9 all: make server advertise when it has legacy SHA-1 indexes, use that info
when its wholeref index lines contain any "sha1-" wholeref lines.

Then, have the client use that info when uploading files to also
compute the sha-1 (in addition to the sha-224) and ask the server for
both.

By client, we mean changes to cmd/camput, to pkg/client functions (for
camput and other Go clients, such as the apps), and to the web UI.

Also, add a -sha1 mode to devcam server, as well as the
CAMLI_SHA1_ENABLED env var, to facilitate testing/debugging of the
legacy sha1 mode.  Fix/document unrelated options in devcam server
while we're at it.

Change-Id: I369679d2a4719f6f8b409f78dd4d065e13a3f62f
2018-01-31 01:10:52 +01:00
Brad Fitzpatrick f9f109ffa1 all: more Camlistore to Perkeep renamings
Updates #981

Change-Id: I7fd4958bceaa4ada63d07aacf08af11cc03f32d7
2018-01-30 13:31:59 -08:00
Paul Lindner 459c75410e all: more renaming of Camlistore to Perkeep
Change-Id: I118e3cbcf20d80afeffc84f001388c4556f21628
2018-01-30 03:02:56 -08:00
Brad Fitzpatrick d4ff75359c pkg/client: reduce the number of New constructors, return error by default
This removes NewDefault and NewFromParams.

Now the default way to create a client is:

    client.New() -> (*Client, error)

Specifying a server is optional and now requires
client.OptionServer(server).

If the caller really wants to log.Fatal on error, they can use
client.NewOrFail.

Also, some of the boilerplate from GopherJS callers is now promoted to
be the default behavior in the client package.

Change-Id: Icb106cf3e13cc492fe5b2f7f240e1ad4227eaf33
2018-01-24 07:42:04 -08:00
Brad Fitzpatrick 38f10a7bd0 all, testhooks: use sha224 by default, add hook for some tests to use sha-1
Remove the blob.SHA{1,224}From{Bytes,String} constructors too. No
longer used. This adds blob.RefFromBytes which was missing. We had
blob.RefFromString. Now everything uses blob.RefFrom* instead of
specifying a hash function.

Some tests set a flag to force use of SHA-1 because there was too much
golden data to update. We can remove those one-by-one over time as we
fix up tests.

Updates #537

Change-Id: Ibe6428089a6221594c2b751f53f98b03b5a28dc2
2018-01-09 20:03:38 -08:00
Paul Lindner 463832ed76 clients/web/embed: Move vendor/embed to clients/web/embed and adjust paths
Addresses #889
Fixes #1024

Change-Id: I9fa1dc246b0a3a1a4e4ec7abab0a7828fdfdb41e
2018-01-09 22:33:03 +00:00
Brad Fitzpatrick 617ac40512 all: rename cammount to pk-mount
Updates #981

Change-Id: Ie1d2e3c0fd7e2b7a7801ceac0fd8f93f62d56651
2018-01-08 09:54:41 -08:00
Paul Lindner 266447a7e7 dev/devcam: import vendor files as-is, do not require gofmt
Change-Id: If3afd3769dbad0a20bd8ff2f2bbb5eb9f89f345a
2018-01-07 19:14:08 -08:00
Brad Fitzpatrick c3d05cdce9 Move more packages out of pkg/ and into internal/
Moved hashutil, httputil, osutil, netutil,
images, media, magic, video, and rollsum.
2018-01-02 21:03:30 -08:00
Brad Fitzpatrick d6a0b05df0 Rename import paths from camlistore.org to perkeep.org.
Part of the project renaming, issue #981.

After this, users will need to mv their $GOPATH/src/camlistore.org to
$GOPATH/src/perkeep.org. Sorry.

This doesn't yet rename the tools like camlistored, camput, camget,
camtool, etc.

Also, this only moves the lru package to internal. More will move to
internal later.

Also, this doesn't yet remove the "/pkg/" directory. That'll likely
happen later.

This updates some docs, but not all.

devcam test now passes again, even with Go 1.10 (which requires vet
checks are clean too). So a bunch of vet tests are fixed in this CL
too, and a bunch of other broken tests are now fixed (introduced from
the past week of merging the CL backlog).

Change-Id: If580db1691b5b99f8ed6195070789b1f44877dd4
2018-01-01 16:03:34 -08:00
Paul Lindner ba92702834 all: lint fixes for "should omit 2nd value from range"
Change-Id: I7bb19d376f96a39ecae7dbdb4d6808f704bae5fb
2017-12-13 11:31:25 -08:00
Paul Lindner 15feaeb24c all: lint fixes for 'error strings should not be capitalized or end with punctuation or a newline'
Change-Id: I9c3766a51ac8be694ae76befff4b6fa9a85e34eb
2017-12-11 06:13:25 -08:00
Paul Lindner f28eb3cce8 Remove appengine server and the old python blobserver
Change-Id: I734f4621f77ce902050e9b37805e7f8912f75e97
2017-11-30 19:13:37 -08:00
Mathieu Lonjaret 3911a41bd5 Merge "cmd/*: always output on fatal errors" 2017-10-22 15:07:06 +00:00
mpl a6a3cac0df cmd/*: always output on fatal errors
This CL addresses issues #685 and #862.

The general problem is that some critical errors, that lead clients such
as camput to exit with failure, are not displayed when not running in
verbose mode.

The reason that happens is because of code such as:

	if *cmdmain.FlagVerbose {
		log.SetOutput(cmdmain.Stderr)
	} else {
		log.SetOutput(ioutil.Discard)
	}

which means that in non-verbose mode we discard absolutely all log
messages, even those that would be printed during a Fatal* call.

To address that problem, we introduce a logger, as well as the Printf
and Logf functions using it, in pkg/cmdmain. These two functions only
output when *cmdmain.FlagVerbose is true.

Commands such as camput or camtool should now always:
1) log.SetOutput(cmdmain.Stderr) in init().
2) use log.Printf for messages that should always be printed.
3) use cmdmain.Printf/Logf for messages that should only be printed when
*cmdmain.FlagVerbose is true.
4) use log.Fatal for critical errors.
5) optionally, set the Verbose and Logger of the client(s) they are
using.

Also, camput and camtool are now relying on the global -verbose flag
from cmdmain, instead of having to define one for each subcommand.

fixes #685
fixes #862

Change-Id: I088032fd28184a201076097bf878894b22a8a120
2017-10-19 02:22:44 +02:00