The publisher app required GopherJS which has been behind upstream Go
for many years. This just removes all GopherJS stuff, breaking the publisher
app in the process. We'll have to rewrite it.
Signed-off-by: Brad Fitzpatrick <brad@danga.com>
AppEngine lets you do just about anything these days. No need to cater
to the ancient limited version.
Signed-off-by: Brad Fitzpatrick <brad@danga.com>
Since we do not systematically regenerate app/publisher/js/zsearch.go
anymore (because it's part of building the web UI process),
go test ./app/publisher/js
would fail. But we want go test ./... to succeed, so we add build tags
in ./app/publisher/js so that it gets ignored by default.
Change-Id: Ia77689ed937411a628e903189433b70be659e941
- correct logging that logged functions instead of their value
- use ID vs Id naming
- use correct function names in comments
Change-Id: I61562cef7ebac7337ec6c85312cdf7915cb1a84b
I had intended for this to be a small change.
I was going to just add context.Context to the BlobReceiver interface,
but then I saw blob.Fetcher could also use one, so I decided to do two
in one CL.
And then it got a bit infectious and ended up touching everything.
I ended up doing SubFetch in the process by necessity.
At a certain point I finally started using context.TODO() in a few
spots, but not too many. But removing context.TODO() will come in the
future. There are more blob storage interfaces lacking context, too,
like RemoveBlobs.
Updates #733
Change-Id: Idf273180b3f8e397ac5929c6d7f520ccc5cdce08
Part of the project renaming, issue #981.
After this, users will need to mv their $GOPATH/src/camlistore.org to
$GOPATH/src/perkeep.org. Sorry.
This doesn't yet rename the tools like camlistored, camput, camget,
camtool, etc.
Also, this only moves the lru package to internal. More will move to
internal later.
Also, this doesn't yet remove the "/pkg/" directory. That'll likely
happen later.
This updates some docs, but not all.
devcam test now passes again, even with Go 1.10 (which requires vet
checks are clean too). So a bunch of vet tests are fixed in this CL
too, and a bunch of other broken tests are now fixed (introduced from
the past week of merging the CL backlog).
Change-Id: If580db1691b5b99f8ed6195070789b1f44877dd4
Use the go/parser to parse, go/ast to modify and
go/format to write the types needed for publisher/js
instead of running godoc.
Change-Id: I567fac9e6cdd0cb29fbd097b0d478fd9f35864fb
This switches most usages of the pre-1.7 context library to use the
standard library. Remaining usages are in:
app/publisher/main.go
pkg/fs/...
Change-Id: Ia74acc39499dcb39892342a2c9a2776537cf49f1
According to https://golang.org/pkg/html/template/#ErrorCode
(ErrPredefinedEscaper), the template engine already performs
sanitization equivalent to urlquery on any pipeline, and therefore makes
this extra urlquery call unnecessary, and even maybe harmful.
Change-Id: I7fcce395bf015b64022d1ac66b42069cdefb69eb
Apparently the html escaper was not being used in the right order/place,
and the auto-escaper does a better job than us at escaping?
For reference: https://go-review.googlesource.com/c/go/+/37880
I have noticed some odd results when trying other escaping combinations,
but that's a Go problem, if any, that I will investigate independently.
Fixes#960
Change-Id: I43e6eca26404d0bcbc7a48764393186f1d112dbc
If we keep app/publisher/js/zsearch.go, when running make.go, it first gets mirrored into
CAMLI_ROOT/tmp/build-gopath-nosqlite/src/camlistore.org/app/publisher/js/zsearch.go
, with the same modtime as app/publisher/js/zsearch.go.
Then in genSearchTypes,
CAMLI_ROOT/tmp/build-gopath-nosqlite/src/camlistore.org/app/publisher/js/zsearch.go gets
regenerated anyway, because its modtime is older than that of
CAMLI_ROOT/tmp/build-gopath-nosqlite/src/camlistore.org/pkg/search/describe.go
Then in a subsequent run of make.go, the mirroring overwrites the newly regenerated file,
because its modtime is different (indeed, it is newer) than that of
app/publisher/js/zsearch.go
And the cycle repeats ad vitam eternam.
I think if we added app/publisher/js/zsearch.go to mirrorIgnored, it
might fix the problem, but I don't see the point in even keeping
zearch.go in git at all, so I propose with this CL to simply remove it.
If removed, it is simply generated at the first make.go run, and never
again afterwards (except if pkg/search/describe.go changes).
Fixes#957
Change-Id: Ia6fcc50ce33513a2003809783fc323ab36a60b52
When the DownloadHandler handles a zip archive requests, it writes the
zip archive being built directly to the network response, in order to
avoid writing any of the files to disk or to memory. As a consequence,
as soon as the archive starts being built, if any error reading one of
the files occurs, it can't properly report it to the client as the
response had already started being sent.
In this change, if the DownloadHandler's Fetcher is a caching fetcher,
we try reading all the requested files before starting the zip archive.
As a result, we can error early and properly if any of the files can't
be read. And since the blobs get cached as they are read the first time,
reading them a second time when actually building the archive should not
be too costly.
Change-Id: I6477d82149b08b1db1471ca9ad77fef254929db0
Change the css to allow variable width images, keep constant height only.
Use a fixed limit maxThumbWidthRatio to limit thumbnail image width
for panoramic images.
Fixes#902
Change-Id: I60c16c63b018680885c67f00b47e2e96c8dba47e
Also remove the scheme variable from the template that
tells what protocol the backend uses to talk to camlistore.
Fixes#920
Change-Id: Ia25e99d0f1b77077158f761f11a5c6bacfa2dc3b
We want to add a feature for clients (the web UI), where they can select
a bunch of files and ask the server for a zip archive of all these files.
This CL modifies the DownloadHandler so it does exactly that upon
reception of a POST request with a query parameter of the form
files=sha1-foo,sha1-bar,sha1-baz
This CL also adds a new button to the contextual sidebar of the web UI,
that takes care of sending the download request to the server.
known limitations: only permanodes with file as camliContent are
accepted as a valid selection (i.e. no sets, or static-dirs, etc) for
now.
Implementation detail:
We're creating an ephemeral DOM form on the fly to send the request.
The reason is: if we sent it as a Go http request, we'd have to read
the response manually and then we'd have no way of writing it to disk.
If we did it with an xhr, we could write the response to disk by
creating a File or Blob and then using URL.createObjectURL(), but we'd
have to keep the response in memory while doing so, which is
unacceptable for large enough archives.
Fixes#899
Change-Id: I104f7c5bd10ab3369e28d33752380dd12b5b3e6b
In other words, when on the camliRoot node, there is an attribute such
as:
camliPath:foo = sha1-foo
Where sha1-foo is a permanode that is not a set (i.e. it does not have
camliMembers or camliPaths), and typically is a permanode with some
camliContent.
Change-Id: Ib827130bb2456c4c0d7bfb40e40a425515ee1bde
Instead of reporting "404 not found" make it clear
publisher is working, but needs an explicit path.
Change-Id: Ic686b82335ba36e0649dd563831b1221a8579e0d
As the requests to the publisher are proxied through Camlistore's app
handler, there's no point in the publisher having its own autocert
Manager to request a certificate. Therefore, the publisher reuses
(readonly) camlistored's autocert CacheDir to get its certificate.
It follows that, for now, Let's Encrypt only works for the publisher if
it is running on the same host as camlistored (or more precisely, if they
share the same filesystem).
Fixes#458
Change-Id: Icf3be2913f85f9ec6f94b831ad58e1949b4d6961
Or to be more precise, golang.org/x/crypto/acme/autocert
The default behaviour regarding HTTPS certificates changes as such:
1) If the high-level config does not specify a certificate, the
low-level config used to be generated with a default certificate path.
This is no longer the case.
2) If the low-level config does not specify a certificate, we used to
generate self-signed ones at the default path. This is no longer always
the case. We only do this if our hostname does not look like an FQDN,
otherwise we try Let's Encrypt.
3) As a result, if the high-level config does not specify a certificate,
and the hostname looks like an FQDN, it is no longer the case that we'll
generate a self-signed. Let's Encrypt will be tried instead.
To sum up, the new rules are:
If cert/key files are specified, and found, use them.
If cert/key files are specified, not found, and the default values,
generate them (self-signed CA used as a cert), and use them.
If cert/key files are not specified, use Let's Encrypt if we have an
FQDN, otherwise generate self-signed.
Regarding cert caching:
On non-GCE, store the autocert cache dir in
osutil.CamliConfigDir()/letsencrypt.cache
On GCE, store in /tmp/camli-letsencrypt.cache
Fixes#701Fixes#859
Change-Id: Id78a9c6f113fa93e38d690033c10a749d1844ea6
When receiving a file, we were only trying to guess its MIME type
through its contents (pkg/magic). We're now making a better effort at it
by guessing from the filename extension if needed.
Also:
pkg/magic: get rid of all the extra video extensions that are already
covered by mime.TypeByExtension. Because it's redundant and
confusing.
app/publisher, pkg/types/camtypes: also use mime.TypeByExtension as an
extra effort. Especially since a reindex would be necessary to benefit
from the pkg/index change.
There are other places in Camlistore that could use such an effort.
Maybe we should have a camtypes.*FileInfo.MIME() method that tries all
the ways to guess the MIME type of the file?
Change-Id: Ib9a2bc42af77c5394dac578ae415524b5111ad4e
Brad Fitzpatrick