So far, when building camlistored docker image for CoreOS, we were not
using make.go, and we were neither running gopherjs nor embedding the
resources (but rather provide the UI resources at their default
filesystem location).
Now that we're using gopherjs for the web UI, it is a hard dependency
for the camlistore server.
We could reproduce the steps in make.go to build gopherjs, run it to
build the web ui resources, and then move the resources at the right
place, but since make.go already does the equivalent work it seems
to make more sense to use it, which is the main point of this CL.
Similarly, it seems to make more sense to now build a binary with the
resources embedded, which is the default make.go behaviour, instead of
building a "raw" camlistored, and provide the resources as additional
directories in the container image, so this CL takes that approach too.
Finally, it was necessary to add the "-static" flag to make.go, so we
can keep on building a static camlistored binary, that does not rely on
libc for DNS. Because our container image is FROM SCRATCH, with just the
necessary binaries, in order to get a container image of a reasonable
size.
One drawback of now using make.go in
misc/docker/server/build-camlistore-server.go is we're doing some
unnecessary (since we're already running in the isolation of a
container) copying to the virtual gopath, but that seems a very tiny
price to pay. Especially considering how rarely we run that code.
Change-Id: I416c86d366cd4ed2d3b8b1636a6a65a83b9f15d7
We want to add a feature for clients (the web UI), where they can select
a bunch of files and ask the server for a zip archive of all these files.
This CL modifies the DownloadHandler so it does exactly that upon
reception of a POST request with a query parameter of the form
files=sha1-foo,sha1-bar,sha1-baz
This CL also adds a new button to the contextual sidebar of the web UI,
that takes care of sending the download request to the server.
known limitations: only permanodes with file as camliContent are
accepted as a valid selection (i.e. no sets, or static-dirs, etc) for
now.
Implementation detail:
We're creating an ephemeral DOM form on the fly to send the request.
The reason is: if we sent it as a Go http request, we'd have to read
the response manually and then we'd have no way of writing it to disk.
If we did it with an xhr, we could write the response to disk by
creating a File or Blob and then using URL.createObjectURL(), but we'd
have to keep the response in memory while doing so, which is
unacceptable for large enough archives.
Fixes#899
Change-Id: I104f7c5bd10ab3369e28d33752380dd12b5b3e6b
This change adds an "About" entry to the header menu, that pops up an
"About" dialog when clicked on.
But, under this innocent sounding feature is actually a more important
change: this feature is partly added through the use of gopherjs
generated code. In addition, the element added (a menu item) is
integrated
seamlessly into the menu as a React element, thanks to the use of
github.com/myitcv/gopherjs/react.
pkg/env had to be broken with build tags, because importing
"cloud.google.com/go/compute/metadata" (or one of its deps) seems to be
breaking the generated javascript.
https://github.com/camlistore/camlistore/issues/904
github.com/myitcv/gopherjs/react at rev
c04b811da4a086defd882a94cc1901da2d2158b0
honnef.co/go/js/dom at rev 24aa052bc5c63cfb9383bf59493ee48621ca788c
Issue #904Fixes#798
Change-Id: Ic09b94014d520277f8178727293787ece468babd
rev b9bcb1da229a59cc1e1d168401662cb6450aae08
Also added pkg/misc/fakefsnotify just so we only have to do:
- "github.com/fsnotify/fsnotify"
+ fsnotify "camlistore.org/pkg/misc/fakefsnotify"
in vendor/github.com/gopherjs/gopherjs/build/build.go
in order to avoid vendoring fsnotify, without having to grossly
patch the gopherjs code as we previously did.
Change-Id: Ic106a6afa1117d2061415355c236433f835895f1
The default location for the gopherjs go version needed to be adjusted
for when CAMLI_GOPHERJS_GOROOT is not provided.
Change-Id: Ieda2e6dbc002891e977684be50577753f3ae9a7e
WARNING: this app is still experimental, and even its data schema might
change. Do not use in production.
This change adds a Camlistore-based port of the scanning cabinet app
originally created by Brad Fitzpatrick:
https://github.com/bradfitz/scanningcabinet
Some of it is inspired from the App Engine Go port of Patrick Borgeest:
https://bitbucket.org/pborgeest/nometicland
The data schema is roughly as follows:
-a scan is a permanode, with the node type: "scanningcabinet:scan".
-a scan's camliContent attribute is set to the actual image file.
-a scan also holds the "dateCreated" attribute, as well as the
"document" attribute, which references the document this scan is a part
of (if any).
-a document is a permanode, with the node type: "scanningcabinet:doc".
-a document page, is modeled by the "camliPath:sha1-xxx" = "pageNumber"
relation, where sha1-xxx is the blobRef of a scan.
-a document can also hold the following attributes: "dateCreated",
"tag", "locationText", "title", "startDate", and "paymentDueDate".
Known caveats, in decreasing order of concern:
-the data schema might still change.
-the scancab tool, to actually create and upload the files from physical
documents, is practically untested (since I do not own a scanner).
-some parts, in particular related to searches, are probably
sub-optimized.
-the usual unavoidable bugs.
Change-Id: If6afc509e13f7c21164a3abd276fec075a3813bb
The change in I7c80976d46ce8d69e394d226c685c8554176ea0f wasn't working
for me. make.go might be a little smarter by requiring the exact Go
version that it knows the pinned gopherjs needs, instead of just
checking it is not devel, but in any case it seems sensible to always
respect the env var.
Successfully built like this after this patch:
CAMLI_GOPHERJS_GOROOT=$(go env GOROOT) go1.8rc3 run make.go
Change-Id: I58f398531cd65847b5bf82f9b9d0aa26bbe9b90d
Issue #839 suggests to not cut the rev at all, and let it be user
defined. I have no good argument against, but I'm taking the more
conservative approach suggested by others in issue #888.
Fixes#888
Change-Id: Id65b1b074e6646a756353119a12081330775e066
CAMLI_GOPHERJS_GOROOT has effect only if verifyGopherjsGoroot is called,
which only happened if go version was "devel".
This CL runs the verifyGopherjsGoroot check for go1.8rc and go1.8beta
versions too.
Change-Id: I7c80976d46ce8d69e394d226c685c8554176ea0f
server/camnetdns: update to cloud.google.com/go imports.
vendor: add github.com/miekg/dns at rev
3f1f7c8ec9ead89493df11f2c3d8bec353a2c2c0 as a dep of server/camnetdns
Change-Id: Ief8e0ad3e09a6c3511c72a7991309808f855d3df
Because our go generate line is:
//go:generate go run gensearchtypes.go -out zsearch.go
which will run a binary of gensearchtypes.go built for whatever $GOOS is
set to. Which will fail if $GOOS is different from runtime.GOOS (the
cross-compiling case).
I suppose it means that the day pkg/search becomes GOOS differentiated,
we may have to introduce an -os flag to gensearchtypes.go, since it
calls go doc on camlistore.org/pkg/search, whose output might depend on
GOOS?
Change-Id: I1ea32bb9190300120887ee8614dcdd2d1391a954
vendoring: update gopherjs to rev
45518c130e5bd1525f20110830a4986365a153de
Patch on upstream to remove fsnotify dep added (for reference) as:
vendor/github.com/gopherjs/gopherjs/nofsnotify.diff
make.go: make Go1.7 the required version
Fixes#838
Change-Id: I2013ee4832a26f8be3a8b42f02e40a347674ec9a
Using go:generate to call a shell script with some go doc + sed hackery.
we could probably do it better with go/types later if needed.
Change-Id: Ie1cf04d418b8b498f83f7029eb736dbc779feeb5
Run gopherjs to generate trivial javascript code that is used by the
publisher app.
Context:
https://github.com/camlistore/camlistore/issues/798#issuecomment-226902924
github.com/gopherjs vendored in at rev
f3c437955da554f2643747a598b0cc772a749f3f
PLEASE NOTE that this copy of gopherjs has been modified to avoid
depending on fsnotify. Hence the -w flag and the gopherjs serve command
are most likely broken.
Diff for that modification:
https://gist.github.com/mpl/ac9033bb28207401b7cedc3d74e6c096
Dependencies for building gopherjs:
kardianos/osext 29ae4ffbc9a6fe9fb2bc5029050ce6996ea1d3bc
neelance/sourcemap 8c68805598ab8d5637b1a72b5f7d381ea0f39c31
spf13/cobra c678ff029ee250b65714e518f4f5c5cb934955de
spf13/pflag 7f60f83a2c81bc3c3c0d5297f61ddfa68da9d3b7
golang.org/x/crypto/ssh/terminal
c197bcf24cde29d3f73c7b4ac6fd41f4384e8af6
golang.org/x/tools/go/types/typeutil
ac02106e04bdb66a2db0413d931012bea165d7e0
github.com/gopherjs/jquery vendored in at
fbbfc4bbe29a29cb05788b66be44e0ac7f43cac7
jquery vendored in at 2.2.3
Change-Id: I7ff2d8e43e8a963f5ac1d13a2c936f263f7c53fc
pkg/fs/fs_test.go depends on bazil.org/fuse/syscallx, which in turn
depends on golang.org/x/sys/unix
We already had the assembly code vendored in for golang.org/x/sys/unix,
but we weren't mirroring it in make.go
This didn't show with 'go run make.go' because it only affects a test.
And it didn't show with 'go test ./pkg/fs' either, because the assembly
would then be found in GOPATH.
Only devcam test would detect it.
Change-Id: Ibb90a5fb986a3d0308ff9f1f8df8d5866cfec8c5
Because it makes full integration with gopherjs impossible (without
polluting the user's GOPATH), as long as
https://github.com/gopherjs/gopherjs/issues/415 is not fixed.
Also it is kind of antithetical with the point of make.go anyway.
We still rely on CAMLI_MAKE_USEGOPATH for the integration tests that run
make.go to know that they shouldn't recursively create another temp
GOPATH (when they're already in such a temp dir, because they're started
through devcam test).
Change-Id: Icc6af46ec5976fdf08e9b8bf4249e307a15499cf
That should make it easier to catch missing dependencies. Because
'go run make.go -website' will ignore whatever's in your GOPATH, and
only use what's in our vendor.
Change-Id: I94648633a9fa69a5742b7e83031dab34470b9cde
Version in binaries will now be:
"X.Y (git rev xxxxx)"
if X.Y is specified with --tarball_version,
"xxxx"
otherwise.
Also uniformized the flags names and use in other programs.
Fixes#665
Change-Id: I958094f69b279437504854a3ff10e924d42c4fb6
Because camweb now depends on pkg/deploy/gce, which has global vars
initialized with calls from osutil.
And test website from devcam test
Change-Id: I233c229e96ded5bfde4e2c1fa7a18bbf757ab1fb
Even if sqlite libs are installed on the host, since sqlite as an
indexer seems to be less efficient than kvfile (and way less efficient
than leveldb) for us, we should not build camlistored with it, unless
user-specified.
Related: issue #632 (relevant benchmarks posted there).
Change-Id: I6f87e554ee03ccb8dbbcf4d918971dba715647f4
See https://golang.org/s/go15vendor
This commit moves the google.golang.org/cloud/... packages. The rest
of third_party will move later.
This commit is a dependency for some cloud logging work which we'll
develop in Camlistore's tree.
It's fundamentally flawed: the whole point of building in Docker is to
build in a hermetic environment. You can't mix building some parts in
Docker and some parts not. We'll build it only in Docker when making
release images, never depending on the host configuration.
Change-Id: I5fb283b17489fba18e89398e0d3f425c7f6159cd
mpl