The default server config on GCE (as deployed by the launcher) did not
have a share handler. This CL adds one, so that users can benefit from
the new sharing feature from the web UI.
Also, the button for sharing from the web UI does not appear anymore if
the config that the web UI gets from the discovery does not have a
"shareRoot", because it's a strong hint that the server does not have a
share handler.
Change-Id: I6c444995339fda8dba864b1d6729fb7c1b6d72bd
So far, when building camlistored docker image for CoreOS, we were not
using make.go, and we were neither running gopherjs nor embedding the
resources (but rather provide the UI resources at their default
filesystem location).
Now that we're using gopherjs for the web UI, it is a hard dependency
for the camlistore server.
We could reproduce the steps in make.go to build gopherjs, run it to
build the web ui resources, and then move the resources at the right
place, but since make.go already does the equivalent work it seems
to make more sense to use it, which is the main point of this CL.
Similarly, it seems to make more sense to now build a binary with the
resources embedded, which is the default make.go behaviour, instead of
building a "raw" camlistored, and provide the resources as additional
directories in the container image, so this CL takes that approach too.
Finally, it was necessary to add the "-static" flag to make.go, so we
can keep on building a static camlistored binary, that does not rely on
libc for DNS. Because our container image is FROM SCRATCH, with just the
necessary binaries, in order to get a container image of a reasonable
size.
One drawback of now using make.go in
misc/docker/server/build-camlistore-server.go is we're doing some
unnecessary (since we're already running in the isolation of a
container) copying to the virtual gopath, but that seems a very tiny
price to pay. Especially considering how rarely we run that code.
Change-Id: I416c86d366cd4ed2d3b8b1636a6a65a83b9f15d7
Previously the b2 library would not refresh authorization token,
resulting in errors after 24 hours of uptime.
Also, fix RemoveBlobs not to return an error on not found names. This
was working correctly with a previous iteration of the b2 library, and
was not fixed when the API changed.
github.com/FiloSottile/b2: d05beb60354dacaa61460156f5c498bc25aecbcf
Change-Id: I8d735abd64616b82fd95807e15ffdd7d450a4ed6
Services often have a free download quota, which we can use to fetch
some random blobs, to compare with the source.
Change-Id: I526e62580af4cae7004d756721336d01c2f020c4
* Switch getTags to return a frequency map
* Removed old tag cloud library because it has been [archived and is not available](https://code.google.com/archive/p/visapi-gadgets/)
* Switched to [jQCloud](http://mistic100.github.io/jQCloud/) and vendored in the CSS and JS
* Add type-ahead suggestion for tag search box and tag edit page
Change-Id: I69b00b107d531f744c1a96cd986dee34fb3dfe06
Since we don't actually ever remove blobs (until we add garbage
collection, and even then), if a share claim gets deleted (with a delete
claim), the only knowledge of the deletion resides in the Index.
So when the share handler verifies the sharing chain, there is nothing
preventing it from reading a supposedly deleted share claim (from the
blobserver), and concluding that the share chain is valid.
This change adds the index handler to the share handler, so it can check
the deletion status of a share claim, and hence support "share
cancellation".
Fixes#914
Change-Id: I572fdddee30e745aa2d2a6720c83c8e8c916515d
Since -update_golden reorders lexicographically, and it seems we forgot
to systematically use it in previous updates, the files are now
unordered. So I'm running it now, so that future real changes don't get
polluted and hard to review in the reordering churn.
Change-Id: I7618f2e8dcc22f19850bb15c57084fc9688d67ce
To share a file or directory through Camlistore's sharing
mechanism (https://camlistore.org/doc/sharing).
Only single-item sharing supported for now.
On success, the share URL is displayed in a dialog.
If the item is a file, it can be fetched with any HTTP client through
that URL. If the item is a dir, the share URL is of the claim, so one
must follow the transitive chain of sharing to get the directory's
children, i.e. use camget -shared.
Fixes#880
Change-Id: I324a684dec10225e5b8314dda7f1b77a0de6f727
We want to add a feature for clients (the web UI), where they can select
a bunch of files and ask the server for a zip archive of all these files.
This CL modifies the DownloadHandler so it does exactly that upon
reception of a POST request with a query parameter of the form
files=sha1-foo,sha1-bar,sha1-baz
This CL also adds a new button to the contextual sidebar of the web UI,
that takes care of sending the download request to the server.
known limitations: only permanodes with file as camliContent are
accepted as a valid selection (i.e. no sets, or static-dirs, etc) for
now.
Implementation detail:
We're creating an ephemeral DOM form on the fly to send the request.
The reason is: if we sent it as a Go http request, we'd have to read
the response manually and then we'd have no way of writing it to disk.
If we did it with an xhr, we could write the response to disk by
creating a File or Blob and then using URL.createObjectURL(), but we'd
have to keep the response in memory while doing so, which is
unacceptable for large enough archives.
Fixes#899
Change-Id: I104f7c5bd10ab3369e28d33752380dd12b5b3e6b
Minimizing redundant info when setting up an app:
* prefix - Now optional and defaults to the handler prefix the whole app is attached to
* serverListen - Defaults to the host part of serverBaseURL
Change-Id: I395e71c54b9e81849681fe406ff1b5f412e92c9c
This change adds an "About" entry to the header menu, that pops up an
"About" dialog when clicked on.
But, under this innocent sounding feature is actually a more important
change: this feature is partly added through the use of gopherjs
generated code. In addition, the element added (a menu item) is
integrated
seamlessly into the menu as a React element, thanks to the use of
github.com/myitcv/gopherjs/react.
pkg/env had to be broken with build tags, because importing
"cloud.google.com/go/compute/metadata" (or one of its deps) seems to be
breaking the generated javascript.
https://github.com/camlistore/camlistore/issues/904
github.com/myitcv/gopherjs/react at rev
c04b811da4a086defd882a94cc1901da2d2158b0
honnef.co/go/js/dom at rev 24aa052bc5c63cfb9383bf59493ee48621ca788c
Issue #904Fixes#798
Change-Id: Ic09b94014d520277f8178727293787ece468babd
This update the default configuration such that `devcam server` and
`devcam server -hello` work again. Changes in whether prefix and listen
were required on the application left both of those targets broken.
The default server target was only broken briefly, when the scanning
cabinet was introduced.
The `server -hello` command has been broken for months.
This change is a temporary one. A more proper fix is present in
https://camlistore-review.googlesource.com/#/c/9206/.
Once that change is merged, this commit can be reverted.
Change-Id: If692760da265502fece3b981bdd34d4d5bea5690
We relied on missTrackFetcher to return errMissingDep when the
underlying Fetch() returned os.ErrNotExist. The caller could then know
how to act if some indexing operation failed because of an errMissingDep
error.
This was wrong for 2 reasons:
1) if a function fn(tf blob.Fetcher) error does:
if _, _, err := tf.Fetch(br); err != nil {
return fmt.Errorf("wrapping this error in a nicer error
message: %v", err)
}
when we call err := fn(tf), we lose the ability to directly determine
whether err is an errMissingDep. We'd have to parse the error string,
which is gross.
This is exactly what happens in populateDir, when we call
dr.StaticSet().
And in order to fix issue #738, we want to be able to tell when a call
to dr.StaticSet() failed because the underlying Fetch() operation
failed.
2) The blob.Fetcher interface specifically states that os.ErrNotExist
should be returned when a blob is not found. We were breaking that rule
by returning errMissingDep.
In order to address both 1) and 2), it seemed like we could add an err
field to missTrackFetcher to keep track of when an os.ErrNotExist
occurred during a Fetch, and let Fetch return an os.ErrNotExist.
However, that would not work, as a missTrackFetcher is used concurrently
by several callers, so a given caller wouldn't be able to tell whether
"its" Fetch failed or a Fetch from a concurrent caller failed.
Therefore, we introduce trackErrorsFetcher, that has such an error field,
and that wraps the missTrackFetcher. All the callers can keep on sharing
the missTrackFetcher, but each of them initialize their own
trackErrorsFetcher, and can check the errors field after a failed call to a
function is suspected to be the result of a failed Fetch.
Also added a test to demonstrate that issue #738 is fixed.
Fixes#738
Change-Id: Ia5c3081b71c77be1e8cff0bbc847ade68f019bf9
rev b9bcb1da229a59cc1e1d168401662cb6450aae08
Also added pkg/misc/fakefsnotify just so we only have to do:
- "github.com/fsnotify/fsnotify"
+ fsnotify "camlistore.org/pkg/misc/fakefsnotify"
in vendor/github.com/gopherjs/gopherjs/build/build.go
in order to avoid vendoring fsnotify, without having to grossly
patch the gopherjs code as we previously did.
Change-Id: Ic106a6afa1117d2061415355c236433f835895f1
There's a race that takes place at the end of the reindexing process.
In func (x *Index) Reindex(), we wait for all the reindexing goroutines
to be done, with wg.Wait. However, any (or all) of those goroutines
could have triggered (they call indexBlob, which calls
blobserver.Receive, which calls noteBlobIndexed, which can send on
tickleOoo) an asynchronous out of order reindexing which is NOT waited
on.
The race can be trivially demonstrated by changing:
WaitTickle:
for range ix.tickleOoo {
+ time.Sleep(5*time.Second)
in receive.go, and running:
go test ./pkg/index/ -run TestReindex_*
This CL rewrites the out of order indexing implementation to make it
simpler, and in the process, fixes the above bug.
Fixes#756
Change-Id: If79fb1ad8869cefce4a095ef2becdba333732bce
The default location for the gopherjs go version needed to be adjusted
for when CAMLI_GOPHERJS_GOROOT is not provided.
Change-Id: Ieda2e6dbc002891e977684be50577753f3ae9a7e
Because gopherjs master branch still needs Go 1.7.
So we either need to include a Go 1.7 with CAMLI_GOPHERJS_GOROOT in the
travis build (which I will look into later), or wait for gopherjs to
support Go 1.8.
Change-Id: I8e991de279bf822e1b5ef64dcae241880fe3eb73
WARNING: this app is still experimental, and even its data schema might
change. Do not use in production.
This change adds a Camlistore-based port of the scanning cabinet app
originally created by Brad Fitzpatrick:
https://github.com/bradfitz/scanningcabinet
Some of it is inspired from the App Engine Go port of Patrick Borgeest:
https://bitbucket.org/pborgeest/nometicland
The data schema is roughly as follows:
-a scan is a permanode, with the node type: "scanningcabinet:scan".
-a scan's camliContent attribute is set to the actual image file.
-a scan also holds the "dateCreated" attribute, as well as the
"document" attribute, which references the document this scan is a part
of (if any).
-a document is a permanode, with the node type: "scanningcabinet:doc".
-a document page, is modeled by the "camliPath:sha1-xxx" = "pageNumber"
relation, where sha1-xxx is the blobRef of a scan.
-a document can also hold the following attributes: "dateCreated",
"tag", "locationText", "title", "startDate", and "paymentDueDate".
Known caveats, in decreasing order of concern:
-the data schema might still change.
-the scancab tool, to actually create and upload the files from physical
documents, is practically untested (since I do not own a scanner).
-some parts, in particular related to searches, are probably
sub-optimized.
-the usual unavoidable bugs.
Change-Id: If6afc509e13f7c21164a3abd276fec075a3813bb
mpl