Part of the project renaming, issue #981.
After this, users will need to mv their $GOPATH/src/camlistore.org to
$GOPATH/src/perkeep.org. Sorry.
This doesn't yet rename the tools like camlistored, camput, camget,
camtool, etc.
Also, this only moves the lru package to internal. More will move to
internal later.
Also, this doesn't yet remove the "/pkg/" directory. That'll likely
happen later.
This updates some docs, but not all.
devcam test now passes again, even with Go 1.10 (which requires vet
checks are clean too). So a bunch of vet tests are fixed in this CL
too, and a bunch of other broken tests are now fixed (introduced from
the past week of merging the CL backlog).
Change-Id: If580db1691b5b99f8ed6195070789b1f44877dd4
to Rev 48b2ede4844e13f1a2b7ce4d2529c9af7e359fc5
The qr package has moved from code.google.com/p/rsc/qr to it's new
canonical home at rsc.io/qr
Change-Id: Ibb04ee7e83c9707ff253a91abb4f60f9b150d61c
Notably:
-do not load any markers on an empty search query, because that would
mean loading absolutely all of the items with a location, which seems
like a bad idea.
-use different markers for different nodes. For now, foursquare
checkins, file images, and files have their own marker.
-vendor in https://github.com/lvoogdt/Leaflet.awesome-markers to achieve
the above, which relies on Font Awesome, which we already have in.
icons available for the markers: http://fontawesome.io/icons/
-when no location can be inferred from the search query, set the view to
encompass all markers that were drawn.
-when a location search is known, draw a rectangle representing the
results zone.
-use thumber for image in marker popup
-use title, if possible, instead of blobRef for link text in marker
popup
-switch to directly using OpenStreetMap tiles, instead of MapBox ones.
https://storage.googleapis.com/camlistore-screenshots/Screenshot_20170622-232359.png
Change-Id: Ibc84fa988aea8b8d3a2588ee8790adf6d9b5ad7a
When the DownloadHandler handles a zip archive requests, it writes the
zip archive being built directly to the network response, in order to
avoid writing any of the files to disk or to memory. As a consequence,
as soon as the archive starts being built, if any error reading one of
the files occurs, it can't properly report it to the client as the
response had already started being sent.
In this change, if the DownloadHandler's Fetcher is a caching fetcher,
we try reading all the requested files before starting the zip archive.
As a result, we can error early and properly if any of the files can't
be read. And since the blobs get cached as they are read the first time,
reading them a second time when actually building the archive should not
be too costly.
Change-Id: I6477d82149b08b1db1471ca9ad77fef254929db0
We want to add a feature for clients (the web UI), where they can select
a bunch of files and ask the server for a zip archive of all these files.
This CL modifies the DownloadHandler so it does exactly that upon
reception of a POST request with a query parameter of the form
files=sha1-foo,sha1-bar,sha1-baz
This CL also adds a new button to the contextual sidebar of the web UI,
that takes care of sending the download request to the server.
known limitations: only permanodes with file as camliContent are
accepted as a valid selection (i.e. no sets, or static-dirs, etc) for
now.
Implementation detail:
We're creating an ephemeral DOM form on the fly to send the request.
The reason is: if we sent it as a Go http request, we'd have to read
the response manually and then we'd have no way of writing it to disk.
If we did it with an xhr, we could write the response to disk by
creating a File or Blob and then using URL.createObjectURL(), but we'd
have to keep the response in memory while doing so, which is
unacceptable for large enough archives.
Fixes#899
Change-Id: I104f7c5bd10ab3369e28d33752380dd12b5b3e6b
The main motivation is that we want to use gopherjs +
github.com/myitcv/gopherjs/react , which requires a recent react.
This CL also fully switches to only using minified js files.
The changes in server/camlistored/ui are the result of some react
API changes, such as:
- wraps component instantiation in React.createElement
- React.classSet -> classNames
- Need tbody between table and tr in a couple places
- component array elts need unique keys (longstanding warning)
- change root elt from body to a child div (to fix warning)
- don't check isMounted in container render function
- fix a bunch of style keys using kebab-case instead of camelCase
Change-Id: Idc22d07704688ccde3e4f67d4f188a5ad32b6e32
When reindexing on a (My)SQL based sorted.KeyValue, we should recreate
the database schema from scratch, which means dropping the tables.
However, index.Reindex just calls Wipe on the newly created
sorted.KeyValue, which only deletes the rows, and does not drop the
tables.
Therefore, this CL changes the implementation of Wipe in the MySQL case,
so that it takes care of dropping the tables, and doing everything that
needs to be done afterwards to set up the sorted.KeyValue.
In addition, with the introduction of the sorted.NeedWipeError, we detect
upon initialization of a sorted.KeyValue if it failed because it needed
a schema update. If that is the case, and we're in reindex mode, we can
fix the sorted.KeyValue with a Wipe and carry on.
Finally, we introduce the new sorted.NewKeyValueMaybeWipe function that
automatically wipes a KeyValue when a NeedWipeError was returned upon
its creation.
Next, do the same with other sorted SQLs.
Fixes#806
Change-Id: I2032781cbf453a364880bd3e2e8b3c09aac7aed9
Some of the publisher features have moved from the server-side app to
the client-side app (the browser) thanks to gopherjs. Some of these
features imply doing some search queries against Camlistore, which
requires authentication. The server-side app receives the necessary
credentials on creation, from Camlistore. However, we can't just
communicate them to the client-side (as we do with the web UI) since the
publisher app itself does not require any auth and is supposed to be
exposed to the world.
Therefore, we need to allow some search queries to be done without
authentication.
To this end, the app handler on Camlistore now assumes a new role: it is
also a search proxy for the app. The app sends an unauthenticated search
query to the app handler (instead of directly to the search handler),
and it is the role of the app handler to verify that this query is
allowed for the app, and if yes, to forward the search to the Camlistore's
search handler.
We introduce a new mechanism to filter the search queries in the form of
a master query. Upon startup, the publisher registers, using the new
CAMLI_APP_MASTERQUERY_URL env var, a *search.SearchQuery with the app
handler. The app handler runs that query and caches all the blob refs
included in the response to that query. In the following, all incoming
search queries are run by the app handler, which checks that none of the
response blobs are out of the set defined by the aforementioned cached
blob refs. If that check fails, the search response is not forwarded to
the app/client.
The process can be improved in a subsequent CL (or patchset), with finer
grained domains, i.e. a master search query per published camliPath,
instead of one for the whole app handler.
Change-Id: I00d91ff73e0cbe78744bfae9878077dc3a8521f4
This lets the UI work (and lets people hack on the UI) by installing
camlistored the normal Go way (go install) and just running
camlistored, without using devcam server.
Change-Id: I45779640c5b2286ca7061a626c07b3edcdc9c4ad
Previously pkg/jsonconfig and pkg/errorutil
Copied from go4.org at rev d1b8a2fb2de6160036e4801aa5e4d855571078b8
Change-Id: I673ed55b0825baa2607289b6082f205100261d7a
They were internal packages (under pkg), which we are now moving to
go4.org, so we in turn need to vendor them in now.
Change-Id: I92224f731404d0bd4ca1c57492bed37cb3367ed4
Signhandler was moved from the ui to the root handler and genconfig
adjusted accordingly.
New discovery struct types to replace the JSON maps are defined in
pkg/types/camtypes. The changes are related to Issue #539.
Change-Id: Ib3f12feeaf6fa81107de2d468cab491723e590ae
This makes downloads and the thumbnailer's reading of full images use
blobpacked's WholeRefFetcher fast path, when available. The end result
is many fewer blob fetches.
Updates #532
Change-Id: I3bfd1cb0f04f68f81b80a63378c7b365bd73c974
Instead of /ui/?b=<hash>, /ui/?p=<hash>, /ui/?d=<hash>, etc...,
it's now just: /ui/<hash>
This makes constructing links easier, because you don't have to
know whether a blob is also a permanode, or a directory, or
whatever.
Change-Id: I771fa1b1699b9009269c7eee632e7e8fd5335331
Creates new page @ /ui/mobile.html for generating a QR code that
embeds configuration data for setting up mobile clients.
Imports code.google.com/p/rsc/{gf256,qr} version:
parent: 149:a3cbaf2339c4 tip
This addresses the server and web components of
https://camlistore.org/issue/372
Change-Id: If9239dd138707080afd7dedb5770d04893cbf0ba
Move from one pool for all thumbnail resizing to per-ImageHandler
instace.
Uses the value of "maxResizeBytes" if specified in the "handlerArgs"
section of a "publish" or "ui" handler's low-level config.
If we end up wanting to share pools, i.e. if all published
handlers should share a pool, we'll need to do a bit more work.
Implements suggestion from https://camlistore.org/r/1803 review.
Further improves on https://camlistore.org/issue/316.
Change-Id: Ia93fad119b546064173ac3e2c7f0ab8509744628
The detail view now loads with only two HTTP requests: one for the
piggy, and one for the big image. No other resources are loaded,
even the blob descriptions, because they are cached by SearchSession,
which is shared between the search ui and the detail ui.
Likewise, when you go back to the search results page, no additional
requests are done.
Also:
- When you press back on detail page, scroll position is now usually
restored (if you refresh between index and detail, it still won't
restore).
- Detail page live updates! It came for free with using SearchSession.
Change-Id: I58ada7bdd30afc2644860fbda6167fc5ce865b0d
In camlistored, only react.min.js is served. In devcam all files in
the read directory are served.
Change-Id: I0cb8a4c328e550b5303edef4be7571d39d123340
Fixes warning 'Resource interpreted as Image but transferred with MIME
type text/xml:' on Chrome, and more importantly returns the previously
missing icons to my camlistore instances. Everyone else must be
hitting cache, or something is weird with my setup. I see missing
icons on Safari / Chome on Mac, and Firefox / Chrome on Linux.
Change-Id: I4809d8c7948a49fe80fdf1a97ac98b2161ba0307
Previously, every time you restarted the server, it forgot about all
thumbnails previously generated. (in practice it didn't/doesn't
matter with a single user, though, since they're still cached in the
browser and we always reply to If-Modified-Since immediately without
checking the cache) But it'll matter more with the Publish handler.
Also, rename some stuff, clean up some stuff, drop an unused interface.
And then necessarily change the serverconfig low-level generator to use
a kvfile for the thumbmeta map when using local disk for blobs.
--
Change-Id: I4dcfcb21429a440aa118794c03f7abf7bd69c33b
To select an item, click the checkmark that overlays on rollover.
To navigate, just click anywhere else on the item.
Also move selection from additive by default rather than exclusive.
Change-Id: I47e6ca1942fe1fdaaf615f6ab0e0a547f1a51735
-dev-appengine populates the default sourceRoot.
-app.yaml filters out the go files from the static
resources
-the UI handler warns that sourceRoot is required
Change-Id: I816456a882ff4b0573bc8962cce85139d8a0f611
Brad Fitzpatrick
