Since we renamed Camlistore to Perkeep, we also renamed the docker image
deployed by the launcher, as well as the systemd service that this
docker image runs in.
This means existing users on GCE do not get updates when they restart
the service or reset their instance.
This change adds a check and a function that allows perkeepd to fix the
user-data metadata of its own GCE instance, which is where the names
mentioned above are defined.
Therefore, if we deploy a new camlistore docker image containing that
code, the existing users should get a server capable of self-fixing its
own instance into a perkeep one.
Change-Id: I9ff66f684f0ad647d15e5fcdd08d26c524a18329
This change makes the Permanode aspect come before the Map in the list
of aspects. That way, if there is no specific aspect selected for a
permanode, the default one should then be Permanode rather than Map.
I'm not sure it's not going to break a specific workflow though.
Change-Id: I3d54dc84c7f4fb313d8ed0f7285db69b4ecbf366
We migrated to using the long form for key IDs sometime ago, but we
never updated the client of the gpgchallenge accordingly, so in
functions that take a keyID as an argument, it would fail to find the
given long keyID within a key ring.
However, for usability reasons, we want to keep the short form in the
hostname of the instance (and hence as the DNS entry), so we revert to
using the short form when perkeepd is setting its host name.
Change-Id: I8373016671fdead32780a04f6d64045e81cc9cd1
We had checks in place to return an error if a search query had both
no constraint and no expression. However:
1) we accepted " " as a valid search expression (which is the
work-around that was used by the web UI for the index page)
2) the rest of the code seems to be coping well if we remove these
checks, which this CL does.
This allows us to remove the q=" " trick in the web UI, which does not
seem to be breaking anything, and has the side-effect of improving the
map aspect story (issue 1081). Because with this change, the index page
is not considered anymore as the result of a search query, which means
if one then clicks on an image, one then gets to the image permanode as
a "standalone" item, instead of as one item part of set.
As a consequence:
1) If one then clicks on the map aspect, one gets only this item on the
map (with the proper zoom level), instead of all the items from the
index page.
2) The other side of the coin is left/right navigation does not "work"
anymore in this context, since the item is not part of a set anymore.
In other words, this change makes the index page (i.e. the empty search)
a somewhat particular case, and makes it behave a little differently
from the result of any other (non-empty) search.
Updates issue #1081
Change-Id: Ibbb1711c67c7cf8adaf94ab5d2c68c3755e4983b
Also disable the annoying mouse hover enter/exit behavior. Require
clicks. TODO: more focus events elsewhere should close the menu, but
this is better than before.
Also, rename the PIGGY variable to KEEPY for now, in prep for changing
the logo from a pig to the Keepy the Perkeep Parakeet mascot.
Updates #981
Change-Id: I48d9dd7267701b39122836aaad708726b1c105ac
We actually don't need to know the filename of an image for a valid
thumber to be created, so this change is removing such a strict check,
which was making foursquare venue photos not be displayed.
Also fix broken initialization of the thumber in foursquare container.
Change-Id: Iefb36fc403a3c1101d83f6af916b447e149f5acf
We still need to figure out the root cause of the problem, i.e.
why a sha224 importer node was created in the first place, when a
valid sha1 one already existed. Next CL.
Updates #1069
Change-Id: I00b4aab9a4f67b9c4ee198774378630615f6d11d
The blobserver uses a base blobserver in read-only mode,
and uses another stage blobserver to record changes.
In a sense it is a read-write view on the master
blobserver that never changes base itself.
Change-Id: I39c7d7bbac713c32fd17710fab43754b546ebb3b
The download handler was setting the Content-Disposition to attachment
when the MIME type was "application/octet-stream". That means when e.g.
a plain text file (and probably others) is (incorrectly) detected as
"application/octet-stream", it is saved to disk instead of being opened
as text by the browser, when using the "View Original" button of the web
UI.
Conversely, when using the "Download" button of the web UI, the file
should always be saved to disk, and never be opened by the browser.
Therefore, we add the ?inline request parameter to the downloadhandler
GET handler to unambiguously request whether the response should have an
inline or an attachment Content-Disposition.
The View Original button action now always requests an inlined response.
The Download button action now detects whether the selection was only
about one file, and if yes it requests the file as an attachment
(instead of using the POST handler that zips all the requested files).
Change-Id: I4bbc42f17fb4ea0aff9e17056e80156611d4b419
Check that all blobpacked zips are in the blobpacked meta, and
vice-versa (that all entries in meta do exist as a zip in large
storage).
As the current recovery code would not fix the case of stale entries
(large blobRefs in the blobpacked index of large blobs that don't exist
anymore), this change also adds a new recovery mode, which wipes the
existing blobpacked index, before rebuilding it.
In doing so, the recovery var in blobpacked pkg, as well as the
flagRecovery in camlistored.go have been changed to ints instead of bools,
to take into account that we now have several modes of operation for
recovery.
Fixes#946
Change-Id: I1fe76b805af34933e362d70c9f27bfd5403e3f3a
The current maximum size for a schema blob is 1MB. For a large enough
directory (~20000 children), the resulting static-set JSON schema is
over that maximum size.
We could increase that maximum, but we would eventually hit the maximum
blob size (16MB), which would only allow for ~300000 children. Even if
that is an uncommon size, it is technically possible to have such large
directories, so I don't think it would be reasonable to restrict users
to such a limit. So it does not seems like enough of a solution.
The solution proposed in this CL is to spread the children of a
directory (when they are more numerous than a given maximum, here set to
10000) onto several static-sets, recursively if needed. These
static-sets (subsets of the whole lot of children) are stored in the new
"mergeSets" field of their parent static-set schema. The actual fileRefs
or dirRefs, are still stored in the "members" field of the subset they were
spread in. The "mergeSets" and "members" field of a static-set are therefore
mutually exclusive.
Fixes#924
Change-Id: Ibe47b50795d5288fe904d3cce0cc7f780d313408
when its wholeref index lines contain any "sha1-" wholeref lines.
Then, have the client use that info when uploading files to also
compute the sha-1 (in addition to the sha-224) and ask the server for
both.
By client, we mean changes to cmd/camput, to pkg/client functions (for
camput and other Go clients, such as the apps), and to the web UI.
Also, add a -sha1 mode to devcam server, as well as the
CAMLI_SHA1_ENABLED env var, to facilitate testing/debugging of the
legacy sha1 mode. Fix/document unrelated options in devcam server
while we're at it.
Change-Id: I369679d2a4719f6f8b409f78dd4d065e13a3f62f
This removes NewDefault and NewFromParams.
Now the default way to create a client is:
client.New() -> (*Client, error)
Specifying a server is optional and now requires
client.OptionServer(server).
If the caller really wants to log.Fatal on error, they can use
client.NewOrFail.
Also, some of the boilerplate from GopherJS callers is now promoted to
be the default behavior in the client package.
Change-Id: Icb106cf3e13cc492fe5b2f7f240e1ad4227eaf33
I had intended for this to be a small change.
I was going to just add context.Context to the BlobReceiver interface,
but then I saw blob.Fetcher could also use one, so I decided to do two
in one CL.
And then it got a bit infectious and ended up touching everything.
I ended up doing SubFetch in the process by necessity.
At a certain point I finally started using context.TODO() in a few
spots, but not too many. But removing context.TODO() will come in the
future. There are more blob storage interfaces lacking context, too,
like RemoveBlobs.
Updates #733
Change-Id: Idf273180b3f8e397ac5929c6d7f520ccc5cdce08
The web UI support the old sha1 blob hash only.
Add initial support for sha224 blob refs.
Also add sha2.js and sha224.js to goog/crypt in the embedded resources.
Change-Id: If52d75d254d6163a737c56cf1676be1ef7eddb31
As the priority is to fix GCE instances, the port for the http-01
challenge is not configurable for now (80) even not on GCE, but it will
be in a follow-up change.
update golang.org/x/crypto/* (for acme) to rev
13931e22f9e72ea58bb73048bc752b48c6d4d4ac
update golang.org/x/sys/* to rev
fff93fa7cd278d84afc205751523809c464168ab (because unix is a dep of
crypto)
remove warning about Let's Encrypt security issue from pkg/deploy/gce
I had to manually exclude vendor/golang.org/x/crypto/acme/jws_test.go
for now because it contains a private key, and git whines about it, and
i could not override it.
Fixes#1033
Change-Id: Ie4f2049e97892dee9ab513300a5f12e64976aec8
This blobserver is just "cat"ing the given "read" storages.
This is read-only, so you should use some other storage to augment this for
writing and removing - for example the "cond" storage is perfect for this.
My use-case is to use blobpacked with large=diskpacked, small=filesystem,
but consolidate the small blob storage into a diskpacked + filesystem
after the filesystem becomes huge.
Another use-case is joining separately built camlistore servers into one.
(For me, they have to be separated later, so I've built them separately,
but I've to use it joined for a month).
Change-Id: I4e7e42cd59286f0f34da2f6ff01e44439771d53c
Calls to net.Dial* are prohibited with GopherJS. This can happen if the
client's transport is set by the user.
This change forces transportForConfig to return nil when the client
package is compiled with gopherjs, in order to make sure that a call to
newClient returns a client with a nil transport.
Change-Id: I577457bd7d924d31710168086dc2b394df3d1ae0
Notably: pkg/misc all moves.
And pkg/googlestorage is deleted, since it's not used. Only the
x/net/http2/h2demo code used to use it, but that ended in
https://go-review.googlesource.com/33230 (our vendored code is old).
So just nuke that dir for now. When it's refreshed, it'll either be
gone (dep prune) or new enough to not need googlestorage.
Also move pkg/pools, pkg/leak, and pkg/geocode to internal.
More remains.
Change-Id: I2640c4d18424062fdb8461ba451f1ce26719ae9d
Part of the project renaming, issue #981.
After this, users will need to mv their $GOPATH/src/camlistore.org to
$GOPATH/src/perkeep.org. Sorry.
This doesn't yet rename the tools like camlistored, camput, camget,
camtool, etc.
Also, this only moves the lru package to internal. More will move to
internal later.
Also, this doesn't yet remove the "/pkg/" directory. That'll likely
happen later.
This updates some docs, but not all.
devcam test now passes again, even with Go 1.10 (which requires vet
checks are clean too). So a bunch of vet tests are fixed in this CL
too, and a bunch of other broken tests are now fixed (introduced from
the past week of merging the CL backlog).
Change-Id: If580db1691b5b99f8ed6195070789b1f44877dd4
mpl