As far as I understood, for the auto-uploading to work reliably, the
file observers need to be permanently referenced, so they can watch for
new file creations constantly. It seems that it was not the case, as
they are referenced by the upload service, which itself was meant to be
stopped as soon as it's done uploading.
Moreover, as android memory management is getting more aggressive, it is
apparently getting more likely that the service would get killed anyway,
even if it did not stop itself.
This change is therefore two-fold:
1) The service does not try to stop itself anymore when it's done
uploading. Also, the activity starts it before binding to it (because
otherwise a service that has been automatically "bound-started" gets
destroyed as soon as all its binders disconnect from it).
2) The service is declared to run on the foreground, to make it the
least likely possible for the system to kill it. That means adding a
notification to which it is tied. The notification action leads to the
settings page, so that the auto-uploading can be toggled on/off.
At first I tried creating a new service class, that would be the
smallest possible (having only the file observers), to reduce the size
of what would have to be in the foreground. But it proved harder than I
thought and would require more time to rework the app as a whole, and
I'm not sure that's the best thing to do right now. Which is why I
resorted to make the whole UploadService a foreground one. I do not know
yet whether this is an acceptable trade-off.
In addition, this CL tries harder to find the directories with
interesting files to upload, by trying directories on the SD card too.
Fixes#1049
Change-Id: I311f31acc0a1db43791a7502a45a9f2da92cf39a
Use correct camlistored recovery flag values 0,1,2 as optional
selection values when restarting from the WebUI status page.
Fixes#1071
Change-Id: Ia537e412290e2b954fb6c604a24b9ffef70acde7
corpus.AppendPermanodeAttrValues and corpus.PermanodeAttrValue now take
a GPG key ID as their signer filter attribute, instead of a blob Ref.
Fix search package accordingly.
Fixes#1069
Change-Id: I0ef519fc16cf357adb84c32fa6e5bf92d8a0de26
We still need to figure out the root cause of the problem, i.e.
why a sha224 importer node was created in the first place, when a
valid sha1 one already existed. Next CL.
Updates #1069
Change-Id: I00b4aab9a4f67b9c4ee198774378630615f6d11d
search -1 will output one blob ref per line instead of JSON, suitable
for piping into xargs.
Fixes#1034
Change-Id: I7bf812767fa9859ff5f5d98858abb688906f20f6
Use the names "lower" and "upper" from OverlayFS
instead of "base" and "stage".
Also make the deleted keyvalue (and thus support for deletion) optional,
and improve readability for the StatBlobs method.
Change-Id: Ic3f36609bf4599251f9ba7c648f513b788550298
Go 1.10 for Perkeep and Go 1.9 for GopherJS as requirements is apparently
not explained well enough by the error message. So until we improve on
that, it's better to keep both requirements at the same version.
And we can update both soon to 1.10 when GopherJS supports 1.10 too.
Change-Id: I9f015dd27c18abad0b48559f2f40aa44362b000a
Adds header link icons on hover. Check for headers (h1-h4) with
defined id's and adds link icon with relevant fragment link.
Fixes#1028
Change-Id: Ic9c8744efe12dda251f4a461adb0b09cf84604c3
The blobserver uses a base blobserver in read-only mode,
and uses another stage blobserver to record changes.
In a sense it is a read-write view on the master
blobserver that never changes base itself.
Change-Id: I39c7d7bbac713c32fd17710fab43754b546ebb3b
The download handler was setting the Content-Disposition to attachment
when the MIME type was "application/octet-stream". That means when e.g.
a plain text file (and probably others) is (incorrectly) detected as
"application/octet-stream", it is saved to disk instead of being opened
as text by the browser, when using the "View Original" button of the web
UI.
Conversely, when using the "Download" button of the web UI, the file
should always be saved to disk, and never be opened by the browser.
Therefore, we add the ?inline request parameter to the downloadhandler
GET handler to unambiguously request whether the response should have an
inline or an attachment Content-Disposition.
The View Original button action now always requests an inlined response.
The Download button action now detects whether the selection was only
about one file, and if yes it requests the file as an attachment
(instead of using the POST handler that zips all the requested files).
Change-Id: I4bbc42f17fb4ea0aff9e17056e80156611d4b419
The index and search pkg now rely on the GPG identity to be in the long
form (16 chars). But the server configuration historically was using the
short form (8 chars).
This change therefore forces the GPG identity ending up in the generated
low-level config to be in the long form.
Also, fix the functions in pkg/jsonsign to generate the long form as well,
to ease the consistent use of it everywhere else.
Fixes#1047
Change-Id: I1333150316caa2c5e179dde0c92b850a362ad17e
The current go version checks in make.go assume a single character
minor version number. This change switches the constants to integers
instead of characters and uses a regex to check for versions.
Updates #1004
Change-Id: I7df14b69447119ecfa8379453f5c7022f20d6b5e
Check that all blobpacked zips are in the blobpacked meta, and
vice-versa (that all entries in meta do exist as a zip in large
storage).
As the current recovery code would not fix the case of stale entries
(large blobRefs in the blobpacked index of large blobs that don't exist
anymore), this change also adds a new recovery mode, which wipes the
existing blobpacked index, before rebuilding it.
In doing so, the recovery var in blobpacked pkg, as well as the
flagRecovery in camlistored.go have been changed to ints instead of bools,
to take into account that we now have several modes of operation for
recovery.
Fixes#946
Change-Id: I1fe76b805af34933e362d70c9f27bfd5403e3f3a
Remove the following transitive packages with constraints. dep
does not enforce these:
- github.com/cznic/fileutil
- github.com/cznic/lldb
- github.com/cznic/mathutil
- github.com/edsrzf/mmap-go
- github.com/golang/protobuf
- github.com/gopherjs/jsbuiltin
- github.com/pkg/errors
Rerun dep ensure -update and resync packages.
Change-Id: I3a6613459628dc963516e840a453d2bc2d25835a
It is being automatically removed by dep ensure since
3dfd895def but its removal breaks the web
UI.
This change alone is therefore probably not enough, we need to teach dep not
to remove it.
Change-Id: I61d50e703e786fdebc10446784dc84d43a0c1620
Use the now renamed libsqlite3 build pragma. This requires
having sqlite3 installed on the system instead of using the
c code bundled in vendor (which is removed by the dep tool
by default)
1.6.0 corresponds with rev
github.com/mattn/go-sqlite3@6c771bb9887719704b210e87e934f08be014bdb1
Changelog: See https://github.com/mattn/go-sqlite3/releases
Change-Id: I48fc0bc005715b83805837f544e7fa7875e6d56a
-android plugin 3.0.1
-gradle 4.1 (requirement for the above)
-gomobile to 069be623eb8e75049d64f1419849b3e92aab1c81
-go 1.9.3
-removed permission to write on storage from the manifest
-added permission to read from storage on the manifest
-added permission checking code for reading files on storage, which is
compulsory with modern android APIs.
-add missing code to update textByteStatus and textFileStatus TextViews
Notes:
-the app won't work with a server older than rev
6e8a5930c9 as the server/client protocol
to check for existing files on the server changed at this point.
-i think with Android O there's a much higher chance that the
CamliFileObserver gets garbage collected, which breaks the
auto-uploader. We probably need to switch to a different architecture,
like running as a Foreground Service, or using the Job Scheduler to
regularly fire up the file observer. In another CL.
Updates #1049
Change-Id: I079da8e5df5be820c3985749a9e109cf0775b495
The current maximum size for a schema blob is 1MB. For a large enough
directory (~20000 children), the resulting static-set JSON schema is
over that maximum size.
We could increase that maximum, but we would eventually hit the maximum
blob size (16MB), which would only allow for ~300000 children. Even if
that is an uncommon size, it is technically possible to have such large
directories, so I don't think it would be reasonable to restrict users
to such a limit. So it does not seems like enough of a solution.
The solution proposed in this CL is to spread the children of a
directory (when they are more numerous than a given maximum, here set to
10000) onto several static-sets, recursively if needed. These
static-sets (subsets of the whole lot of children) are stored in the new
"mergeSets" field of their parent static-set schema. The actual fileRefs
or dirRefs, are still stored in the "members" field of the subset they were
spread in. The "mergeSets" and "members" field of a static-set are therefore
mutually exclusive.
Fixes#924
Change-Id: Ibe47b50795d5288fe904d3cce0cc7f780d313408
Introduction of an Owner type in pkg/index.
Related changes in pkg/index and pkg/serverinit.
Fixes#1047
Change-Id: Ifc316865833349d5202b5085e8f2c1235f3f2220
- correct logging that logged functions instead of their value
- use ID vs Id naming
- use correct function names in comments
Change-Id: I61562cef7ebac7337ec6c85312cdf7915cb1a84b
mpl