This switches most usages of the pre-1.7 context library to use the
standard library. Remaining usages are in:
app/publisher/main.go
pkg/fs/...
Change-Id: Ia74acc39499dcb39892342a2c9a2776537cf49f1
This CL addresses issues #685 and #862.
The general problem is that some critical errors, that lead clients such
as camput to exit with failure, are not displayed when not running in
verbose mode.
The reason that happens is because of code such as:
if *cmdmain.FlagVerbose {
log.SetOutput(cmdmain.Stderr)
} else {
log.SetOutput(ioutil.Discard)
}
which means that in non-verbose mode we discard absolutely all log
messages, even those that would be printed during a Fatal* call.
To address that problem, we introduce a logger, as well as the Printf
and Logf functions using it, in pkg/cmdmain. These two functions only
output when *cmdmain.FlagVerbose is true.
Commands such as camput or camtool should now always:
1) log.SetOutput(cmdmain.Stderr) in init().
2) use log.Printf for messages that should always be printed.
3) use cmdmain.Printf/Logf for messages that should only be printed when
*cmdmain.FlagVerbose is true.
4) use log.Fatal for critical errors.
5) optionally, set the Verbose and Logger of the client(s) they are
using.
Also, camput and camtool are now relying on the global -verbose flag
from cmdmain, instead of having to define one for each subcommand.
fixes#685fixes#862
Change-Id: I088032fd28184a201076097bf878894b22a8a120
This CL changes the GCE launcher to work with the new features of
camlistored: i.e. that it can automatically get a hostname in
camlistore.net, and that it can get an HTTPS certificate from Let's
Encrypt, for said hostname.
In order for the user to easily (without having to look at the logs)
know what their hostname is, camlistored stores it as the
"camlistore-hostname" key in the custom metadata of the GCE instance.
The deployer can then query for that key, to report the hostname on the
instance creation success page.
Change-Id: Iaaef2d51f34fa5e1e0ee90097919abab7ee72a12
When fetching shared blobs, we rely on the share chain to verify if a
blob can be reached. This chain is updated whenever we fetch an
additional link of the chain, by updating the Client.via map. However,
when some blobs of the chain are already cached in camget's DiskCache,
because we get them from the cache, we don't fetch them with
Client.FetchVia, which means the Client.via map isn't updated. And thus
the chain is broken.
This change adds Client.UpdateShareChain, and sets it as a hook to be
called by the CachingFetcher in the event of a cache hit. That way, we
ensure that the share chain is updated even when we get blobs from the
cache (instead of from the Client).
We also add a mutex to guard Client.via, because it is accessed by
concurrent smartFetch calls in case of a static-set.
As FetchVia was undocumented and not used by anyone, I made it
unexported. We can always export it again later when needed.
Fixes#856
Change-Id: I767cbec4b6f382cbccc25c0b97782b2a7472deb8
rm google.golang.org/cloud
add cloud.google.com/go at a47b182e769f5e75f5fc927ff6ee2678f7f552cf
update google.golang.org/api to 63cb68f1e3834e44683ca062ddf06cb9a889380a
update google.golang.org/grpc to
0e6ec3a4501ee9ee2d023abe92e436fd04ed4081
update go4.org to f5283521d7365fb2875408726e9cbf349f173767
fix in cmd/ pkg/ server/
TODO(mpl): fix misc/docker tools as well. next CL.
Fixes#832
Change-Id: I842b968a0afea8a5822913bd614d67cdbe50ee63
The google Endpoint is now available in upstream oauth2, we no longer
need to define it ourselves.
Fixes#810
Change-Id: I37db869ff7696c53ba456ecba55e72c8c86dd119
There were basically two bugs in bytesHaveSchemaLink:
-typo "blob" vs "bytes" camliType. this wouldn't happen if we used
constants from a pkg for camliTypes, instead of hardcoding them..
-returning early instead of checking all the parts of a file.
Added a test for each of those bugs. They look like they could be
refactored but they're subtly different enough that it might not be
worth it.
Context: working on issue #527
Change-Id: Ifd0bd311ae8a115de2a56994b89127db6f315a75
In addition to outputting the JSON config needed for cloudstorage_test,
also provide the colon-delimited string needed for server-config.json.
Also update URL for Google API Console
Fixes#715
Change-Id: I3784ad3442bcf0c99cb738a3beed7b9e5fbcac68
The describe requests were launching a storm of RLocks which weren't
safe in the presence of goroutines trying to acquire write locks.
Instead, make the corpus locking the responsibility of the caller and
add Lock/Unlock/RLock/RUnlock methods to the index and move locking up
a level.
This also adds a fair bit of context.Context plumbing which was used
in earlier debugging.
Fixescamlistore/camlistore#709
Change-Id: I8d7254d1e1da541f8c080d62f5408aac807fd3b1
previously, you could use the -limit flag to specify a number of desired
results. this change allows specifying the "continue" token in
subsequent commands without resorting to a raw query.
Change-Id: Ib2ec43bb228e5504fa6d6e4ee58207243f014ab1
move to vendor
fix (lots) of breakages due to API changes
bazil.org/fuse: 37bfa8be929171feec943f3496bc4befdeaf10db
Change-Id: Ic92faf7585371027f3c521f7a24ae56d70d18547
Technically, we don't need to, because we know that up.fileOpts is always
initialized when using camput file.
But we might as well be clean and consistent with the other options such
as wantVivify and wantFilePermanode.
Change-Id: I0ef157e5ade3095c42cdc288846fda8cf5db8a3c
The import path was added to the go file that included the package
documentation if one existed. Otherwise, I used what seemed to be the
primary file for the package.
Fixes#689
Change-Id: If51be0e86529fd6f179e80af6781e639f8550fd2
Most of it replaced with vendor/go4.org/types and
vendor/go4.org/readerutil
u32 went where needed in pkg/blobserver/*
invertedBool went in pkg/types/serverconfig
atomics64 went in pkg/fs
Change-Id: I230426cda35be4b45ed67e869f14e6fdae89be22
Also add a flag to camtool discovery:
$ camtool discovery -httpversion
HTTP/2.0
Fixescamlistore/camlistore#661
Change-Id: I6db392e01bd176a2757ebdb4a0e881eebaec9b03
This change is in anticipation of moving pkg/images to go4.org, where it
should not depend on packages in third_party.
So:
third_party/github.com/nf/cr2 -> vendor/github.com/nf/cr2
third_party/github.com/rwcarlsen/goexif -> vendor/github.com/rwcarlsen/goexif
third_party/golang.org/x/image/tiff -> vendor/golang.org/x/image/tiff
Note that third_party/go/pkg/image/jpeg was also a dependency of
pkg/images. We had vendored image/jpeg from tip at the time because it
offered advantages over the version from Go1.3
(https://github.com/camlistore/camlistore/issues/463).
Since we now depend on Go1.5, we can go back to depend on the stdlib
version, so we simply remove third_party/go/pkg/image/jpeg and adjust
the imports accordingly.
Change-Id: Ifc8ffae0551102e644a0a0c67f3ff89e04df15c7
The conflict was introduced in cea5e332ce
by linking pkg/cmdmain in, which introduced its public flags.
Change-Id: I6374c2fb1657f6fd534c1342a5947ef6db16a3e2
At rev c3c16077b235db4c6b2a2a6121c2d429707f4ed4
And use it in new package pkg/camlegal
And remove pkg/legal
Change-Id: I5e398aa5f145a02a04d33719276fd2f8b82ab12b
Previously pkg/jsonconfig and pkg/errorutil
Copied from go4.org at rev d1b8a2fb2de6160036e4801aa5e4d855571078b8
Change-Id: I673ed55b0825baa2607289b6082f205100261d7a
They were internal packages (under pkg), which we are now moving to
go4.org, so we in turn need to vendor them in now.
Change-Id: I92224f731404d0bd4ca1c57492bed37cb3367ed4
If no metaIndex is specified, just go on, but don't panic.
Also try to clear the logic a little bit (don't overwrite indexConf).
Change-Id: I43e61ac3d5cd239cc113cb034ff3dd61eaaa14fd
Brad Fitzpatrick