Arbitrary limit, but prevents things from exploding during request spikes.
Fixescamlistore/camlistore#683
Change-Id: I3699d25416fb6bbc1ea4f3b984b0ce05e6abf31b
This change allows (for example) to hook a blobserver and an indexer
together in a way similar as they would be in a default Camlistore setup.
This is particularly useful to setup tests that require a realistic
indexer, without having to create a heavier test with e.g. test.World.
This actual goal of this change is described below. I kept these changes
together because the one described below illustrates the need for the
synchandler changes.
pkg/index/stress/: stress test the indexer with more data
The goal of these benchmkarks is to help us compare the various sorted
implementations, so we can pick the most efficient one as a default.
A second, harder, goal would be to add tests that help us estimate the
reliability of the various kv stores, again so we can rule out the buggy
ones, or help fix them.
Change-Id: I8a1fe50398a4ba09c03786502b68b6c2599e5984
Otherwise when openOrCachedDB is called for the second time,
it would return an actually closed DB.
Change-Id: I820834e508fed1a34ab7bd19a8a91e398badff9c
Previously pkg/jsonconfig and pkg/errorutil
Copied from go4.org at rev d1b8a2fb2de6160036e4801aa5e4d855571078b8
Change-Id: I673ed55b0825baa2607289b6082f205100261d7a
See https://golang.org/s/go15vendor
This commit moves the google.golang.org/cloud/... packages. The rest
of third_party will move later.
This commit is a dependency for some cloud logging work which we'll
develop in Camlistore's tree.
In a few places, cznic/kv use was hardcoded, instead of picking the user
set kv type, if any. This change makes sure we use the user-defined one,
if set, syndtr/leveldb otherwise.
Relatedly, the low-level config generated for diskpacked now uses that
same principle for its internal index. Also, if a diskpacked is
configured without an index, it now defaults to syndtr/leveldb.
Please note that this patch DOES NOT change the default kv type in the
automatically generated high-level config.
Related: #632
Change-Id: I55dbd385230878e92776e6b83d34c36fd1fa9382
Store the strings, as the TODO said.
This memoization saves 9/10 of running time:
BenchmarkSql-4 (before) 300000 5429 ns/op
BenchmarkSql-4 (after) 3000000 430 ns/op
Change-Id: I4edabff96043620b5af3f5b507ddd75d65b22ff1
Also, delete my old gce package from third_party and only use the
google metadata package (which my gce package became, and which was
also already vendored into third_party)
Fixes#596
Change-Id: I64fd6f1e9dc6f433466f91f81efd2ecbf039334f
oauth2 at 267028f9bc2a1177dc5769be38c68c1b4fbe91c4
google cloud at 2e43671e4ad874a7bca65746ff3edb38e6e93762
google api at fc402b0d6f2a46ba7dcf0a4606031f45fb82a728
(partial commit only, since it's 16+ MB)
And update all callers from the old
code.google.com/p/google-api-go-client over to google.golang.org/api.
Change-Id: Ide190e04bb863928f98369a7c4d2b8c19fda1f66
Modify pkg/sorted/kvfile/kvfile.go to not have a named
return in Next - just as in pkg/sorted/leveldb/leveldb.go.
Change-Id: I0d46a86b33d2e1d2992d8a41f08979eb21aaddbb
Because we do not want the database name to be optional with a
placeholder (/*DB*/) in the statement that creates the database, since
it is not actually optional there - as opposed to when creating tables.
Change-Id: I05351d76d95071492d763758a11454f219524510
This is so configurations using the same database server for many
databases (multiple queues + the index) can all share the same
underlying *sql.DB
Change-Id: Iae7d4835fe9045008709c953e36b729729b1b6f4
from github.com/go-sql-driver/mysql at 9a7aa3606b82e2081a13a008ada88dfdb96c20fd
Context: http://camlistore.org/issue/428
Change-Id: Ia93ac35da02933bd03d2c6b7b8e5e549ba0472cb
Two important related changes:
1) sorted/mysql now takes into account the host given in the config
2) the required tables are now automatically created by NewKeyValue
http://camlistore.org/issue/263
Change-Id: I0043f36edb0630d6484148508d3a1e08c8e88a94
Related changes:
Split docker-related test-helper functions from pkg/sorted/mongo.
These helper functions are now also used in the pkg/blobserver/mongo
tests too.
Also fixed a typo in a comment in pkg/blob/fetcher.go and a missing variable in
debug output in pkg/blobserver/storagetest/storagetest.go
Addresses http://camlistore.org/issue/127
Change-Id: I8b6f57f9ced066d6f83788fdcc87be6619c65c3c
Conflicts:
pkg/blob/fetcher.go
Under a second to start and test against a real MongoDB instace in Docker:
$ go test -v camlistore.org/pkg/sorted/mongo
=== RUN TestMongoKV
--- PASS: TestMongoKV (0.79 seconds)
PASS
ok camlistore.org/pkg/sorted/mongo 0.791s
Test is skipped in short mode, or if docker isn't available.
Change-Id: I6f3acc96357cf9c50054b3df6727661c9329e6bf
Brad Fitzpatrick