Commit Graph

12 Commits

Author SHA1 Message Date
Brad Fitzpatrick 0457e9359a Start of making QueueCreator optional for replication sources.
Change-Id: I82290991208c6e8953bdc63424760b378437674d
2013-09-01 16:19:07 -07:00
mpl e036f96488 sync: delay copy retry on specific errors
http://camlistore.org/issue/206

Change-Id: I1dd07149352e3af6b39bcb86ed2312f19c3bae30
2013-08-30 19:51:04 +02:00
Brad Fitzpatrick b24cad68dd Cleanup: remove BlobHub and time.Duration waits from storage interface
Move up a layer to the HTTP.  Also, start to remove ContextWrapper
stuff.  We've done it differently for App Engine instead, and will do
it differently yet moving forward.

Also add blobserver.Receive and use it in most places, moving checksum
verification up a layer.

Bunch of other cleanup and TODO fixing too.

Much simpler and cleaner.

Change-Id: I12e56c5d4e53bfcf82bdd8fb0b6d57c248ff605c
2013-08-21 13:57:28 -07:00
mpl 4acc10e6e4 serverconfig: idle synchandler when no localdisk as primary storage
Because no localdisk means either s3 or google is the primary,
and none of them support efficient replication.

1) Added a dummy synchandler constructor for when config has "idle"
2) Set "idle" for synchandler config when no localdisk
3) fixed corresponding tests

Also:
- added error (and test) when no localdisk and both s3 and google
in config
- added s3 + mysql test

http://camlistore.org/issue/201

Change-Id: I861fdca0c203bc0181ab6d548adab501ed98d2f0
2013-08-21 15:17:13 +02:00
Brad Fitzpatrick 0bdf20884b all: delete pkg/blobref; convert all from *blobref.BlobRef to new blob.Ref
Change-Id: Id2dfb7f19452bedf4f3c9310b36227fd8117b225
2013-08-03 19:54:30 -07:00
Brad Fitzpatrick 9468e5ba70 More docs. Every package is documented now.
misc.CountingReader moves into readerutil.

pkg/atomics is folded into pkg/types.

pkg/test/testdep is folded into pkg/test, with better name/docs.

Old cruft from pkg/webserver is deleted.

Change-Id: I3f72d8b29804254ef944995fb085837c878f79f5
2013-07-07 21:12:30 -07:00
Brad Fitzpatrick ca58d8e2e0 Remove noisy var _ = log.Printf lines.
Change-Id: Ia58b8ef5f271f542ae4fe61c7fb1497322770322
2013-06-14 12:55:55 -07:00
mpl 3129e70380 discovery: add synchandlers
Change-Id: I2934c4a7d926770eaf59cb82f0fe48b6c8deb225
2013-01-12 01:55:32 +01:00
Brad Fitzpatrick caa50142dc sync: fix fd leak, update a TODO
Change-Id: I772ac951723300e4e5e767d7a25d6d4bcda3e825
2012-12-31 18:21:50 -08:00
Brad Fitzpatrick a41269e78e Reindex all dev-server blobs into memindex on restart.
Required some sync work (full syncs on start, blocking full syncs on
start, and also adding a dev-only hack to force a depedency from
search -> sync, to control the handler initialization order, otherwise
publish handlers would race with the sync handler and they'd create
new "blog" and "pics" permanodes and we'd end up with duplicates).
2012-11-07 22:40:17 +01:00
Brad Fitzpatrick 5c4d0f71f5 gofmt 2012-11-07 18:49:14 +01:00
Brad Fitzpatrick 0714a463c9 Update from r60 to [almost] Go 1.
A lot is still broken, but most stuff at least compiles now.

The directory tree has been rearranged now too.  Go libraries are now
under "pkg".  Fully qualified, they are e.g. "camlistore.org/pkg/jsonsign".

The go tool cannot yet fetch from arbitrary domains, but discussion is
happening now on which mechanism to use to allow that.

For now, put the camlistore root under $GOPATH/src.  Typically $GOPATH
is $HOME, so Camlistore should be at $HOME/src/camlistore.org.

Then you can:

$ go build ./server/camlistored

... etc

The build.pl script is currently disabled.  It'll be resurrected at
some point, but with a very different role (helping create a fake
GOPATH and running the go build command, if things are installed at
the wrong place, and/or running fileembed generators).

Many things are certainly broken.

Many things are disabled.  (MySQL, all indexing, etc).

Many things need to be moved into
camlistore.org/third_party/{code.google.com,github.com} and updated
from their r60 to Go 1 versions, where applicable.

The GoMySQL stuff should be updated to use database/sql and the ziutek
library implementing database/sql/driver.

Help wanted.

Change-Id: If71217dc5c8f0e70dbe46e9504ca5131c6eeacde
2012-02-18 21:53:06 -08:00