Also fixed fakeindex clock to use a time.Time.
And synchronized the fakeindex and the indextests clocks to the
same origin.
Change-Id: I986abca300d61241069132619028f470cacd61da
When indexing upon a blob reception, we first populate
all the mutations in a map instead of in a batch mutation.
Then we transfer all the mutations in a batch and commit
it immediately. This makes the window when the batch mutation
is open much shorter, and will ease future indexing because
it allows reading from the index while writing the mutations
to the map.
Change-Id: I276282388f59ca543835bfa5ec64986453b23fe1
The index entry prefixed by "claim" had no keyType and
was always built "by hand" with pipes concatenation.
This change adds the documented keyPermanodeClaim to fix
that.
Change-Id: Ic59f7dbcccc6b223b155d5bffbf8e636209800cb
This method will be used to find out when a permanode or claim
was (un)deleted for the last time.
This change also introduces the keyDeletes index entry which
allows to find the permanodes/claims that a delete claim deletes.
Finally, since some index keys have changed, this CL introduces
a check for an index schema version, to force users to reindex
when needed.
http://camlistore.org/issue/191
Change-Id: I6f8e5d3030a6ddc8b21ff5dc5db311f2ea16f613
Start of something that builds and 'works', but barely fleshed out at all.
Likely to change drastically yet.
Change-Id: I7fba25528a341ff2dbbc4dbb579f2450d7889320
This change introduces a new index entry
to help with finding the children of a static directory.
It also fixes ResolvePrefixHop so that it takes
into account static directories, and not only collections.
This is the first step to support publishing static directories.
http://camlistore.org/issue/179
Change-Id: I5666e5caa6c782004054ae4c19a6b6119d4fda8b
Running 'go run dev/envvardoc/envvardoc.go' now shows:
'All environment variables are documented'
I also took the liberty of cleaning-up our mishmash of logic for handling
boolean environment variables, and cleaned up a couple other spots that didn't
seem right.
This change adds docmentation for all variables starting with (CAM|DEV|AWS).
This leaves some variables still undocumented. If there are variables worth
documenting in the following list, maybe we should rename them to have a
CAM{LI} prefix for consistency's sake:
APPDATA pkg/osutil/paths.go:86
APPDATA pkg/osutil/paths.go:102
DISPLAY pkg/misc/gpgagent/gpgagent.go:126
GOPATH pkg/fileembed/genfileembed/genfileembed.go:321
GOPATH pkg/osutil/paths.go:168
GOPATH pkg/test/world.go:54
GOPATH server/appengine/build_test.go:77
GPGKEY cmd/camput/init.go:77
GPG_AGENT_INFO cmd/camput/init.go:153
GPG_AGENT_INFO pkg/misc/gpgagent/gpgagent.go:50
HOME pkg/jsonsign/keys.go:79
HOME pkg/jsonsign/signhandler/sig.go:64
HOME pkg/osutil/paths.go:36
HOMEPATH pkg/osutil/paths.go:34
PKG_CONFIG_PATH pkg/index/sqlite/dbschema.go:59
RUN_BROKEN_TESTS pkg/fs/fs_test.go:67
SKIP_DEP_TESTS pkg/test/testdep.go:29
TERM pkg/misc/gpgagent/gpgagent.go:133
TERM pkg/misc/pinentry/pinentry.go:99
TESTING_PORT_WRITE_FD pkg/webserver/webserver.go:135
TEST_GPGAGENT_LIB pkg/misc/gpgagent/gpgagent_test.go:27
USER pkg/netutil/ident.go:135
USER pkg/osutil/paths.go:45
USERNAME pkg/jsonconfig/eval.go:228
USERNAME pkg/osutil/paths.go:43
VERBOSE_FUSE pkg/fs/fs_test.go:133
VERBOSE_FUSE_STDERR pkg/fs/fs_test.go:137
XDG_CONFIG_HOME pkg/osutil/paths.go:104
Change-Id: Ief28710d3deefd1e65247cb5d3b1d8dde73e1f2d
Does a few things:
1) Adds gotaglib to third_party. If you'd like to review that, feel
free, though there's a bit of organization I'd like to do first.
2) Adds an "audioTag" key type.
3) Indexes wholerefs by various audio tags. Doesn't yet add a map from
wholeref to tags, but I can add that next.
Change-Id: I8e2a5bc27260086bad3351ac57973d1ac23cff44
Also added -port flag, and untied the port to the
password in devcam server, always 'pass3179' now.
http://camlistore.org/issue/212
Change-Id: Iadf693951763d47907be17c53d4807a4706eb150
Move up a layer to the HTTP. Also, start to remove ContextWrapper
stuff. We've done it differently for App Engine instead, and will do
it differently yet moving forward.
Also add blobserver.Receive and use it in most places, moving checksum
verification up a layer.
Bunch of other cleanup and TODO fixing too.
Much simpler and cleaner.
Change-Id: I12e56c5d4e53bfcf82bdd8fb0b6d57c248ff605c
misc.CountingReader moves into readerutil.
pkg/atomics is folded into pkg/types.
pkg/test/testdep is folded into pkg/test, with better name/docs.
Old cruft from pkg/webserver is deleted.
Change-Id: I3f72d8b29804254ef944995fb085837c878f79f5
* Fix dbinit command in dev server
* Make sslmode configurable in dbinit
* Don't fail on index schema setup when plpsql language already exists
Change-Id: Ic70461963f6266172d21cda8e6f071e2442d7651
The immediate concern was to make the index case sensitive,
but the values should be stored in binary anyway as this
layer is just obeying the IndexStorage interface and should
not be concerned with character sets.
http://camlistore.org/issue/130
Change-Id: Iee913b0bd9e5c0c32a5c5570310c2ac5d0cfddf8
images: DecodeConfig to get the predicted width
and height after EXIF correction
search&index: add GetImageInfo and use it in search
to predict the thumbnail dimensions
http://camlistore.org/issue/115
Change-Id: I358136a2ab03ea09c8f8fd2fa0dc574921c819c5
This change allows to find all roots (permanodes with the
"camliRoot" attribute) from the search.html ui page.
To achieve that, func (x *Index) SearchPermanodesWithAttr
now uses a prefix string without the query part if the
query is "".
http://code.google.com/p/camlistore/issues/detail?id=35
Change-Id: I396fba683e6e7b2296d1f0df8009c07e3c2cd09d
The mongo indexer didn't have any authentication
scheme, so this change adds the user:pass@host
authentication.
The mongo indexer tests now use the root:root
credentials.
Change-Id: I7867f46e3b5f0a4a1a50f86f26d76c0bfd69e9ac
Now all tests pass like this:
bradmac-2:camlistore.org $ SKIP_DEP_TESTS=1 go test ./pkg/...
? camlistore.org/pkg/auth [no test files]
ok camlistore.org/pkg/blobref 0.023s
ok camlistore.org/pkg/blobserver 0.032s
? camlistore.org/pkg/blobserver/cond [no test files]
? camlistore.org/pkg/blobserver/google [no test files]
ok camlistore.org/pkg/blobserver/handlers 0.031s
ok camlistore.org/pkg/blobserver/localdisk 1.693s
? camlistore.org/pkg/blobserver/remote [no test files]
? camlistore.org/pkg/blobserver/replica [no test files]
? camlistore.org/pkg/blobserver/s3 [no test files]
? camlistore.org/pkg/blobserver/shard [no test files]
? camlistore.org/pkg/cacher [no test files]
ok camlistore.org/pkg/client 0.029s
? camlistore.org/pkg/errorutil [no test files]
? camlistore.org/pkg/fileembed [no test files]
? camlistore.org/pkg/fileembed/genfileembed [no test files]
? camlistore.org/pkg/fs [no test files]
ok camlistore.org/pkg/googlestorage 0.034s
? camlistore.org/pkg/httputil [no test files]
ok camlistore.org/pkg/index 0.298s
? camlistore.org/pkg/index/indextest [no test files]
ok camlistore.org/pkg/index/mongo 0.524s
ok camlistore.org/pkg/index/mysql 0.020s
ok camlistore.org/pkg/jsonconfig 0.022s
ok camlistore.org/pkg/jsonsign 0.155s
ok camlistore.org/pkg/lru 0.015s
ok camlistore.org/pkg/magic 0.030s
? camlistore.org/pkg/misc [no test files]
ok camlistore.org/pkg/misc/amazon/s3 0.028s
ok camlistore.org/pkg/misc/gpgagent 0.018s
ok camlistore.org/pkg/misc/httprange 0.026s
? camlistore.org/pkg/misc/pinentry [no test files]
? camlistore.org/pkg/misc/resize [no test files]
ok camlistore.org/pkg/netutil 0.088s
ok camlistore.org/pkg/osutil 0.016s
ok camlistore.org/pkg/rollsum 0.027s
ok camlistore.org/pkg/schema 0.017s
ok camlistore.org/pkg/search 0.016s
ok camlistore.org/pkg/server 0.039s
ok camlistore.org/pkg/serverconfig 0.031s
? camlistore.org/pkg/test [no test files]
? camlistore.org/pkg/test/asserts [no test files]
? camlistore.org/pkg/test/testdep [no test files]
? camlistore.org/pkg/webserver [no test files]
Change-Id: I9c9b0ee4f51cbcf785aed665bbb494339dd84261
It now uses the generic indexing layer, rather than its own schema.
Also, move it under pkg/index/, like mongo.
Also, link in mongo to the main binary.
Not using the tests yet (leaving for mpl :))
Change-Id: I47a2203ca6c0b5244cd458aedec46227c4363c62
fixed indexer so that it builds as a package
added mgo as a third party
fixed mgoindexer
fixed (hackishly) tests so that they pass
I ignored everything memindex for now
Change-Id: I3c03ecfa859cf92b088a759239278b1c0edaf8aa
A lot is still broken, but most stuff at least compiles now.
The directory tree has been rearranged now too. Go libraries are now
under "pkg". Fully qualified, they are e.g. "camlistore.org/pkg/jsonsign".
The go tool cannot yet fetch from arbitrary domains, but discussion is
happening now on which mechanism to use to allow that.
For now, put the camlistore root under $GOPATH/src. Typically $GOPATH
is $HOME, so Camlistore should be at $HOME/src/camlistore.org.
Then you can:
$ go build ./server/camlistored
... etc
The build.pl script is currently disabled. It'll be resurrected at
some point, but with a very different role (helping create a fake
GOPATH and running the go build command, if things are installed at
the wrong place, and/or running fileembed generators).
Many things are certainly broken.
Many things are disabled. (MySQL, all indexing, etc).
Many things need to be moved into
camlistore.org/third_party/{code.google.com,github.com} and updated
from their r60 to Go 1 versions, where applicable.
The GoMySQL stuff should be updated to use database/sql and the ziutek
library implementing database/sql/driver.
Help wanted.
Change-Id: If71217dc5c8f0e70dbe46e9504ca5131c6eeacde
Brad Fitzpatrick