Commit Graph

581 Commits

Author SHA1 Message Date
Brad Fitzpatrick 7ffb3fb66c index: unreverse time
Change-Id: I1ae6a0c0695e0dfa6626999879dfbb9a05c787df
2011-11-27 11:41:20 -05:00
Brad Fitzpatrick 5ea8cf8d26 index: add doc, notes, reverse time, start of tests.
Change-Id: I13bae252be9616f39decbfb4cc60eaa92c0f3db9
2011-11-27 11:36:30 -05:00
Brad Fitzpatrick 3dd1dcbb70 simplify the Index.GetRecentPermanodes method a bit
Change-Id: Iced53892d3a1ab7633762e8c59986a59d5dc3d90
2011-11-27 10:52:39 -05:00
Brad Fitzpatrick 20975cf2f8 rearrange some indexer code, comments, add some todos.
Change-Id: I94b1a68a8ff0095e2a758d01dcbe8253da57fb27
2011-11-27 10:46:51 -05:00
mpl 0e71f20415 auth: added username, camput can do https
Change-Id: I14a047f1a164b64062d94da0db7a64a658b086ae
2011-11-27 01:29:24 +01:00
Brad Fitzpatrick fff17c762b jsonconfig: remove use of container/vector (not part of Go 1)
Change-Id: I55e9595f351ca16589287a8729243a2fe456d2c4
2011-11-26 16:10:56 -05:00
Brad Fitzpatrick d12b671c4c jsonconfig: add file include tests
Change-Id: I0db5b4017c4dd68f70f7d90124c49990454c16f0
2011-11-26 16:05:08 -05:00
Brad Fitzpatrick 911478aa30 Add dev-server --memory flag to disable mysql indexer and use in-memory index.
This is for testing the new generic indexing infrastructure.

Change-Id: I83798558a3de6ab2c53ae10e47f7189461043420
2011-11-26 15:30:22 -05:00
Brad Fitzpatrick 04b79123c6 jsonconfig: expand elements in arrays too. more tests.
Change-Id: If40d9ece8d57ebb91e191aea5caa8dcc304733c3
2011-11-26 15:26:42 -05:00
Brad Fitzpatrick 90a3d4b6a5 jsonconfig: bool support in _env expansions.
Change-Id: I165a6f304b23fd6bff50a7addfebff9dfabc7075
2011-11-26 14:37:10 -05:00
Brad Fitzpatrick cbdf4aeb6f Merge "force upper bound when rollsum splitting" 2011-11-12 18:29:55 +00:00
Brad Fitzpatrick 5c12ac37f3 generic index: untested stat support.
Change-Id: Idff5ff19d7b18ef6581c6c8bc344230470c468ef
2011-11-10 18:58:04 -08:00
Brad Fitzpatrick 0c8284f1af memory index enumerate support. untested.
Change-Id: I28df97ed6385675545a74a6e01e779cc627ef033
2011-11-10 10:28:44 -08:00
Brad Fitzpatrick cc85ec5bda start of enumerate and stat for generic indexer
Change-Id: Iabac9959a239d2418ab351e37895c7f0721e4e58
2011-11-10 09:47:32 -08:00
Brad Fitzpatrick edf4da987e more index work
Change-Id: I8c8885aafb6b12fe9f24760ba8168ed553cee616
2011-11-09 17:15:58 -08:00
Brad Fitzpatrick 6b8428cda7 More index work; start of Index and BatchMutation definitions.
Change-Id: Ie59d973e9bef50ffa054d2b61123add04740a4ff
2011-11-09 08:21:19 -08:00
Brad Fitzpatrick 4625afad79 more generic index work: start of in-memory leveldb index for test/development.
Change-Id: Iec26c7ffcfc8c87f247d6316f6495d97c6a9a5c4
2011-11-07 08:40:31 -08:00
Brad Fitzpatrick 803044a526 add Makefile to .gitignore in lib/go
Change-Id: I27bb9ee39a246f89437b02068687d34e817e94c7
2011-11-07 08:38:56 -08:00
Brad Fitzpatrick 3b4ab16730 move mysqlindexer's blobsniffer to common index package.
Change-Id: I36c1700982c0a51de1521309ba9a03876fde06d4
2011-11-07 08:17:04 -08:00
mpl d7ce1b71ff started on the histogram (issue 18)
Change-Id: I5f90e8c96664221281ac04b4d49de53f352d90fa
2011-11-06 17:32:33 +01:00
Brad Fitzpatrick 8f2a133210 add leveldb testdata files
Change-Id: Ica2829ad06f6edd94109b238209bad6e88c4e963
2011-11-01 18:37:58 -07:00
Brad Fitzpatrick aaa2e83e93 start of generic indexer
Change-Id: I863e829dc03422daff243f67a3bdf48e25c014b0
2011-11-01 18:37:27 -07:00
mpl e9650edc63 force upper bound when rollsum splitting
Change-Id: I4ec43cf42049060ccab7bbb3f8c598895d4c52d9
2011-10-31 01:17:34 +01:00
Brad Fitzpatrick b7b52a29d9 Add leveldb-go to the build.
Change-Id: I697e99cf17010d8224be302c9abba3978d709c5b
2011-10-30 12:47:04 -07:00
Brad Fitzpatrick 07bc415eec Merge "hacky (but working) draft for caching thumbnails" 2011-10-28 17:34:59 +00:00
mpl 93ea543b39 hacky (but working) draft for caching thumbnails
Change-Id: I3c4714e76089c406019cf051c07ab05546af34ed
2011-10-28 19:25:10 +02:00
Brad Fitzpatrick b223044009 app engine work: WrapContext on cond, replica, and aeindex.
Change-Id: I8667413c241acd99387184d561beee4d1aec623c
2011-10-27 22:08:55 -07:00
Brad Fitzpatrick 59ee1aba43 replica: better error handling / logging
Change-Id: I7a21cff6a617a9ce68a5173705970ab4fb68d4a9
2011-10-27 14:27:09 -07:00
Brad Fitzpatrick c71dee2bc2 app engine index: close dest channel in GetRecentPermanodes
Change-Id: I6c23f255513ef62a223550c31ec608b2ad62850c
2011-10-27 14:01:10 -07:00
Brad Fitzpatrick a53ef6e257 more App Engine bootstrap work.
Change-Id: I11a141b833bf1940aadcd0cd20adc53da8ea4151
2011-10-25 19:40:50 -07:00
Brad Fitzpatrick ec64139bf1 get: gofmt, and better 404 message
Change-Id: I1c8f62223399f6d7b68c48435a078e83899bdbe8
2011-10-18 14:35:44 -07:00
Brad Fitzpatrick e351525afd Move all the camlistored handlers (UI, signing, etc) into a package.
Change-Id: Idf63c83cd7faf006e4a9047b47a16ea2950e5147
2011-10-18 11:12:01 -07:00
Brad Fitzpatrick 916073b87b fileembed: extend environment-variable mode to hard-coded directory loading.
also, slurp to memory mode.  for App Engine.

Change-Id: Ib6de390231e4669e7a6145a8d40b646744fd028c
2011-10-18 11:10:02 -07:00
Brad Fitzpatrick c38d9f6bc7 serverconfig: don't mask handler setup errors
Change-Id: If701f69292f0887a64ef10fea5830cb623c1ef27
2011-10-18 10:42:09 -07:00
Brad Fitzpatrick 70aa373ab9 schema: sort imports, add test-target to quiet dev_appserver noise.
Change-Id: I006980e9774ab2a31d2f202bf3bc987dd51508a9
2011-10-18 10:35:48 -07:00
Brad Fitzpatrick fe2d2187ee sort imports after gofix
Change-Id: Ia1437192bfda7d37d78e4c657ef19eef5d1e4ad0
2011-10-17 17:07:59 -07:00
Brad Fitzpatrick a77365cf53 Don't use syscall in camli/schema
Change-Id: I0ad2c765a80b5bab95f4f9eb146e2dc0e93fb4fe
2011-10-17 17:07:31 -07:00
Brett Slatkin 130b05eaeb App Engine: Fixes /bs/camli/stat to return the correct upload URL
Change-Id: I9062917fe052652c3714c99ab7f572f8f20e5dba
2011-10-13 00:00:54 -07:00
Brad Fitzpatrick da562c8bed jsonsign: don't check for default secretring file unless we don't have an entity fetcher
Change-Id: I9ea58b09c5ad5465f64ce36d0cbc4edcc00c5d9d
2011-10-12 22:59:47 -07:00
Brad Fitzpatrick abf719be0b fix bug where storageAndConfig.WrapContext was losing its Configer
Change-Id: I77d89830f0607e7633e14fa1bd4893afe394486d
2011-10-12 22:29:42 -07:00
Brad Fitzpatrick d0063aa48d fix stat crash
Change-Id: Ieb959b215a3be3f47781426a0a8be4b7421b1af6
2011-10-12 22:02:11 -07:00
Brad Fitzpatrick 73030a9ae9 fix crash
Change-Id: I11e9c5639e698090db26e8710b55aef7f2a64f96
2011-10-12 21:52:01 -07:00
Brad Fitzpatrick 11acecce92 Start of making canLongPoll configurable for App Engine to set it to false, for now.
Change-Id: Ib4d7a306276a7292e7d8a8838c3b026d38b5fb6b
2011-10-10 18:04:20 -07:00
Brad Fitzpatrick 453fd92fbe client: don't register client flags by default
Change-Id: I73a3242f1cfd2af9a63dd53c7174843a234c5f18
2011-10-10 17:38:18 -07:00
Brad Fitzpatrick 38b30381e0 client: always sent a ContentLength on upload posts.
Change-Id: I531d7f74b5dd3490e7d0725140ea5cd1f358c704
2011-10-10 17:01:52 -07:00
Brad Fitzpatrick 4fc24cba87 App Engine get support, untested, and fix Config shadowing breaking stat
Change-Id: I488a81c7968af746b727bcfc3ed8a4649107a26c
2011-10-09 18:41:54 -07:00
Brad Fitzpatrick e9991b1702 Barely tested App Engine stat handler support.
Change-Id: Ia8abc53e3f538198773940f9515ff551e97eb8de
2011-10-06 23:24:40 -07:00
Brad Fitzpatrick 2600d227eb introduce ContextWrapper to get App Engine its Context.
Change-Id: I789440f01a42b1172b6c70ff14f00ff5bd45786a
2011-10-06 17:44:30 -07:00
Brad Fitzpatrick 2096e2b911 Use := instead; err isn't shadowed
Change-Id: I12a3ef991105a2b48fd786003491507313925ce6
2011-10-05 17:30:54 -07:00
Brad Fitzpatrick 3294330774 skip this test for now
Change-Id: I135bdf3527abe63af91ca38710f032341bca6de1
2011-10-05 07:41:54 -07:00
Brad Fitzpatrick 3f5631d2d8 serverconfig: push abs path business to caller 2011-10-04 16:22:12 -07:00
Brad Fitzpatrick 7c2f4b57ef Temporary (?) experiment: start targetting release versions.
So we can run on App Engine sooner than later.

Change-Id: Ia572a16d02316716489fdd38c05d4cb63706004e
2011-10-03 16:50:01 -07:00
Brett Slatkin 1b926f66ff Break dependency from fileembed to syscall package.
Change-Id: Ibe518b1063636428beb5beeda96b01ab36d3716d
2011-09-30 05:37:09 +00:00
Brett Slatkin b967f04d4e camlistored: Refactor main() into tiny main and loader library
This will make running on App Engine a lot easier, letting us share code.

Change-Id: Ibe989387d44d94869c31c8306683e49af261496a
2011-09-30 05:07:04 +00:00
Brad Fitzpatrick 8c72348a19 Rename Stat->StatBlobs, Remove->RemoveBlobs; add StatReceiver.
Also, return missing errors in google storage stubs.

Change-Id: I5b168bbcd8a2996e3db4ffc35d2ac06006b3a2d3
2011-09-28 19:37:28 -07:00
Brad Fitzpatrick 7f48b1e9a9 split WriteFileFromReader into two parts; existing API and new one taking a map
Change-Id: Icac3aced7cdb0cf67c369f7198ff959f5a416e9c
2011-09-28 11:07:07 -07:00
Brad Fitzpatrick 6c78d45507 make Client.Stat use auth; fix handle size -1 bugs
Change-Id: I5ca46e9996e018727b082895d76a047b62ffed2d
2011-09-28 11:06:16 -07:00
Brad Fitzpatrick 13806ff26d add remote.NewFromClient constructor
Change-Id: Id49805a427de0c5adcc2aca956cd96ea817a41f8
2011-09-28 11:05:35 -07:00
Brad Fitzpatrick 8b758385d5 put SizedBlobRef String method on the value
Change-Id: Ibfefbc3d21bf4f0cdf7ef729982a02478416802b
2011-09-28 11:05:02 -07:00
Brad Fitzpatrick 42792ccaa0 gofmt
Change-Id: Ia0167859e9acea25f0a2d8c8efff53bb3683add5
2011-09-27 16:02:28 -07:00
Brad Fitzpatrick 12f236da37 Start of major camput command-line cleanup.
So far only "init" and "file" are done:

$ grep -r RegisterCommand clients/go/camput
clients/go/camput/init.go:      RegisterCommand("init", c)
clients/go/camput/files.go:     RegisterCommand("file", cmd)

New command line syntax is:

Usage: camput [globalopts] <mode> [commandopts] [commandargs]

Examples:

  camput init

  camput file [opts] <files/directories>

  camput permanode [opts] (create a new permanode)

  camput share [opts] <blobref to share via haveref>

  camput blob <files>     (raw, without any metadata)
  camput blob -           (read from stdin)

  camput attr <permanode> <name> <value>         Set attribute
  camput attr --add <permanode> <name> <value>   Adds attribute (e.g. "tag")
  camput attr --del <permanode> <name> [<value>] Deletes named attribute [value]

For mode-specific help:

  camput MODE -help

Change-Id: I4aeabeff6450beab83c13d94c16bd007addb8b58
2011-09-25 17:40:01 -07:00
Brad Fitzpatrick 65a9728674 gpgPath wasn't used anywhere
Change-Id: I783a70d8cccea6d36a8f451e28c71e60a826eea0
2011-09-25 17:38:58 -07:00
Brad Fitzpatrick 0d31843933 upload: make channel buffered to prevent goroutines potentially hanging; close pipe explicitly to help catch errors
Change-Id: I47e150dba75cad6c9fc58e2895e4bde5f55cac21
2011-09-24 14:13:57 -07:00
Brad Fitzpatrick 8cef40d581 Merge branch 'master' of ssh://camlistore.org:29418/camlistore 2011-09-20 18:30:29 -07:00
Brad Fitzpatrick e6b53c1f34 client: close Body earlier, don't gaurd against nil
Change-Id: I6b06db6d586967478b2ae6d880587b0234d8eb0b
2011-09-20 18:30:15 -07:00
Brad Fitzpatrick d4c0490942 localdisk: ignore link race
Change-Id: If85c3dcea5fdde7c9fb9e24b4074c34230c98ed5
2011-09-20 18:28:49 -07:00
Brad Fitzpatrick f1f121fef6 Now that Go bug 2267 is fixed, update this.
Change-Id: I3794c12e68231a3e4d0fdb3f6ee376f934833f50
2011-09-20 12:04:10 -07:00
Brad Fitzpatrick 6f91469776 camput statcache
Change-Id: I0b469b3497b7ae87f0de039a1398c3102139cdf4
2011-09-17 15:14:37 -07:00
Brad Fitzpatrick 9489f98911 gob support for blobref
Change-Id: I03c0cc9130d694f7015d2c92a6cf5b715b379883
2011-09-16 17:59:15 -07:00
Brad Fitzpatrick f9e453e666 add CacheDir to osutil
Change-Id: I8f6b1d6c8d1b275d329eeac2d91e15a3582e59fb
2011-09-16 17:56:49 -07:00
Brad Fitzpatrick e636911300 add notes file
Change-Id: Ib61e0bbb14447e5997d3c1ac525526e5c3ad8e3e
2011-09-16 11:56:49 -07:00
Brad Fitzpatrick a05b4949cb shrink some columns, use ENGINE= instead of TYPE=
Change-Id: Idac91f3ff1ea3dbab9bb11f7ee4b1f383ab1878b
2011-09-15 21:57:32 -07:00
Brad Fitzpatrick d35c4a4ddc minor client/upload changes
Change-Id: I02ec44ff72575e547fee5e44667d648cf06b54f8
2011-09-15 21:52:33 -07:00
Brad Fitzpatrick 3eb3b994ca Go update; gofix for image changes
Change-Id: Id58f0abe112506ba665df37b99414360c236fc43
2011-09-15 21:21:02 -07:00
Brad Fitzpatrick 31488a1d5f Issue 43: clean and rename the "files" table and its columns
Part of Issue 17 (the files/bytes change)

Details:
http://code.google.com/p/camlistore/issues/detail?id=43
http://code.google.com/p/camlistore/issues/detail?id=17

Change-Id: Ic6ebc018edf51f92bf2063c42fa19f80fe597181
2011-09-13 19:56:58 -07:00
mpl d18bd115f2 update fields when inserting permanode post claim.
fixes issue 41.

Change-Id: Id81b4d9df273c7c71388b7ac93f5eb626a5e298c
2011-09-13 23:31:05 +02:00
Brad Fitzpatrick cd3eb19b75 Merge "Issue 17: file vs. bytes schema change" 2011-09-13 19:18:14 +00:00
Brad Fitzpatrick c0ea8dbcb1 Issue 17: file vs. bytes schema change
Details:
http://code.google.com/p/camlistore/issues/detail?id=17

Change-Id: Ia1237952a2a66f1dcacd00c5c28eb360d9f1d1d7
2011-09-13 11:57:50 -07:00
mpl 03e2da9f9a set error status code in serveRecentPermanodes
Change-Id: If7e982b1c35700823ca181e1331e404766a9ebd5
2011-09-04 22:13:16 +02:00
mpl b0dda5e86f Various renamings. dropped getTagged.
Change-Id: I20982074071b819110fe07cb795ffcb7d6c8560e
2011-09-04 00:48:49 +02:00
mpl aa83d21048 added search by title,
merged with search by tag.
added general search.

Change-Id: I98244e62dbd8a4c75527c4a029bd78f8978cbadc
2011-09-02 13:45:55 +02:00
Brad Fitzpatrick 26da0d2be4 db: make stmt.Exec also return Results
Change-Id: I5f32e77fbae7db0e9f601b7f581fd55dc261e67f
2011-08-28 22:16:58 +04:00
Brad Fitzpatrick 1cb46af3d5 db: optional Execer interface
Change-Id: Ied473f38c3cef5b68e59e1e557b6dbcfc410ac2d
2011-08-28 22:12:41 +04:00
Brad Fitzpatrick 10fecbff2c db: Stmt.QueryRow, more docs, tests
Change-Id: I6ae89798520f170ebcb922946435c10348e8e7a9
2011-08-28 21:56:32 +04:00
Brad Fitzpatrick 13f060be3f db: rename to NullableString, add docs
Change-Id: I861f3f88ce2678ebf4a0d53905825b4ecd3ba448
2011-08-28 21:37:25 +04:00
Brad Fitzpatrick 33d4b06492 db: more type conversions
Change-Id: I8e950fcad9d7e4d3609c779c17de87e64feaad9f
2011-08-28 21:32:28 +04:00
Brad Fitzpatrick 543b3876a2 db: implement MaybeString and ScannerInto
Change-Id: I3884e0b004bd0a9b9ad1de838ad1aeeb4f303fa5
2011-08-28 20:58:32 +04:00
Brad Fitzpatrick 0f5a0467d1 db: scanning, working queries, and where clauses in fakedb
Change-Id: Iac9cf913a8857ba122bc3e155c11506df529104c
2011-08-28 20:50:05 +04:00
Brad Fitzpatrick 02c5ccfd2a db: more type conversion work, this time in the db package
Change-Id: Iff03c9b24bb73d6389d2763f85cd3c7135b4202c
2011-08-28 20:49:00 +04:00
Brad Fitzpatrick 577916939d db: query work
Change-Id: I5cf36f64d6894ce7522129db2724044fe0e1ae30
2011-08-28 16:34:03 +04:00
Brad Fitzpatrick 88c89757e1 gofixes for latest Go changes
Change-Id: Ib7b3aea5adff0e1683bcb421cc3549c31d9fe174
2011-08-25 19:14:47 +04:00
Brad Fitzpatrick 7f2f506132 camli/db: typecheck & convert pre-bound statement parameters
Change-Id: Ic9b6dfa7b1fef0c6046c2e0454e996e2d5a8dfa6
2011-08-25 18:50:45 +04:00
Brad Fitzpatrick 7095174178 camli/db: move SubsetValue into dbimpl
Change-Id: I81e965fd3e24d642925651225e1cc968ab015a72
2011-08-25 18:35:39 +04:00
Brad Fitzpatrick 3a29e1c245 camli/db: more type conversion work
Change-Id: Icbbf4a084017478ba74df85f4c31be6541d7b44e
2011-08-25 18:25:12 +04:00
Brad Fitzpatrick 2f1527a71d camli/db: start of more tests
Change-Id: If8d04b9d088363f4aed8fb1959c2847e9a527792
2011-08-25 17:51:22 +04:00
Brad Fitzpatrick df6b7ac86e Merge branch 'master' of ssh://camlistore.org:29418/camlistore 2011-08-25 17:33:46 +04:00
Brad Fitzpatrick 3c886de92b camli/db: start of type conversion work
Change-Id: I5467eee286e1752b3961c6fe3018fb78795b362f
2011-08-25 17:31:24 +04:00
mpl f8429bbf03 tiny typo
Change-Id: I2aec96e56c056be976528c5c2e81e8fc03d4a29a
2011-08-24 23:10:28 +02:00
Brad Fitzpatrick a9853a4622 Merge branch 'master' of ssh://camlistore.org:29418/camlistore 2011-08-24 18:23:31 +04:00
Brad Fitzpatrick 57a5439648 camli/db: some more insert work
Change-Id: I80af5b64ba834674df36bafea75ab84ce5922ea3
2011-08-24 18:23:14 +04:00