Rooba
/
perkeep
mirror of https://github.com/perkeep/perkeep.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,816 Commits 44 Branches 11 Tags 59 MiB
Commit Graph

582 Commits

Author SHA1 Message Date
Brad Fitzpatrick 8f38212408 Update for gofix httpheaders 2011-06-16 18:38:21 -07:00
Brad Fitzpatrick a170c567a3 Merge "auth: add SendUnauthorized" 2011-06-15 08:38:14 -07:00
Brad Fitzpatrick 6b77e2dd57 Merge "httputil: add ForbiddenError and RequestEntityTooLargeError" 2011-06-15 08:37:18 -07:00
Julien Danjou b07e7f61b3 auth: add SendUnauthorized 
Change-Id: I6bf97fee5097af56dfa4b1d1cd1157f5e15d08e5
Signed-off-by: Julien Danjou <julien@danjou.info>
 2011-06-15 11:48:36 +02:00
Julien Danjou 310e0474be httputil: add ForbiddenError and RequestEntityTooLargeError 
Change-Id: Ie013c2b79bd75b3c1ba14f94b334bb009b61c4d3
Signed-off-by: Julien Danjou <julien@danjou.info>
 2011-06-15 11:48:28 +02:00
mpl 34c776c81c thumbnail support 
Change-Id: Ie0ee67ce278c1d9790c43a9270fc909dd6e68923
 2011-06-15 10:42:18 +02:00
Brad Fitzpatrick 20c4ef7154 search / file work 2011-06-12 00:20:57 -07:00
Brad Fitzpatrick 72c4a42d06 more jpeg sniffing 2011-06-12 00:13:06 -07:00
Brad Fitzpatrick 2086b96993 errorText to error 2011-06-11 20:10:10 -07:00
Brad Fitzpatrick 691273a284 index more about files: filename, properties set, mime type 2011-06-11 10:55:35 -07:00
Brad Fitzpatrick 42d80271d8 show children in describe 2011-06-11 10:12:39 -07:00
Brad Fitzpatrick 60e3fa2574 more content -> camliContent cleanup 2011-06-11 09:51:08 -07:00
Brad Fitzpatrick d2b9f80a03 send error response JSON 2011-06-11 09:30:01 -07:00
Brad Fitzpatrick 179b706248 don't promote camliContent attr, make member be camliMember 2011-06-11 00:27:17 -07:00
Brad Fitzpatrick 3c6c4207be download handler: support for verifying a digest in a HEAD request 
for use in javascript uploads.
 2011-06-09 18:33:26 -07:00
Brad Fitzpatrick 6f35e66385 let file reader be closed 2011-06-09 18:28:07 -07:00
Brad Fitzpatrick bdfde0a5fe Advertise the dev server password in the realm. 2011-06-09 16:09:21 -07:00
Brad Fitzpatrick 0ada583435 UI: js file upload work 2011-06-09 15:00:20 -07:00
Brad Fitzpatrick 3b3b682c1e Search interface + impl to find file schema refs by their content ref 2011-06-09 12:55:38 -07:00
Brad Fitzpatrick e5acdb2e38 send 400 or 500 with some json replies 2011-06-09 12:54:53 -07:00
Brad Fitzpatrick 97978b56d4 don't retry forever indexing files. 2011-06-08 18:56:56 -07:00
Brad Fitzpatrick c61317703c indexer: index file sha1s, sizes (for js search lookup later) 2011-06-08 18:26:57 -07:00
Brad Fitzpatrick fa3715134a Start of indexer indexing file schema's content digests. 2011-06-08 17:49:31 -07:00
Brad Fitzpatrick a4c43c359f Add blobref.SeekerFromStreamingFetcher, moving it from UI code. 2011-06-08 17:48:56 -07:00
Brad Fitzpatrick f31959c24c Update for Go: use http.Values 2011-06-08 13:41:39 -07:00
Brad Fitzpatrick d65fc1391a schema & blobref: implement JSON [Un]Marshal interfaces, clean up code 2011-06-07 17:00:21 -07:00
caine tighe 1fabcfbeb0 osutil/paths.go : update for windows support 2011-06-07 16:25:44 -04:00
Brad Fitzpatrick 1f0aec0612 rollsum: add a benchmark, and more attribution 2011-06-06 19:09:58 -07:00
Brad Fitzpatrick 39ac2c072e schema: filereader support for hash tree files. 2011-06-06 13:30:54 -07:00
Brad Fitzpatrick 319a12f982 schema/filereader: support zero regions, start of subfile support 2011-06-06 11:31:45 -07:00
Brad Fitzpatrick fac6b7f020 schema filewriter: rollsum chunking version 2011-06-06 08:54:31 -07:00
Brad Fitzpatrick 5f86c64b95 schema filereader: verify top-level size matches part sizes when reading 2011-06-06 08:50:20 -07:00
Brad Fitzpatrick 8ab1532a15 rollsum: OnSplit, Bits. add camput debug mode for it. 2011-06-04 16:14:33 -07:00
Brad Fitzpatrick 742051f813 rollsum: Port of apenwarr's bar rollsum to Go. 2011-06-04 12:00:53 -07:00
Brad Fitzpatrick 1ed6a4a50c search: remove dup attributes 2011-06-04 10:18:38 -07:00
Brad Fitzpatrick 8d84a4de10 Add blobref.SeekTester interface; fixing UI downloads from cond storage. 2011-06-04 09:58:50 -07:00
Brad Fitzpatrick b652dc62e8 make CachingFetcher also a StreamingFetcher 2011-06-04 09:12:02 -07:00
Brad Fitzpatrick 59e577c023 Rename blobref.Fetcher to blobref.SeekFetcher 2011-06-04 08:56:03 -07:00
Brad Fitzpatrick 0df00167ec cond: implement it (routing incoming blobs) and use it. 2011-06-04 08:46:42 -07:00
Brad Fitzpatrick 0a7752afe5 jsonconfig: RequiredStringOrObject, OptionalStringOrObject 2011-06-04 08:45:03 -07:00
Brad Fitzpatrick c6cfe2fda4 Start of 'cond' handler. 2011-06-03 21:52:56 -07:00
Brad Fitzpatrick 811f75954a magic: add a fake text/ mime type for public keys 2011-06-03 18:41:26 -07:00
Brad Fitzpatrick 9f2daec131 UI/search: show all claims for a permanode 2011-06-03 15:23:23 -07:00
Brad Fitzpatrick b28ae35c35 UI: deleting tags 2011-06-03 14:16:00 -07:00
Brad Fitzpatrick 2db013a3b4 Add a dev-pass-camput script, improve error messages. 2011-06-02 16:57:49 -07:00
Brad Fitzpatrick 9b3280ce10 Update to new Go exec API 2011-06-02 16:47:40 -07:00
Brad Fitzpatrick 5725dbaf75 Fix localdisk queue error on dup uploads. 2011-05-31 18:58:58 -07:00
Brad Fitzpatrick 0dfc7a5770 Fix a performance TODO in schema.FileReader 2011-05-31 10:35:45 -07:00
Brad Fitzpatrick 2da1ef4093 UI: file download support 2011-05-31 10:20:28 -07:00
Brad Fitzpatrick 2c66ab38ad whoops, was using offset wrong in filewriter 2011-05-30 19:24:19 -07:00
Brad Fitzpatrick ae221c0712 Fighting GoMySQL library; see https://groups.google.com/group/gomysql/browse_thread/thread/594ffd08c6370261 2011-05-30 18:10:13 -07:00
Brad Fitzpatrick 7de40d3b5c more mysql paranoia 2011-05-30 17:20:15 -07:00
Brad Fitzpatrick 335f3187a8 blob info page 2011-05-30 16:41:56 -07:00
Brad Fitzpatrick a3dc3f3ad7 close stmt on exit? 2011-05-30 16:39:56 -07:00
Brad Fitzpatrick 1c6470b3b3 Search: add a describe handler for javascript UI 2011-05-30 15:44:25 -07:00
Brad Fitzpatrick 4e08e4805f Move some key stuff from camlistored to jsonsign, use from camput. 2011-05-30 12:38:26 -07:00
Brad Fitzpatrick 10c3ec682c convert search to use handler registry 2011-05-29 22:52:31 -07:00
Brad Fitzpatrick c4fa4d25b4 Protocol change: remove some upload restrictions that were only for App Engine. 
Now effective SHOULDs.  Not changing the spec yet, though.  Need to fix App Engine python impl.
 2011-05-29 21:39:51 -07:00
Brad Fitzpatrick 884e8a63ea fix WriteFileFromReader bug when chunks already exist. 2011-05-29 18:22:56 -07:00
Brad Fitzpatrick f9b879d3bf Add schema.WriteFileFromReader and use it in UI upload helper 2011-05-29 10:50:17 -07:00
Brad Fitzpatrick 90405dc424 schema: PopulateRegularFileMap take just size, not FileInfo, add NewCommonFilenameMap 2011-05-29 10:39:41 -07:00
Brad Fitzpatrick 5ea8b64217 client: Use blobref.Sha1FromString 2011-05-29 10:38:53 -07:00
Brad Fitzpatrick f4512d3c6c blobref: add Sha1FromString helper and SizedBlobRef Equal/String 2011-05-29 10:38:21 -07:00
Brad Fitzpatrick eb0bf1e3d1 schema: move FileReader to its own file 2011-05-26 16:46:27 -07:00
Brad Fitzpatrick 50abe93da6 gofmt on camput and little style things 2011-05-26 16:41:49 -07:00
Brad Fitzpatrick 4317da0308 camlistored: convert ui, root, jsonsign to use handler registry 2011-05-26 07:34:39 -07:00
Brad Fitzpatrick 9ad7346474 blobserver registry: finish RegisterHandlerConstructor stuff 
going to use it now.
 2011-05-25 22:15:55 -07:00
Brad Fitzpatrick 513b594dbc replica: notify blob hub 2011-05-22 21:23:23 -07:00
Brad Fitzpatrick ccf33926c8 Replica blob handler for live replication. 2011-05-22 21:22:21 -07:00
Brad Fitzpatrick 11c959caab jsonconfig: clean up duplication, add Optiona/RequiredInt 2011-05-22 21:20:54 -07:00
Brad Fitzpatrick 9e7233bd4c shard: notify hub when file received. 2011-05-22 16:02:44 -07:00
Brad Fitzpatrick 7fb3c5a5e1 Implement MergedEnumerate; finishes shard handler. 2011-05-22 12:39:54 -07:00
Brad Fitzpatrick 3765151953 Move ChanPeeker into blobref pkg. Need it elsewhere. 2011-05-22 12:38:50 -07:00
Brad Fitzpatrick 21797e2150 Make shard Enumerate use new blobserver.MergedEnumerated; not yet implemented. 
Will need for replica target too.
 2011-05-21 18:19:37 -07:00
Brad Fitzpatrick 941ac98281 Shard handler implementation, except for enumerate. 2011-05-21 14:32:25 -07:00
Brad Fitzpatrick 5b1657a2d4 Make blob storage constructors take a blobserver.Loader 2011-05-21 13:40:17 -07:00
Brad Fitzpatrick 621d3e2cc9 Start of 'shard' handler. 2011-05-21 09:26:20 -07:00
Brad Fitzpatrick 0e9921e6f3 Update to get dev-server working again after jsonsign & Go changes. 2011-05-21 08:51:49 -07:00
Brad Fitzpatrick 415f2b903a pinentry: default to reading from stdin if pinentry not found 2011-05-17 20:48:57 -07:00
Brad Fitzpatrick 19b822efdd More gpgagent, pinentry, and jsonsign work. 2011-05-17 20:25:00 -07:00
Brad Fitzpatrick e9918ac445 Flesh out gpgagent library. 2011-05-17 19:32:01 -07:00
Brad Fitzpatrick 7af8ab94e6 use cachekey in code + test 2011-05-16 22:06:03 -07:00
Brad Fitzpatrick 2ac8794abe Start of gpgagent client too, similar to pinentry. 2011-05-16 21:52:01 -07:00
Brad Fitzpatrick 27ddde1db4 Add caching of decrypted entity private keys. 2011-05-16 14:35:38 -07:00
Brad Fitzpatrick ca19b8ddaa jsonsign: use Go's openpgp library; don't shell out to gpg 2011-05-16 09:01:35 -07:00
Brad Fitzpatrick bbf3408920 client config: use JSON errorutil.HighlightBytePosition helper 2011-05-15 21:56:17 -07:00
Brad Fitzpatrick 5cf560c600 osutil paths: let CAMLI_CONFIG_DIR environment override the default config directory 2011-05-15 21:55:46 -07:00
Brad Fitzpatrick e2c955bddb blobref fetcher: use osutil.CamliConfigDir, don't duplicate 2011-05-15 21:55:19 -07:00
Brad Fitzpatrick 2bc5d53aea pinentry library to be used with jsonsign shortly here. 2011-05-14 18:43:07 -07:00
Brad Fitzpatrick 95ea9ac630 Use new Go http.Request.SetBasicAuth 2011-05-14 17:14:54 -07:00
Brad Fitzpatrick 6faab75120 gofix: open for fuse 2011-05-14 16:40:39 -07:00
Brad Fitzpatrick 35e61582db gofix: get finalurl 2011-05-14 16:40:31 -07:00
Brad Fitzpatrick c5e93ceecd Rename CorruptBlobError -> ErrCorruptBlob, per Go convention 2011-05-12 19:19:49 -07:00
Brad Fitzpatrick 860f020ec4 This bug is already filed. 2011-05-12 19:18:20 -07:00
Brad Fitzpatrick 475561d74c Remove old partition interface. 2011-05-12 19:17:59 -07:00
Brad Fitzpatrick f3c7d79a2d Update NoImplStorage to latest API; add type check 2011-05-12 19:16:08 -07:00
Brad Fitzpatrick 8bfcb6a5a3 Move CountingReader to new camli/misc library 2011-05-11 08:19:23 -07:00
Brad Fitzpatrick 54057fb914 fix some logging printfs 2011-05-11 08:07:02 -07:00
Brad Fitzpatrick 76dd41e18a localdisk: use defer to reliably close the channel. missed a case on enumerate timeout before. 2011-05-11 06:55:14 -07:00
Brad Fitzpatrick 7e3c711395 localdisk: Create queue should create directory too 2011-05-11 06:30:11 -07:00
Brad Fitzpatrick 68898203e5 Add TODO to rename StreamingFetcher to be Fetcher 2011-05-11 06:29:40 -07:00
Brad Fitzpatrick 9a6b0799d2 use filepath.Join, not fmt.Sprintf 2011-05-11 06:20:35 -07:00
Brad Fitzpatrick d4f63d89d3 Clarify enumerate docs: after and maxWaitSec are mutually exclusive 2011-05-11 06:11:44 -07:00
Brad Fitzpatrick 480350936e Revert "Work around Go http bug with zero length files." 
This reverts commit 1e7ff7b46c.

Fixed with: http://codereview.appspot.com/4529050
 2011-05-11 04:59:45 -07:00
Brad Fitzpatrick 1e7ff7b46c Work around Go http bug with zero length files. 2011-05-10 19:29:20 -07:00
Brad Fitzpatrick 257e550740 use blobref.SizedBlobRef as a value, not a mutable *blobref.SizedBlobRef on heap 2011-05-10 16:13:37 -07:00
Brad Fitzpatrick a0e38478d3 blobserver/remote: ReceiveBlob, which is now easy after previous commits. 2011-05-10 15:51:10 -07:00
Brad Fitzpatrick cb0498a7a9 Work around gotest/6g bug. 2011-05-10 15:22:46 -07:00
Brad Fitzpatrick d43263035a client: allow UploadHandle.Size to be -1, and associated cleanup, including: 
-- new StorageConfiger interface & use it in camlistored.
-- bunch of little bug fixes and TODOs done.
 2011-05-10 14:55:12 -07:00
Brad Fitzpatrick bce74ef24c add TODO 2011-05-10 13:04:20 -07:00
Brad Fitzpatrick b99db2dd17 implement Client.Stat 2011-05-10 13:02:48 -07:00
Brad Fitzpatrick 70892e9f23 client: parseStatResponse 2011-05-10 12:56:59 -07:00
Brad Fitzpatrick 8e7c92be02 gofmt only 2011-05-10 12:56:40 -07:00
Brad Fitzpatrick 67b1d6cb43 More remote blobserver work 2011-05-10 11:35:48 -07:00
Brad Fitzpatrick 651fac6a3d put directional constraints on all the channel types 2011-05-10 11:21:28 -07:00
Brad Fitzpatrick c4eb6bb68d Some work on remote blobserver & camli client; allow client to use alt http clients 2011-05-10 06:25:18 -07:00
Brad Fitzpatrick a31431a11b use new Go API http.NewRequest 2011-05-10 05:59:36 -07:00
Brad Fitzpatrick a010e605fa flag cleanup: remove some GPG flags 2011-05-10 05:00:53 -07:00
Brad Fitzpatrick ab2a1e1fb4 jsonconfig: add optional object 2011-05-09 14:17:30 -07:00
Brad Fitzpatrick a5bdaef70d better client error message 2011-05-09 14:16:59 -07:00
Brad Fitzpatrick c619cdcdf1 Allow dev-* scripts listening on port 0, and then log resolved port. 2011-05-09 12:22:31 -07:00
Brad Fitzpatrick ec0c73f08d allow default value in jsonconfig _env expansion 2011-05-09 12:12:54 -07:00
Brad Fitzpatrick 17a804b7c1 Move TLS setup to config file, not flags. Use jsonconfig for root config. 2011-05-09 11:49:02 -07:00
Brad Fitzpatrick 0d3f7305cb Remove --showimages demo flag; make it an env variable. 2011-05-09 11:16:48 -07:00
Brad Fitzpatrick 1b7aceac3f Automatic blobserver queue creation from sync Handler. 2011-05-09 11:08:14 -07:00
Brad Fitzpatrick bed26de507 Cleanup: remove partitions from interfaces. WIP but compiles. 
Still need to fix up blobhub notification for localdisk
mirroring, since now localdisk can't find the notification
hub for mirrored partitions.  (and also can't be configured
yet)
 2011-05-09 09:11:18 -07:00
Brad Fitzpatrick 77cfa748f5 Start of remote (proxy) blobserver type 2011-05-09 06:32:24 -07:00
Brad Fitzpatrick 771393f89e update for multipart return EOF change 2011-05-07 05:37:54 -07:00
Brad Fitzpatrick b68e0184b1 in verify reply, include SignerKeyId 2011-05-06 05:37:17 -07:00
Brad Fitzpatrick 1f6bee57c7 wire up verification handler to web UI 2011-05-05 21:45:24 -07:00
Brad Fitzpatrick 6d0639054d woot, signing handler hooked up to web UI now. 2011-05-05 21:21:30 -07:00
Brad Fitzpatrick 1a14a121d0 improve jsonsign discovery handler 2011-05-05 17:15:32 -07:00
Brad Fitzpatrick 962d65877d some sig discovery work 2011-05-05 16:41:00 -07:00
Brad Fitzpatrick 73a7329401 blobref memory store 2011-05-05 16:31:19 -07:00
Brad Fitzpatrick 54efd345d8 start of blobref.MemoryStore 2011-05-03 17:55:57 -07:00
Brad Fitzpatrick 1ba5fd168a allow fetcher or streaming fetcher in jsonsign signing 2011-05-03 17:34:16 -07:00
Brad Fitzpatrick 8658894096 add keyring with a passphrase for testing 2011-05-03 07:26:57 -07:00
Brad Fitzpatrick 36989b6dcf Add dev-server helper script to run a full dev config. 2011-05-01 18:36:11 -07:00
Brad Fitzpatrick 12b8190828 jsonconfig expansion support; start making dev-* helper scripts use config files 2011-05-01 16:10:53 -07:00
Brad Fitzpatrick d02b3e8d1b reflect NewValue -> ValueOf (latest Go changes) 2011-04-25 18:19:48 -07:00
Brad Fitzpatrick 3db60c316c camlistored: configuration + cleanup work 
working on gutting partition stuff from everywhere,
and gutting the command-line mode that statically
mounted various dev handlers based on flags.

more stuff moves into the config file now
and jsonconfig is used in more places.
 2011-04-15 22:25:45 -07:00
Brad Fitzpatrick 2bcb748cf0 Make jsonconfig into its own library 
Will move away from camli/blobserver in future commit.
 2011-04-15 22:22:30 -07:00
Brad Fitzpatrick a76551fe4f Highlight position of JSON parser errors. 2011-04-15 17:17:33 -07:00
Brad Fitzpatrick da0cb14ab8 Amazon: untested delete support 2011-04-08 23:22:52 -07:00
Brad Fitzpatrick df73e53494 Amazon fetch support; misc refactorings 
Blobservers only need a streaming fetcher now.
 2011-04-08 23:20:24 -07:00
Brad Fitzpatrick f5174a008b Amazon S3 enumerate support. 2011-04-08 21:05:39 -07:00
Brad Fitzpatrick fd22dafa7b Let storage layers declare their max enumerate size 2011-04-08 21:03:43 -07:00
Brad Fitzpatrick 0de477e15f remove unnecessary channel closes, one of which was buggy 2011-04-07 19:26:05 -07:00
Brad Fitzpatrick c56f82c5dd Updates for Go Open changes. 2011-04-07 10:58:29 -07:00
Brad Fitzpatrick 036e4756cb S3 storage works now, with last night's weekly release of Go. 2011-04-05 09:19:30 -07:00
Brad Fitzpatrick 46542e2e3f Amazon S3 stat/upload support. Basically works. 
Might be a bug or two yet, and requires a yet-unsubmitted fix to
Go's HTTP library.
 2011-04-04 15:15:09 -07:00
Brad Fitzpatrick 17dc73b291 s3 signing: use req.URL.Host if req.Host isn't set 2011-04-04 15:10:50 -07:00
Brad Fitzpatrick f9d167d9cc Start of s3 stat/receive. 2011-04-03 19:59:46 -07:00
Brad Fitzpatrick 87d749f781 use fmt.Errorf instead of os.NewError(fmt.Sprintf(..)) 2011-04-03 19:58:41 -07:00
Brad Fitzpatrick 978a46fd4a add skipStartupCheck option to s3 2011-04-03 19:57:30 -07:00
Brad Fitzpatrick ed9caf9116 boolean config accessors 2011-04-03 19:57:12 -07:00
Brad Fitzpatrick 133c9dea1d Complain about unknown keys in config files. 2011-04-03 19:16:11 -07:00
Brad Fitzpatrick b5528c4a6b s3: add bucket to config 2011-04-03 19:15:09 -07:00
Brad Fitzpatrick 59901467fd Server config file support. 2011-04-03 08:07:40 -07:00
Brad Fitzpatrick 2a61dcfd9e Put some path-related stuff in camli/osutil 2011-04-01 22:26:33 -07:00
Brad Fitzpatrick 173f5615a2 register as s3, not filesystem 2011-04-01 22:13:55 -07:00
Brad Fitzpatrick 68b044592c more s3 client work 2011-04-01 21:49:12 -07:00
Brad Fitzpatrick 2c3ea22973 S3 http request signing 2011-04-01 21:26:10 -07:00
Brad Fitzpatrick 63668853d9 Make s3 storage use registry; start trimming blobserver 2011-04-01 20:45:40 -07:00
Brad Fitzpatrick 00f9ede8a6 Start of blobserver handler registry for config file work. 2011-04-01 18:20:28 -07:00
Brad Fitzpatrick 5634523637 Add camli/netutil library for ident-like lookups. 2011-04-01 12:47:07 -07:00
Brad Fitzpatrick 5d3557772a Fix MySQL mysql.go for net.Dial laddr removal. 2011-03-29 16:05:19 -07:00
Brad Fitzpatrick 8006b0538c Add NanosFromRFC3339 to schema; + tests 2011-03-29 10:29:16 -07:00
Brad Fitzpatrick 84c891a8e5 Start of s3 blobserver support + prep cleanup 2011-03-27 22:06:29 -07:00
Brad Fitzpatrick cb742f43f9 Start of Amazon S3 library. For now signing + tests. 2011-03-27 15:25:21 -07:00
Brad Fitzpatrick d9e09f7513 Export Rfc3339FromNanos, more tests, start of doing reverse for fuse. 2011-03-25 21:43:49 -07:00
Brad Fitzpatrick 3b94bd0578 camli/lru and a few tests 2011-03-25 20:58:17 -07:00
Brad Fitzpatrick 7c1e08c55f Start of simple camli/lru class for fuse caches. 2011-03-25 18:58:14 -07:00
Brad Fitzpatrick cac9d9ff86 schema: camli non-utf8 JSON string decoding 2011-03-25 18:45:10 -07:00
Brad Fitzpatrick 912a5a304c fuse readlink support 2011-03-25 18:36:08 -07:00
Brad Fitzpatrick 28975e8fc3 Change EnumerateBlobs interface to close channel, not send nil. 2011-03-25 09:52:51 -07:00
Brad Fitzpatrick 83a814148b Define blobserver.Cache interface. 2011-03-24 19:57:57 -07:00
Brad Fitzpatrick a774c3dc19 camli/client: implement FetchStreaming, not Fetch 2011-03-24 19:22:35 -07:00
Brad Fitzpatrick 468a4849c4 add StreamingFetcher interface 2011-03-24 19:20:47 -07:00
Brad Fitzpatrick ff25acb45c FileReader tests & bug fixes. 2011-03-24 17:42:34 -07:00
Brad Fitzpatrick db5e3fe358 move json blob TestFetcher to camli/test package 2011-03-24 16:47:51 -07:00
Brad Fitzpatrick 3db542645c schema/fuse file reading, but still buggy in parts 2011-03-24 15:33:15 -07:00
Brad Fitzpatrick a8739674e1 cammount: Start of Open and file ops 2011-03-23 22:04:50 -07:00
Brad Fitzpatrick b366f7c699 cammount: more working Opendir 2011-03-23 19:57:40 -07:00
Brad Fitzpatrick 6ce55da2f4 Update to go-fuse 142724315 2011-03-23 19:14:04 -07:00
Brad Fitzpatrick 95b3214789 Merge branch 'master' of camlistore.org:camlistore 2011-03-22 22:59:48 -07:00
Brad Fitzpatrick 367d4d7bef cammount: more FUSE work; kinda-working Opendir 2011-03-22 22:59:45 -07:00
Brett Slatkin f6a9c55cd8 Merge branch 'master' of camlistore.org:camlistore 2011-03-22 22:50:35 -07:00
Brett Slatkin f674c9a644 added python library for schema blobs 2011-03-22 22:50:14 -07:00
Brad Fitzpatrick 9771c314e3 cammount: more FUSE work: basic GetAttr support 2011-03-22 22:32:11 -07:00
Brad Fitzpatrick 0707a092e0 cammount: more FUSE work, inode lookup 2011-03-22 22:11:27 -07:00
Brett Slatkin 87be928fa9 adds fusepy library 2011-03-22 17:27:27 -07:00
Brad Fitzpatrick 43d058ae77 Merge commit '7262d8a' 2011-03-20 10:25:23 -07:00
Brad Fitzpatrick 042619c4b6 Use GoMySQL in third_party instead. 2011-03-19 21:25:24 -07:00
Brad Fitzpatrick cc516df2bc third_party: add https://github.com/Philio/GoMySQL 
As of commit:

commit 9840aab51952df13259ee88f9d00dcc61eb31fd7
Author: Philio <phil@bayfmail.com>
Date:   Tue Mar 1 19:00:26 2011 +0000

    update to 0.3.0 release
 2011-03-19 21:22:59 -07:00
Brad Fitzpatrick c4a269cc69 Ignore third_party/workdir 2011-03-19 21:21:58 -07:00
Brad Fitzpatrick b94cf58ee7 Update go-fuse mount.go to work with Go weekly API changes. 2011-03-19 20:14:02 -07:00
Brad Fitzpatrick 772ed95a3a third_party: import https://github.com/hanwen/go-fuse/ at 79426f97e60 
commit 79426f97e60ee827aa5745e0f17e8a293e51c0c4
Author: Han-Wen Nienhuys <hanwen@google.com>
Date:   Thu Mar 17 22:21:31 2011 -0300

    Stop serializing writes - the kernel is multithreaded too.
 2011-03-19 20:07:05 -07:00
Brad Fitzpatrick d4ce983b1d Start of camli third_party lib directory & maintenance script. 2011-03-19 20:05:41 -07:00
Brad Fitzpatrick 672eb03400 Add blob fetch handler that prematurely closes for Dan. 
He's fighting with Android HTTP libraries to handle this properly.
 2011-03-19 14:16:08 -07:00
Brad Fitzpatrick a88dd30073 Start of magic sniffing tests. 2011-03-19 01:11:15 -07:00
Brad Fitzpatrick 14530584cb camli/test/asserts: missing format param 2011-03-19 01:10:56 -07:00
Brad Fitzpatrick 3bc3aa1390 Move magic sniffing to its own library, out of MySQL stuff. 2011-03-19 00:48:33 -07:00
Brad Fitzpatrick 184afbc2a3 Auto-generate Go Makefiles. 
Dependencies are now only in one place (in source code, enforced for correctness by Go)
instead of three places.
 2011-03-19 00:27:03 -07:00
Brad Fitzpatrick ae608c6a8c Automatic go dependency inference; phase 1. 
Next phaes will auto-gen makefiles.
 2011-03-18 22:33:43 -07:00
Brad Fitzpatrick 010da9d9a7 Move some libraries from server/go to lib/go. 
These were just in server/ before the directory structure was figured out.
 2011-03-18 21:30:29 -07:00
Brad Fitzpatrick 2e904bd0b0 Move lib/go/ stuff around into a camli/ or camli/misc/ directory. 2011-03-18 21:24:10 -07:00
Brad Fitzpatrick 386ad357b7 camli/testing path cleanup. 
Break lib/go/testing into lib/go/camli/test and lib/go/camli/test/asserts

Moves TestBlob stuff and package 'testing' flags out of everybody's
command flags.

Also moves towards making the project filesystem match the import
paths.
 2011-03-18 21:17:26 -07:00
Brad Fitzpatrick ce0d30eb1a Remove hard-coding of my public key; use config file. 2011-03-15 23:16:24 -07:00
Brad Fitzpatrick ae15620ea2 jsonsign: more tests 2011-03-15 18:01:04 -07:00
Brad Fitzpatrick 33e2d32f99 jsonsign: fix bugs from move to upstream go openpgp; more tests 2011-03-15 17:37:52 -07:00
Brad Fitzpatrick b00566cc6f fix enumerate_test.go; put Recorder (httptest.Recorder) in upstream Go 2011-03-15 10:26:37 -07:00
Brad Fitzpatrick 4cc2669ae2 jsonsign: more work on tests 2011-03-15 09:55:10 -07:00
Brad Fitzpatrick 866d020df1 use OpenPGP library now upstream in Go; start of jsonsign tests. 2011-03-14 22:51:36 -07:00
Brad Fitzpatrick f533c80b52 camput --add-attr and --set-attr support 2011-03-13 22:21:58 -07:00
Brad Fitzpatrick f8fca14d89 Remove fake attributes from testing. 2011-03-13 21:34:15 -07:00
Brad Fitzpatrick a33fc8f959 Search: return real attrs/content/type, not faked. 2011-03-13 20:51:58 -07:00
Brad Fitzpatrick b86c8e409d Search work. Some stuff hard-coded for Dan's Android client work. 2011-03-13 19:27:59 -07:00
Brad Fitzpatrick b2669bbbc4 Make homescreen 'empty' search work. 2011-03-13 17:14:48 -07:00
Brad Fitzpatrick 8946a88bdc Start of wiring up search to mysql to http. 2011-03-13 16:38:32 -07:00
Brad Fitzpatrick 195987bb1b Add new camli/search package and start of interface. 2011-03-13 16:10:15 -07:00
Brad Fitzpatrick f919985072 mysqlindexer: permanodes table 2011-03-13 15:33:14 -07:00
Brad Fitzpatrick bd9f5f9331 schema: include claimDate in claims 2011-03-13 12:30:13 -07:00
Brad Fitzpatrick 6dbafbe0b4 mysqlindexer: claims table 2011-03-13 12:18:00 -07:00
Brad Fitzpatrick 61e5ffa5ee gofmt schema 2011-03-13 11:39:08 -07:00
Brad Fitzpatrick b7edc4b23b Add schema.Superset; use from mysqlindexer 2011-03-13 11:38:53 -07:00
Brad Fitzpatrick 8f4116492e more mime magic for fun 2011-03-12 21:24:28 -08:00
Brad Fitzpatrick ca92bc3903 localdisk: fix enumeration bug, walking into partitions. 2011-03-12 21:11:03 -08:00
Brad Fitzpatrick 12e3730294 increase the enumeration batch size 2011-03-12 20:44:01 -08:00
Brad Fitzpatrick a87ff50a5b mysqlindexer: sniff and write mime/camli type to blobs table 2011-03-12 20:43:18 -08:00
Brad Fitzpatrick bf8f379771 camput: create permanode and claims when uploading a directory 2011-03-12 19:28:18 -08:00
Brad Fitzpatrick f14c6dd55b localdisk: return ENOENT as documented by interface; add test. 2011-03-12 18:28:05 -08:00
Brad Fitzpatrick f70cc6d2e5 Updates to latest Go language + library changes. 2011-03-12 12:32:40 -08:00
Brad Fitzpatrick 3edbfd3f26 Updates for Go release 2011-03-07 2011-03-07 08:37:27 -08:00
Brad Fitzpatrick 77e4ebe48f Epic refactoring around partition handling; MySQL indexer starting to work 
The MySQL indexer storage implementation now does Receive, Enumerate,
and Stat.

So I can now just camsync from a localdisk blobserver to a
mysqlindexer blobserver and push-index stuff.
 2011-03-06 20:11:36 -08:00
Brad Fitzpatrick 0a7a3bda34 Start of some mysqlindexer receive work. 2011-03-06 09:28:02 -08:00
Brad Fitzpatrick bb6a850e99 Untested mysql indexer stat support. 2011-03-06 00:16:32 -08:00
Brett Slatkin 0371e8f66f gofmt 2011-03-05 16:23:47 -08:00
Brett Slatkin 8af57171f3 enumerate-blobs working for mysql indexer 2011-03-05 16:23:12 -08:00
Brett Slatkin dfe4b27fa2 gofmt 2011-03-05 15:32:08 -08:00
Brett Slatkin e36a7679b2 mysql client pool work for indexer; temporary glue in blobserver 2011-03-05 15:10:50 -08:00
Brad Fitzpatrick e3659d3a62 Start of mysqlindexer storage library. 2011-03-05 14:14:00 -08:00
Brett Slatkin 02a3466192 adds -remove to camput 2011-03-05 13:43:24 -08:00
Brett Slatkin 16c672506b implemented go client.RemoveBlobs; no idea if it works yet 2011-03-05 12:46:28 -08:00
Brad Fitzpatrick 4de2eb4ebd Start of Client.Remove for de-queue in camsync. 2011-03-05 11:34:12 -08:00
Brad Fitzpatrick cafaec5dc8 more sync work; start of loop and queue removal 2011-03-05 09:53:51 -08:00
Brad Fitzpatrick bc7340d7e7 sync: move to library, add tests, fix merge bug 2011-03-05 09:00:56 -08:00
Brad Fitzpatrick 113de1f3a3 get handler cleanup, break apart into smaller bits 2011-03-05 00:03:53 -08:00
Brad Fitzpatrick 4d8f10208a camsync: it now syncs 2011-03-04 19:02:26 -08:00
Brad Fitzpatrick 38a85571ca Add dev-blobserver wrapper; remove run.sh 2011-03-02 21:46:07 -08:00
Brad Fitzpatrick 7381cbf4d1 More camsync work. 2011-03-02 20:03:09 -08:00
Brad Fitzpatrick d9bf9a3e14 blobref: faster String(); use camli/testing 2011-03-02 10:18:45 -08:00
Brad Fitzpatrick ad37f29ce3 start of camsync & indexing; add Client.EnumerateBlobs 2011-03-01 18:02:01 -08:00
Brad Fitzpatrick 0584e4a358 Unfork the Go http package (it now has http.Client.Do) 2011-02-28 19:20:26 -08:00
Brad Fitzpatrick 66abaaec73 enumerate: don't allow 'maxwaitsec' with 'after' 2011-02-28 18:11:53 -08:00
Brad Fitzpatrick b3f85acd47 enumerate: tests & long poll support 2011-02-27 17:42:20 -08:00
Brad Fitzpatrick 2e2fea784b Split localdisk.go up into separate files. 2011-02-27 16:18:17 -08:00
Brad Fitzpatrick 5329e66eb6 some enumerate waitsec work 2011-02-26 23:09:57 -08:00
Brad Fitzpatrick f34b4b35d2 Spec and start of longpoll enumerate. 2011-02-26 14:03:10 -08:00
Brad Fitzpatrick 88c1992692 Take a small interface. 2011-02-26 13:44:25 -08:00
Brad Fitzpatrick e1c7b60fa1 localdisk: parallel stats and long-poll stats. 2011-02-26 13:31:23 -08:00
Brad Fitzpatrick b21a036964 Make camli/blobserver's blobhub_test use camli/testing. 2011-02-26 13:30:34 -08:00
Brad Fitzpatrick 01b9a71d71 Fix camli/schema's tests to pass again. 2011-02-26 13:30:07 -08:00
Brad Fitzpatrick fd0dc7d94c Add camli/testing helper package. 2011-02-26 13:29:39 -08:00
Brad Fitzpatrick ac7b64eb56 Fix test file's package. Still no tests. 2011-02-26 13:28:22 -08:00
Brad Fitzpatrick 0e99f12815 Add missing dep. 2011-02-26 13:27:57 -08:00
Brad Fitzpatrick 51283ac159 blobhub impl and tests. 2011-02-26 12:22:26 -08:00
Brad Fitzpatrick 5fbcb110b1 Remove old Makefile 2011-02-26 10:27:44 -08:00
Brad Fitzpatrick 82e9cb70cd Some work long poll stat. 2011-02-26 09:26:23 -08:00
Brad Fitzpatrick d15216ebb0 Writing a (failing) test for Stat waiting. 2011-02-25 09:36:58 -08:00
Brad Fitzpatrick 1300b213f2 Update for Go's new http.Request.Header type. 2011-02-25 07:38:12 -08:00
Brad Fitzpatrick da9a0da296 Fix test hang. 2011-02-24 12:52:36 -08:00
Brad Fitzpatrick 6b4b726c69 Revert "Update for Go's new http.Request.Header type." 
This reverts commit b021345ed6.
 2011-02-24 10:28:00 -08:00
Brad Fitzpatrick 8613c19fcd More work on localdisk unittests. 2011-02-23 18:25:00 -08:00
Brad Fitzpatrick c9a161e31e Go has fsync now. 2011-02-23 18:14:54 -08:00
Brad Fitzpatrick b021345ed6 Update for Go's new http.Request.Header type. 2011-02-23 17:56:40 -08:00
Brad Fitzpatrick 6452c5e7da Start of localdisk tests. 2011-02-22 18:48:48 -08:00
Brad Fitzpatrick 2d036aa69a Start of per-partition blob hubs for realtime/long-poll handlers. 2011-02-09 17:05:55 -08:00
Brad Fitzpatrick 7846c3c434 Unfork encoding/line package; changes now upstream in Go. 2011-02-09 10:59:25 -08:00
Brad Fitzpatrick 246864012d Start of stat long poll spec/code. 2011-02-09 10:57:30 -08:00
Brad Fitzpatrick 73587e6085 Rename the blobserver 'preupload' method to 'stat' 
This is prep for more additions to stat.
 2011-02-08 08:24:16 -08:00
Brad Fitzpatrick 4db299e74b blobserver: library-ify preupload and upload. 2011-02-04 14:31:23 -08:00
Brad Fitzpatrick 1af31f30db blobserver: move get to handlers library + random cleanup 2011-02-03 17:28:05 -08:00
Brad Fitzpatrick 9411f9b8da blobserver: move enumerate to handlers library; make it use Storage interface 2011-02-03 17:08:04 -08:00
Brad Fitzpatrick 866ff37ce1 Add type blobref.SizedBlobRef 2011-02-03 17:06:29 -08:00
Brad Fitzpatrick 197819e4cb Ignore more 2011-02-03 17:05:19 -08:00
Brad Fitzpatrick c027ca9d9e Libraryify: create lib/go/blobserver/handles; move Remove 2011-02-03 15:56:02 -08:00
Brad Fitzpatrick 850623f932 Start ripping up blobserver into libraries, starting with blobserver.Storage 2011-02-03 15:45:35 -08:00
Brad Fitzpatrick d81d6211d5 schema: permissions on symlinks are irrelevant 2011-02-03 08:19:40 -08:00
Brad Fitzpatrick a29b948f3e Update for Go release 2011-02-01.1 2011-02-02 12:29:51 -08:00
Amir Mohammad Saied d6488c32c4 Making the codebase compatible with latest Go release (2011-02-01.1) 2011-02-02 12:27:30 -08:00
Brad Fitzpatrick f7d76303e7 Slap on copyright headers. 2011-01-27 23:07:18 -08:00
Brad Fitzpatrick 3bcba74f41 Fix copyright lines. 
Put this under Google's copyright, not our personal ones.
 2011-01-27 23:03:57 -08:00
Brad Fitzpatrick 47f40c0b98 Recomend running camput --init in error message. 2011-01-26 13:18:28 -08:00
Brett Slatkin f5758f6c68 adds "share" camli object schema; support for camput --share 2011-01-25 22:44:03 -08:00
Brett Slatkin 02717908ef adds -via to camget; adds -reqlog to blobserver 2011-01-25 21:34:17 -08:00
Brad Fitzpatrick ebcc37560c Make encoding/line an io.Reader 
Already submitted to Go at http://codereview.appspot.com/4066043/
 2011-01-25 09:20:41 -08:00
Brad Fitzpatrick 96055b12b8 Add fork of Go's encoding/line. Will modify and unfork later. 2011-01-24 16:56:21 -08:00
Brett Slatkin 5e4128c285 separates python library from example client 2011-01-21 11:29:14 -08:00
Brad Fitzpatrick 6e4f9a17d7 Actually upload permanode; clean up config, uploading, signing, logging 2011-01-18 10:29:38 -08:00