Commit Graph

2029 Commits

Author SHA1 Message Date
Brad Fitzpatrick a93cb4d9b9 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2014-10-19 16:44:19 +02:00
Brad Fitzpatrick a82f603dbc fastjpeg: add a seemingly-missing availability check
Change-Id: I5ab44c1442790e4cceb3f6671f33c57d0c6a4035
2014-10-19 16:44:14 +02:00
mpl 917a09ca6d Merge "blobserver/s3: implement SubFetcher" 2014-10-19 14:13:02 +00:00
mpl b24580d523 blobserver/s3: implement SubFetcher
Change-Id: I42ac364e268463998598990a8bf0df7437f4d05d
2014-10-19 16:11:54 +02:00
Brad Fitzpatrick 4d01c6b011 Merge "test/world: print more info when testing.Verbose()" 2014-10-19 14:11:31 +00:00
Brad Fitzpatrick 5b00309e17 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2014-10-19 13:35:48 +02:00
Brad Fitzpatrick 1f75759371 s3: style fixes
Change-Id: I0c8dacb00f57d374c3148fe5c8017a9f146d36f4
2014-10-19 13:35:42 +02:00
Brad Fitzpatrick 404a342a33 Merge "genfileembed: Add flag for adding build tags" 2014-10-18 19:09:50 +00:00
mpl 53be9298a5 blobserver/s3: add optional in-memory Fetch cache
Change-Id: I77e27555d28002ba01f1501e8a77eb4adbf701fe
2014-10-18 19:34:25 +02:00
Brad Fitzpatrick c0a8aafbe0 Merge "blobserver/s3: use flags instead of conf file in test" 2014-10-18 17:09:04 +00:00
mpl 0967669af0 blobserver/s3: use flags instead of conf file in test
Change-Id: I98c4f9ad6204ec8c534aae3abffaf3410555f86d
2014-10-18 19:03:15 +02:00
Fabian Wickborn 853db6cec8 genfileembed: Add flag for adding build tags
The build tags are written into the resulting go files literally, e.g.
a call like

    genfileembed -build-tags "linux,386 darwin,!cgo"

is written as

// +build linux,386 darwin,!cgo

to the resulting files.

Change-Id: Ic8c7e827e3040a33fe3b920c1aaa1551f902a491
2014-10-17 20:58:15 +02:00
Fabian Wickborn 119b38966a genfileembed: command line flag for pattern file
Offer a choice from which go-file the #fileembed pattern is read. The
default is still "fileembed.go".

Change-Id: Ida98850162adec33a6555b09ad320bd9ee573b8d
2014-10-17 13:11:23 +02:00
Brad Fitzpatrick 4f9c8aea72 blobserver/blobpacked, storagetest: fix context goroutine leak
Change-Id: I09abd7ef203fe267219a52a57f8627ce7e28d80d
2014-10-15 12:28:49 +02:00
Brad Fitzpatrick e0edcdeba3 blobserver/s3: remove stray file
Change-Id: I443e193989139dbb26796488c8cb369d20408508
2014-10-15 12:28:21 +02:00
Brad Fitzpatrick e65fb3a3e9 blobpacked: remove the vestigial ' l ' from the meta rows
Change-Id: I181fb3f64c16538005b6aca750f144ffe13c624b
2014-10-15 12:05:19 +02:00
Brad Fitzpatrick 86720c23dc blobpacked: don't pack if identical wholeref already exists packed
Change-Id: I95f391f7e0373e6ff29cb95343f5cdfee11d878b
2014-10-13 18:55:39 +02:00
Brad Fitzpatrick c0bf5778c9 blobpacked: add storage.Logf method; log to T in tests
Also address the unnecessary goroutine TODO.

Change-Id: I3a59f6338df4bc0a444e0281bc662676ee9dc00c
2014-10-13 12:48:38 +02:00
Brad Fitzpatrick 04726cfad1 test: add NewLogger func
Change-Id: I38cee8fd2e2670088acc1332430aca9725cef13f
2014-10-13 12:47:36 +02:00
Brad Fitzpatrick 749ba38b18 blobpacked: write one of the two wholeMetaPrefix row types, more tests
The identical file test, verifying we don't denormalize two identical files, is currently
skipped and known broken. A new flag is added to test it.

Change-Id: I0408103671a28209307aabb1aa2e40d5e49c56cc
2014-10-12 12:17:06 +02:00
Brad Fitzpatrick dc6af2ed12 blobpacked: delete from small, doc fixes, more tests
Change-Id: Iaef35b60968b2025e838fcf4347cd24b68398be6
2014-10-12 10:44:38 +02:00
Brad Fitzpatrick c9aa1137ad blobpacked: tests for files spanning multiple zip files
Change-Id: I15c0a70c4776068ee2521b3036debad005e62778
2014-10-12 10:14:28 +02:00
Brad Fitzpatrick ca785046bc Merge "serverinit: syndtr/leveldb support" 2014-10-11 14:07:01 +00:00
Brad Fitzpatrick 0f97897f81 blobserver/google/cloudstorage: make it a SubFetcher
For blobpacked.

Change-Id: Ibca3d8a6976d8e1d4058e8d7fcdca6b63f8d8b87
2014-10-11 16:02:31 +02:00
mpl 9b68fd9f32 serverinit: syndtr/leveldb support
Change-Id: Id48a00f3896c38659a53cf131493d5a19dbfe4b7
2014-10-11 15:40:29 +02:00
Brad Fitzpatrick 07ff2e8ca4 blobpacked: update RemoveBlobs TODO with a plan
Change-Id: Ic89c390c15adba96333d3d20795b8bcc204e20f8
2014-10-11 15:00:08 +02:00
Brad Fitzpatrick af4a15b3b8 blobpacked: don't declare ourselves as a BlobStreamer yet
Change-Id: Ie030e640a614a4dfa8b3afb3b30498eba2e3293d
2014-10-11 14:42:32 +02:00
Brad Fitzpatrick 0ae983b8b6 blobpacked: temporarily rename StreamBlobs away so it doesn't implement BlobStreamer
Change-Id: I7906cb60b85d553fcb6632305164acabbc37e72d
2014-10-11 14:17:47 +02:00
Brad Fitzpatrick faa5b4984b Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2014-10-11 14:15:46 +02:00
Salmān Aljammāz 0d6e0c6425 index: avoid shadowing err when retrying to index a full file
Change-Id: Ie683739039116dfb2758c6647382afebaa6e1ece
2014-10-08 18:40:00 +01:00
mpl f15c5a7cd2 index/receive: address last comments from http://camlistore.org/r/3271
Change-Id: Id41278e5e01b9ea9310b392859709a3261dc3f52
2014-10-07 17:21:14 +02:00
Salmān Aljammāz e14c122c52 indexer: images: try a FileReader if the prefix is too small for DecodeConfig
Go's image.DecodeConfig needs more than 1MiB on some images (e.g. some
Lens Blur pics taken with Google Camera). Now we first try a 512KiB header
and retry with a full FileReader if that fails.

https://camlistore.org/bugs/477

Change-Id: I286d15d86a69951737d94dd3692d4e9e1992b324
2014-10-07 12:13:33 +00:00
Brad Fitzpatrick a7c7fd7afd Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2014-10-03 08:53:00 -07:00
mpl 0aa0b18e5f fix SetShareSearch doc
Change-Id: I751b3b209f974830960480460e14c811ee9a5617
2014-09-30 00:38:57 +02:00
Brad Fitzpatrick 1434d2b257 Fix 32-bit builds.
Change-Id: I84c361dc442070802222e33df744876453cb43d3
2014-09-28 11:10:30 -07:00
Brad Fitzpatrick 85ef3b89e9 blobserver/google/cloudstorage: add optional in-memory Fetch cache
So blobpacked can get blobs quickly without HTTP GET requests when it needs
to do all the small blob fetches when writing the zip file.

Change-Id: Ibbab0483920f2ef3b70c60eeebcd0056a8e4364f
2014-09-27 16:53:34 -07:00
Brad Fitzpatrick f99a7a6fa9 blobserver/memory: add cache mode, where old entries are evicted
Change-Id: Id09f981afda8ab55971b5491ff488e696e6b5ae2
2014-09-26 16:41:31 -07:00
Brad Fitzpatrick 43bfb2b01d lru: make RemoveOldest remove the key and value that were evicted
Change-Id: I2bb13b8259617ac0ca64d785b45bbb3d5dbd0fae
2014-09-26 15:56:53 -07:00
mpl 3e8d54aaa6 Merge "serverinit: TestInstallHandlers" 2014-09-25 12:55:42 +00:00
mpl 8cf7e1399b serverinit: TestInstallHandlers
http://camlistore.org/issue/416

Change-Id: I14934dd4e0a415c359a13583ddc295be2d35c781
2014-09-25 14:54:29 +02:00
Mathieu Lonjaret 2b11cc16d4 Merge "Add sorted/leveldb." 2014-09-23 21:01:09 +00:00
Tamás Gulácsi bea59b29fa Remove kvfile-specific code from diskpacked.
In hope to use leveldb.

Change-Id: Ie9c166d48d6a2c1f52f9cfa29e703d8b8ad7f162
2014-09-23 22:40:07 +02:00
Brad Fitzpatrick fc6adc36f4 Merge "Diskpacked: revert append on index .Set error" 2014-09-23 20:25:46 +00:00
Tamás Gulácsi 550a8a51b3 Add sorted/leveldb.
Modify pkg/sorted/kvfile/kvfile.go to not have a named
return in Next - just as in pkg/sorted/leveldb/leveldb.go.

Change-Id: I0d46a86b33d2e1d2992d8a41f08979eb21aaddbb
2014-09-23 18:20:27 +02:00
Mathieu Lonjaret 5bd2dbcbd9 Merge "blobserver/mongo, missing index on "key"" 2014-09-23 14:04:37 +00:00
Brad Fitzpatrick 940c00a7a4 blobpacked: always fall through to the small storage
Change-Id: I8168bb3692d221e41f6bec053a12f7e5e5a1f667
2014-09-22 23:31:10 -04:00
Brad Fitzpatrick ca56ca7356 blobpacked: use blobserver.MergedEnumerate
Change-Id: I62160c56860a6433416373c04cedf12a5c4530bc
2014-09-22 22:08:14 -04:00
Brad Fitzpatrick 8ac4929b38 blobserver: use sub-context in MergedEnumerate and Less instead of String
Change-Id: I30c8b6a6884b95dea54e0776d312a342ffcced7c
2014-09-22 22:06:51 -04:00
Brad Fitzpatrick e996a0ac52 blobserver: used blob.Ref.Less in MergedEnumerate
Change-Id: Id5811c73a5e94eede1a2a21455e7df8be58b828b
2014-09-22 22:06:41 -04:00
Brad Fitzpatrick 9e8264bc96 blobserver: add MergedEnumerate tests
Change-Id: Ie820c6df4851afded0db8f4a58edcd64c1ce4e51
2014-09-22 22:06:30 -04:00