Rooba
/
perkeep
mirror of https://github.com/perkeep/perkeep.git
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
3,400 Commits 44 Branches 11 Tags 59 MiB
Commit Graph

3400 Commits

Author SHA1 Message Date
Bill Thiede 0e00fd6198 search & index: change 'before' logic, and tests. 
Change-Id: I62f43f26b295c46b9da63efe018d324f9105606b
 2013-11-26 21:51:06 -08:00
Brad Fitzpatrick c4b7d70ac3 corpus: start of work on AppendPermanodeAttrValues for permanode attr search 
Change-Id: I969778420474e3413813bb49b1bfd095d9a12389
 2013-11-26 20:28:14 -08:00
Brad Fitzpatrick 9a749ce73b search: efficient in-memory scans of permanode modtimes 
Change-Id: I82d34544a86cd9f3a52c2b7140dd89eac32fbfa9
 2013-11-26 19:47:00 -08:00
Brad Fitzpatrick c733835684 more TODO 
Change-Id: Ib43348a234a1902fa94d3d13b7e10aecb49da0e2
 2013-11-26 19:27:03 -08:00
Brad Fitzpatrick 1455a14e72 search: some refactoring to ease future work speeding PermanodeConstraint queries 
Change-Id: Iac98a632bd40c723481f8e635a2c34780e87a408
 2013-11-26 19:05:43 -08:00
Brad Fitzpatrick a918fac281 search: let GetRecentPermanodes take a before param. 
For infinite scroll.  Will be done better later, but this works for now.

Change-Id: I88dc7793b643226a4a5e3a583c5de5d4c7fa4819
 2013-11-25 20:35:59 -08:00
Brad Fitzpatrick 9aaadc73b8 index: put ImageInfo in memory too 
Change-Id: I8110df05dd14ec3ede2e49902b172ed9bf069bfb
 2013-11-25 19:46:28 -08:00
Brad Fitzpatrick 36019847aa Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-11-25 19:18:16 -08:00
Brad Fitzpatrick 03cc0fa8bd sync: wake up early from sleep if a blob arrives 
Change-Id: I49240d1970e537e3ace36f4cd02315ff3ed9d6b2
 2013-11-25 19:18:13 -08:00
Bill Thiede 80234700b2 Import github.com/golang/glog into third_party. 
Revision imported:

    commit 20cea4dfef0c0151bba0422a6244cc1d3e8292b1
    Author: David Symonds <dsymonds@golang.org>
    Date:   Wed Oct 9 08:22:02 2013 +1100

        Create a relative symlink for log files.

        This matches what C++ does in practice, and permits this code to work
        if -log_dir is a relative path.

Change-Id: Ifb81ae766ed5ab58fd45d3c7f485a065b01ec162
 2013-11-25 13:15:15 -08:00
Brad Fitzpatrick 20273566bf serverconfig: fix another /path/to/secring canonicalization case 
Change-Id: Ia0bdbdf0bdd0279055344d2c14a23eed473d7098
 2013-11-25 08:27:11 -08:00
Brad Fitzpatrick 326b24cc7c Merge "Add diskPack as a high level local storage destination" 2013-11-25 16:20:11 +00:00
Tamás Gulácsi 184686e849 Add diskPack as a high level local storage destination 
As a diskPack boolean flag in the high-level server config,
reusing the blobPath parameter for destination.

Change-Id: I9ec0583b160f89952cdad851e66c7fabf22b7bb4
 2013-11-25 11:23:50 +01:00
Bill Thiede 6b9f81e3f8 serverconfig: fix secring path for update_golden. 
Also regenerated the golden config files.

Change-Id: Id87dcad5f85b060eeee477e1fd3c72bff93d3d5e
 2013-11-24 20:38:00 -08:00
Brad Fitzpatrick fd122a5de6 Blind fix of bug in 3fa6d69405. 
See, tests matter.

Change-Id: I8c42be90db246e3529eac9f62711ea5510db605d
 2013-11-24 18:40:51 -08:00
Brad Fitzpatrick 786e2c7c00 Add a test TODO. 
Change-Id: I380b39fa6ce806386409fc50acaa4ec9b360b05a
 2013-11-24 18:28:59 -08:00
Brad Fitzpatrick 3fa6d69405 client: be robust against servers/proxies returning compressed content 
Change-Id: Id87c26a7d6e6d3f794a5421ece7cd344b10e1a8b
 2013-11-24 18:27:41 -08:00
Brad Fitzpatrick 2fb3d06d49 Remove dependency on the testing package from pkg/index. Polluted flags. 
Change-Id: Id5492fcab022c03e40501830d9253b5605a14ee6
 2013-11-24 17:45:45 -08:00
Brad Fitzpatrick 4c01434929 blobserver: run blobhub receive hooks in parallel. 
Change-Id: I918a124a01a2d75a64c26e136d28ed7f2547509d
 2013-11-24 17:12:56 -08:00
Brad Fitzpatrick 9261c9b303 dev-server-config: no longer need to replicate to a sync. the sync subscribes to its source. 
Change-Id: I7734f47f039cbdd40fcb672ba60d6a2488b42bdc
 2013-11-24 17:11:40 -08:00
Brad Fitzpatrick efd51a08ed devcam: print out temp dir on start-up 
Change-Id: I7f84ffdecc5cb63baa931d5b5de8aca564970a67
 2013-11-24 17:11:06 -08:00
Brad Fitzpatrick cf388c2f2a sync: have handler register receive hook with its source. 
Lets legacy configs work, even without replicating directly to it.

Change-Id: I8bdb8651040794ae346f19d6dd67a0da07505f07
 2013-11-24 16:20:11 -08:00
Brad Fitzpatrick b9fd11a8f6 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-11-24 15:13:24 -08:00
Brad Fitzpatrick eae8a74fc9 serverconfig/genconfig: add kv queues to all sync handlers. 
In the future this might be more configurable (e.g. mysql or sqlite or whatever, or different
paths).  But this works for now.

Change-Id: I85853e6bfdd9063fc2eba5922b138d5552b730a9
 2013-11-24 15:12:11 -08:00
Brad Fitzpatrick f4294cecdf Format pkg/serverconfig/testdata golden files consistently. No semantic changes. 
Change-Id: I8c190baf6514a2cf0580da45220d411b81445733
 2013-11-24 15:08:47 -08:00
Brad Fitzpatrick a7f7fb8741 serverconfig test: simplify pretty-printing, use test.Diff, add flag to update golden files 
Change-Id: I8c259dad8fc6f161f25fb4f63f97c4aef4b703b0
 2013-11-24 15:04:20 -08:00
Brad Fitzpatrick 9e3a9fde51 server/camlistored: add start-up test that tests default production config file 
Marked as broken for now. Will fix in subsequent commits.

Change-Id: If41932a72312ce8fbde1d237c3f84fa0acae83ee
 2013-11-24 15:03:34 -08:00
Brad Fitzpatrick 26f9b630f8 Add func test.BrokenTest and RUN_BROKEN_TESTS to doc/environment-vars.txt 
Change-Id: I43dc7b10ac6175ab8ab5eae4cfdd98bc08515ff5
 2013-11-24 15:00:16 -08:00
Bill Thiede 60e10a99ad blobserver/diskpacked: Remove TODO, close index. 
https://camlistore.org/gw/1f407edd588d8ae33475818d891fc0e0058d1628 makes this
possible.

Change-Id: Ibf21f7f7d3c11139192599d6e18fa0aa5dbc7007
 2013-11-24 14:28:01 -08:00
Brad Fitzpatrick 1f407edd58 sorted: add Close method to KeyValure; register 'kv' impl type for cznic/kv 
Change-Id: Ie4e21450dac9dad3433c65631706f52527502cbc
 2013-11-23 21:07:03 -08:00
Brad Fitzpatrick 2f802e7ecd Fix blobhub test, broken in earlier commit. :/ 
Change-Id: I0c6d1ab672387afcdca7af878be00745c4c29e6f
 2013-11-23 21:05:52 -08:00
Brad Fitzpatrick 3ccdb025c0 sync: add forgotten channel close. 
Change-Id: Ie6d14b0bad1229fc775dc9a0afda349ee163cbda
 2013-11-23 11:20:08 -08:00
Brad Fitzpatrick 62fbbb61a0 encrypt: use key metaIndex for index, not meta; that was used. fixes devcam server. 
Change-Id: I483c696dcc3402537bec136f8847c17767760b3b
 2013-11-23 11:11:05 -08:00
Brad Fitzpatrick 2a781196e7 Add BlobHub.AddReceiveHook; clean up docs, clarify how to use BlobReceiver. 
Change-Id: Icc5b55fcbabc852c7cacc9a806f12098d5e59377
 2013-11-23 11:09:06 -08:00
Brad Fitzpatrick 904fc3fd39 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-11-23 09:09:45 -08:00
Brad Fitzpatrick ab19715dc6 docs and TODOs 
Change-Id: I434c4d00a4dd63d338646376a563f69b122a3c53
 2013-11-23 09:09:40 -08:00
Brad Fitzpatrick bd16141c5d Merge "Strawman web ui style guide." 2013-11-23 17:09:23 +00:00
Aaron Boodman 53ae0830c7 Strawman web ui style guide. 
Change-Id: I6d6ee09bb42f79adad8789e6b9cfb5a9f59dc1f2
 2013-11-23 09:05:17 -08:00
Brad Fitzpatrick b0b054ca27 Doc tweaks 
Change-Id: I7c4cdd9b0cc37b2a40084f1cf11b81d8ce3a39fa
 2013-11-23 09:00:18 -08:00
Brad Fitzpatrick 8cd6084aef Merge "Display a folder icon and make camliPath attributes navigable." 2013-11-23 16:04:57 +00:00
Brad Fitzpatrick 90c1e48afe Rename index.Storage to sorted.KeyValue and move it into a new package. 
Having index.Index and index.Storage both in the same package led to
confusing discussions about "an index". Better names now, and smaller
packages.
 2013-11-22 23:24:54 -08:00
Brad Fitzpatrick cfd02e6c89 Fix dev server config's root storage's fetch/stat. 
Don't want to read from the indexer or a async target.

Change-Id: I72831a6ce3fba6e005c71c3e8ac73557cd268c34
 2013-11-22 22:50:40 -08:00
Aaron Boodman 650bd9b61e Display a folder icon and make camliPath attributes navigable. 
Like we do for camliMember.

Change-Id: I2db51d7397e0f8ab86ea71716fe06099ba22005a
 2013-11-22 22:49:07 -08:00
Brad Fitzpatrick 70475701d1 Get rid of QueueCreator and all its associated complexity. 
Previous TODO entry was:

-- Get rid of QueueCreator entirely. Plan:
     -- sync handler still has a source and dest (one pair) but
        instead of calling CreateQueue on the source, it instead
        has an index.Storage (configured via a RequiredObject
        so it can be a kvfile, leveldb, mysql, postgres etc)
     -- make all the index.Storage types be instantiable
        from a jsonconfig Object, perhaps with constructors keyed
        on a "type" field.
     -- make sync handler support blobserver.Receiver (or StatReceiver)
        like indexes, so it can receive blobs.  but all it needs to
        do to acknowledge the ReceiveBlob is write and flush to its
        index.Storage. the syncing is async by default. (otherwise callers
        could just use "replica" if they wanted sync replication).
        But maybe for ease of configuration switching, we could also
        support a sync mode.  when it needs to replicate a blob,
        it uses the source.
     -- future option: sync mirror to an alternate path on ReceiveBlob
        that can delete. e.g. you're uploading to s3 and google,
        but don't want to upload to both at once, so you use the localdisk
        as a buffer to spread out your upstream bandwidth.
     -- end result: no more hardlinks or queue creator.

Change-Id: I6244fc4f3a655f08470ae3160502659399f468ed
 2013-11-22 14:33:31 -08:00
Brad Fitzpatrick f94dbb35fa serverconfig: show the prefixes in config cycles. 
Change-Id: I3ab42efa7e4d80f15ee338411c010beb11518c86
 2013-11-22 13:59:16 -08:00
Brad Fitzpatrick a57de74641 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2013-11-22 12:57:55 -08:00
Brad Fitzpatrick e25db602ba Register the 'memory' index type. 
Change-Id: If697de1c3c63b2a1990353fc3ba1713f89e27d50
 2013-11-22 12:57:45 -08:00
Brad Fitzpatrick e190090cbd add TODO 
Change-Id: I3e4c33d1bafcdf4f6afde3440995c591c9457787
 2013-11-22 12:40:33 -08:00
Aaron Boodman 9855ab281d Flickr importer: import sets and populate with links to photos. 
Change-Id: I6c53dade3e96373fb3fb9ea4b172c53d5e22d988
 2013-11-22 12:21:03 -08:00
Aaron Boodman 08ea4ed13e Flickr importer: Don't refetch or import unmodified photos. 
Change-Id: I2ade0aa70e3caca8c3c1153ffee448d09d819da6
 2013-11-22 10:13:21 -08:00