Commit Graph

16 Commits

Author SHA1 Message Date
Brad Fitzpatrick 8229c19850 search, index: add WholeRef to pkg camtypes' FileInfo struct
So when you describe a file, you also gets its wholeref.

TODO: we'll need to migrate old indexes to this new format on
start-up.

Change-Id: I4a3fb000d68bde46474275c2070ef285a6d6ecfc
2015-02-04 21:04:39 -08:00
Antonin Amand 31ebc20ba7 Add IsVideo to types/camtypes FileInfo
It is based on MIME type then falls back to extension.

Change-Id: I71d80ec7cdcf94f4fa7feffa8186943abe0279ce
2014-12-02 21:57:10 +01:00
Brad Fitzpatrick f4740e77e8 Aggregate all errors with their URLS in status.json in one place.
Change-Id: If7a540fc3d1a6caa94b5f04d44de3207f15ac4cf
2014-08-29 16:03:44 -07:00
Brad Fitzpatrick a1a7e65666 camtypes: fix build. Don't call osutil path funcs during testing (in init)
Change-Id: I2f8d2966c95e29a3d6d0a0d9b117a0f022e647e7
2014-08-23 20:28:38 -07:00
mpl e8522cca36 website: allow HTML in camli errors. use website for client error.
http://camlistore.org/issue/276

Change-Id: I16f139fe2b6f1200250db668fa415ae42ff2c171
2014-08-21 23:13:56 +02:00
mpl a34f9e2669 publisher app
http://camlistore.org/issue/365

Change-Id: I281fdcbbe6a2bdf15607e75a21bc93b453f82c85
2014-06-15 00:03:03 +02:00
mpl 1e8d46ee45 pkg/client: some cleanup, doc on the website
Changed some keys to be consistent with the server config, and removed
some others.

We're now using jsonconfig to get the keys and validate the config.

Removed some unused funcs/methods, moved some global vars as members of a client.

Also, having two possible defaults for the secret keyring
(.config/camlistore/identity-secring.gpg and .gnupg/secring.gpg)
was annoying and confusing imho, so I removed the latter.

http://camlistore.org/issue/259

Change-Id: Icf79fa8822b62b5d91843698861dd19cb553607a
2013-12-23 20:23:21 +01:00
Brad Fitzpatrick 328706b8c6 index: bunch of TODOs and planning for edge tracking
Also, change to storing a pointer to claims so those pointers can be re-used elsewhere.

Change-Id: Ib0979d622ba48f1e2389575d07d7a0930264ebea
2013-12-09 17:15:34 +04:00
Brad Fitzpatrick 8c46afe490 Shrink camtypes.BlobMeta a bit. Saves 4 bytes of memory per blob.
For me,

Before:
Corpus stats: 148.859 MiB mem: 799465 blobs (89.487 GiB) (282303 schema (6478 permanode, 150962 file (112224 image), ...)

After:
Corpus stats: 146.559 MiB mem: 799465 blobs (89.487 GiB) (282303 schema (6478 permanode, 150962 file (112224 image), ...)

Change-Id: Id421dfc9d72c2a24c228418c01f9a3f05900c941
2013-11-29 13:41:39 -08:00
Brad Fitzpatrick 4eaeee9ae3 Reduce in-memory size of image info dimensions.
For my personal server,

Before:
Corpus stats: 149.884 MiB mem: 787886 blobs (88.853 GiB) (279308 schema (6073 permanode, 150557 file (111819 image), ...)

After:
Corpus stats: 145.172 MiB mem: 787886 blobs (88.853 GiB) (279308 schema (6073 permanode, 150557 file (111819 image), ...)

Change-Id: I47484e46b1eb70c8c4b5ee356cdcb0ceecde94e7
2013-11-29 11:01:41 -08:00
mpl fa32346860 pkg/index: check for deletions in EdgesTo
Also introduced and used kvEdgeBackward

http://camlistore.org/issue/191

Change-Id: Ifdca9f2453ec0c7db55afc40cf0b5f1441b83158
2013-11-28 19:41:40 +01:00
mpl 15bb5f142e pkg/index: use IsDeleted in some of the search methods
1) Done in AppendClaims, PermanodeOfSignerAttrValue,
SearchPermanodesWithAttr, PathsOfSignerTarget,
and PathsLookup.

2) camtypes.Path.DateClaim is now a time.Time, and used
as such in Path search methods.

3) Added kvSignerAttrValue, kvPathBackward, and kvPathForward

4) More Delete tests. We probably want more cases, but that's
a start.

http://camlistore.org/issue/191

Change-Id: I1d42485f2c6d1eea024f70378daacaaad9197768
2013-11-28 16:08:23 +01:00
mpl 45b630f766 pkg/index: change deletes cache structure and meaning
1) The deletes cache now simply reflects all the keyDeleted index entries,
instead of trying to always hold the computed deletion state of the
blobs.

2) The IsDeleted and DeletedAt methods have been changed accordingly.
Moreover, while they previously were only taking into account the most
recent delete claim that affected the input blob, they now take
into account all delete claims that concern said blob.
In other words, a blob is considered "active" only if all of its past
deleters are deleted too (or it does not have any deleter).

http://camlistore.org/issue/191

Change-Id: I1b276dac0b3c6a3fb3beaf96828a45a85d27e4c6
2013-11-27 17:02:03 +01:00
Brad Fitzpatrick e8603b1293 Put claims in memory too for in-memory search. Required index schema version bump.
Change-Id: I194d65476bddea111277cd0b1472c56b5527226b
2013-11-17 16:52:51 -08:00
Brad Fitzpatrick 0b07171c0c index: change GetOwnerClaims interface for efficiency, rename to AppendClaims
Change-Id: Icec235961ed8a09256df87c5fe1455cbad81ae3f
2013-11-17 14:54:30 -08:00
Brad Fitzpatrick 705107ad80 search/index: invert depedency. search now depends on index.
creates new package types/camtypes for misc types needed by both. might eventually go away as
search matures.

Change-Id: Ib771ead7bea39936ba478b7e5d58de997060861b
2013-11-16 15:00:30 -08:00