Commit Graph

36 Commits

Author SHA1 Message Date
Brad Fitzpatrick e93e4f3822 Fix deadlock in search/index.
The describe requests were launching a storm of RLocks which weren't
safe in the presence of goroutines trying to acquire write locks.

Instead, make the corpus locking the responsibility of the caller and
add Lock/Unlock/RLock/RUnlock methods to the index and move locking up
a level.

This also adds a fair bit of context.Context plumbing which was used
in earlier debugging.

Fixes camlistore/camlistore#709

Change-Id: I8d7254d1e1da541f8c080d62f5408aac807fd3b1
2016-04-22 14:57:10 -07:00
Eric Drechsel e0a9699c63
Update bazil.org/fuse
move to vendor
fix (lots) of breakages due to API changes

bazil.org/fuse: 37bfa8be929171feec943f3496bc4befdeaf10db

Change-Id: Ic92faf7585371027f3c521f7a24ae56d70d18547
2016-04-09 12:40:02 -07:00
mpl 12eddf9c19 vendor: go4.org/strutil go4.org/syncutil
They were internal packages (under pkg), which we are now moving to
go4.org, so we in turn need to vendor them in now.

Change-Id: I92224f731404d0bd4ca1c57492bed37cb3367ed4
2015-11-21 00:21:20 +01:00
Salmān Aljammāz 64220e06ff fs: set OpenDirectIO in stats files to avoid having them cached in kernel
bazil.org/fuse switched from having OpenDirectIO on to having it
off by default. This has the side effect of allowing OS X's kernel
to cache the filesystem's files in its UBC, giving us incorrect
stats from cammount's .camli_fs_stats directory. This change sets
it explicitly to get the correct behaviour again.

Updates #556

Change-Id: Ic6400a550e4874086cf5fb725d423fcfde47fd61
2014-12-26 18:09:32 +00:00
Salmān Aljammāz 972c4152fd fs: make a copy of the value buffer when setting an xattr
When setting an xattr bazil.org/fuse gives us a byte slice with the
attribute value to set, but it then reuses its backing array and
we end up with garbage values in our xattr maps. This CL makes a
copy of the slice before it stores it.

Bonus edit: fix a debug message typo in muDir.Remove.

Fixes #556

Change-Id: I820d8cde3065f2949ff4c506ede705194a458afb
2014-12-24 00:08:50 +00:00
Piotr S. Staszewski cac9aa4483 fs: handle directory permanode titles
Set corresponding permanode title on new root,
new directory and directory rename.

Change-Id: I5432e14ef57d6c0c5555b11e895e8b8ad6d8efcf
2014-07-15 08:41:05 +02:00
Eric Drechsel 7e11b360a3 fuse: fix crash populating non-permanode camliPath
https://code.google.com/p/camlistore/issues/detail?id=440

Change-Id: If3b186d03a29ea6b3c9f0945173ede8709e318d6
2014-05-08 22:33:13 +00:00
Brad Fitzpatrick bfe7128e94 fs: avoid possible nil pointer crash
http://camlistore.org/issue/401

Change-Id: I7ae091798e2f250898f14ec325a89cbfad4ee6aa
2014-03-11 13:30:58 -07:00
Tommi Virtanen 6cda29582b third_party: Switch FUSE library to bazil.org/fuse
import bazil.org/fuse at f6649cd46fb76b1e04ba9eeffb773cb189278d31
remove code.google.com/p/rsc/fuse.

Huge thanks to Tommi Virtanen for the fuse support.

Change-Id: I44b95ac1ea344648593c7376f83ca675a56077b3
2014-01-24 13:28:08 -08:00
Dustin Sallings 333cdce979 Reuse memoized mutDirs and mutFiles.
This provides a considerably improved experience for from a single FUSE
client, but not for free.

New dirents introduced outside of FUSE will be visible on the normal
refresh interval.

Removals outside of FUSE are tricky because it's not possible to tell
the difference between a file that was locally created and hasn't made
it into the index yet from a file that was deleted remotely.

Specific dirents that change permanodes outside of FUSE are replaced.

Dirents that change properties outside of FUSE without changing
permanodes (e.g. content, etc...) are never picked up.

Change-Id: I7cc8eaea5951644809053fc97c28455e96508bd4
Fixes:  http://calistore.org/issue/328
2014-01-09 19:58:43 -08:00
Dustin Sallings c2500c5444 fs: implement {mutFile,mutDir}.Access
On at least OS X, the stat(2) call will use an existing node without
issuing Lookup.  This introduces a filesystem inconsistency where a
delete removes an entry for further requests, but FUSE's cached access
of the node is still used for at least stat.

There's a call to Access directly against the node as retrieved from a
prior Lookup.  By invalidating the nodes when they're deleted and
verifying this at Access time, we keep things more consistent.

This isn't exactly camlistore.org/issue/324 as described, but it was
causing one of the tests to fail on OS X

Change-Id: Ib90a1de79d6d3960f96bac5b4fb448c952c4ba66
2014-01-06 23:47:28 -08:00
Dustin Sallings c39fb7136d fs: logging cleanup
Change-Id: I9413ce5397453cb91c9293de4dd7dce62f79ccfe
2014-01-06 23:47:28 -08:00
Dustin Sallings c13717e788 fs: extended attribute support
This includes read-write support for extended attributes for mutable
files, and enough support on all other filesystem paths to prevent OS X
from falling back to AppleDouble files.

Change-Id: I2b275591f3bc69c25e1e9c8d59a7fa4cf0bb97ff
2014-01-02 12:02:10 -08:00
Dustin Sallings e0773c28d1 fs: don't log entire write contents
Change-Id: Ib9e89916e55020ecd498b32230ae8dc189a0116d
2013-12-30 12:54:50 -08:00
Dustin Sallings 3aa0ab0a84 fs: only return a permanode as a directory if we're sure it is
If a well-timed lookup occurs while a file is being created, after the
permanode is available, but before any attribute claims declaring file
content, the file will be assumed to be a directory.

Ideally, FUSE would only hold on to this mistake for about a minute, or
the duration of the attribute cache, but it's much longer in practice.
Even with cammount effectively disabling all caches and returning fresh
dirents with random inodes, FUSE seemingly never sent in another Lookup
request for the broken entry.

While I wouldn't consider this a fix for the condition at large, I do
believe being explicit about what's a file, symlink, or directory is the
correct thing to do.

See also: http://camlistore.org/issue/293

Change-Id: I64d4afc8dfdb87b9b121d161936dd613139f00e2
2013-12-28 13:34:53 -08:00
Dustin Sallings c6768d8875 fuse: Enable tests on Linux; make them pass
This is essentially two things:

1. Trivial changes to fs_test to make it not skip on Linux
2. Move much of the logic that was in *mutFile.Release into *mutFile.Flush

See http://sourceforge.net/apps/mediawiki/fuse/index.php?title=FUSE_tutorial
and the FUSE FAQ for an explanation of Flush vs. Release.

Change-Id: Id63312b3f8ebf12917338b836beb8a7a736d4fe6
2013-12-27 11:51:17 -08:00
Brad Fitzpatrick 55955a7734 fs: fix other case where we're comparing Flags to 0 2013-12-26 14:11:20 -08:00
Brad Fitzpatrick 9af5a970de Mark boring files generated by FUSE as "hidden", then hide in the UI.
Makes for prettier FUSE demos w/ files showing up live in the browser
with websockets.

Change-Id: I41cc9ae0d33db39b4dbbf5b60714cee9a79b7248
2013-12-13 13:27:48 +04:00
Tamás Gulácsi 161fc061d3 use syncutil.Group and syncutil.Gate for paralellizing blobserver stats/removes
- where TODO exist.

Change-Id: If7340cf3bfcd5f5211f646404312c2fd15947bd9
2013-10-10 21:52:12 +02:00
mpl 304f5c4b38 camput attr: support -del
Change-Id: I28460fed5b572ba9a83cfcb04f38e42124fa723f
2013-10-03 15:07:43 +02:00
Brad Fitzpatrick 0bdf20884b all: delete pkg/blobref; convert all from *blobref.BlobRef to new blob.Ref
Change-Id: Id2dfb7f19452bedf4f3c9310b36227fd8117b225
2013-08-03 19:54:30 -07:00
mpl e4e976e8a1 fs: typo in copyright date
Change-Id: I59cbf752971d7f1b9a8e6b291aec814d3b833378
2013-08-02 21:57:16 +02:00
Brad Fitzpatrick dbfbc0c1e5 fs: implement symlinks
Change-Id: Ia2ae00c6e4a5b84239ae4807d31141fdb4f6c220
2013-07-28 12:59:56 -07:00
Brad Fitzpatrick f709fc930c fs: implement Rename. all tests pass now.
Change-Id: I3876aeb6dafd7e4cc5254a741e6938a579373a64
2013-07-27 22:54:55 -07:00
Brad Fitzpatrick a277eb99e0 fs: add a Rename stub. Need to implement for other tests to pass.
Change-Id: I1b41512d3c85eaac5e8977b3a3c590da01af2e98
2013-07-27 18:29:50 -07:00
Brad Fitzpatrick 1e72ebb0cd fs: pass more tests.
read-only Opens need to also not set OpenDirectIO, else the property
is retained on all future operations on that handle.

Change-Id: I954e899bf2c74815813b25a1fa0dc6d42a4189c6
2013-07-27 16:07:32 -07:00
Brad Fitzpatrick 830c6966a1 fs: set inode when statting directory permanodes
Change-Id: Ie20f02b3abab48c719a0de7b0d1409c559c5e69a
2013-07-22 09:53:17 -07:00
Brad Fitzpatrick f0edcaa7d4 fs: fix bug preventing Finder from working.
Apparently we need to update the node's size after a write on a filehandle.
Finder stats it (the node, not the handle) while writing and complains if it's
not the right size.

Change-Id: Ie3e98328a182a4bbb3230c5190e1d57d0d3fb075
2013-07-22 09:52:48 -07:00
Brad Fitzpatrick ff9ab092b1 fs: add .camli_fs_stats root directory; use it in tests.
Currently one a few stats are tracked, but more can be added easily now.

The only test using this is using it to verify that the read-only open optimization
is used.

Change-Id: I1e2fb8a0222ade82631670d7c51e684778d031ae
2013-07-21 21:13:16 -07:00
Brad Fitzpatrick 1149058dbd fs: mutable file append tests
Change-Id: Ibc1b31e7ccc187a92aa1fc8c205b779735126387
2013-07-21 19:01:22 -07:00
Brad Fitzpatrick 33bbd23e32 fs: typo. return read-only file by looking at req flags, not res flags.
Change-Id: I2093f381b99c6ef55a20b88e0bcaade97a3529c7
2013-07-21 18:36:21 -07:00
Brad Fitzpatrick 98eb69b5e1 fs: bunch of read-write FUSE work, debugging, and integration tests.
Change-Id: I74807f693720effb7ae8405259797331f79f59fd
2013-07-21 12:26:05 -07:00
Andrew Gerrand f5950a8c4b fs: support mkdir and unlink
Change-Id: I858766b1680b34201ca1e584d71a687f9714e8fd
2013-07-16 14:22:10 +10:00
Andrew Gerrand 1a9794fcdc fs: add ability to write to and create files
Change-Id: I9310260667e0e091e94a3ba7c3ba0b114c31d41e
2013-07-11 17:22:50 +10:00
Andrew Gerrand 47df629eb9 fs: implement read-only part of mutable file system
Change-Id: Ibbfccecf90edb6b1ad9f0312d5905dd52d329b7b
2013-07-11 10:56:18 +10:00
Andrew Gerrand a9e9cd23ea fs: add base support for mutable roots
Change-Id: I4c8e783f707ee46c9dccd14ab8c213a61146d7dc
2013-07-10 21:10:48 +10:00