Commit Graph

30 Commits

Author SHA1 Message Date
Paul Lindner 459c75410e all: more renaming of Camlistore to Perkeep
Change-Id: I118e3cbcf20d80afeffc84f001388c4556f21628
2018-01-30 03:02:56 -08:00
Brad Fitzpatrick 194d4f9443 blobserver, all: add contexts to ReceiveBlob, Fetch & million resulting deps
I had intended for this to be a small change.

I was going to just add context.Context to the BlobReceiver interface,
but then I saw blob.Fetcher could also use one, so I decided to do two
in one CL.

And then it got a bit infectious and ended up touching everything.

I ended up doing SubFetch in the process by necessity.

At a certain point I finally started using context.TODO() in a few
spots, but not too many. But removing context.TODO() will come in the
future. There are more blob storage interfaces lacking context, too,
like RemoveBlobs.

Updates #733

Change-Id: Idf273180b3f8e397ac5929c6d7f520ccc5cdce08
2018-01-18 16:22:16 -08:00
Brad Fitzpatrick d6a0b05df0 Rename import paths from camlistore.org to perkeep.org.
Part of the project renaming, issue #981.

After this, users will need to mv their $GOPATH/src/camlistore.org to
$GOPATH/src/perkeep.org. Sorry.

This doesn't yet rename the tools like camlistored, camput, camget,
camtool, etc.

Also, this only moves the lru package to internal. More will move to
internal later.

Also, this doesn't yet remove the "/pkg/" directory. That'll likely
happen later.

This updates some docs, but not all.

devcam test now passes again, even with Go 1.10 (which requires vet
checks are clean too). So a bunch of vet tests are fixed in this CL
too, and a bunch of other broken tests are now fixed (introduced from
the past week of merging the CL backlog).

Change-Id: If580db1691b5b99f8ed6195070789b1f44877dd4
2018-01-01 16:03:34 -08:00
Paul Lindner b09cd377d7 Switch to stdlib context from golang.org/x/net/context
This switches most usages of the pre-1.7 context library to use the
standard library.  Remaining usages are in:

  app/publisher/main.go
  pkg/fs/...

Change-Id: Ia74acc39499dcb39892342a2c9a2776537cf49f1
2017-11-26 01:12:26 -08:00
Paul Lindner fa46c3935d Correct various misspelled words
Change-Id: I236e880526e4c2b0bd318da041983d557e0aa885
2017-09-11 08:33:31 -07:00
Brad Fitzpatrick e93e4f3822 Fix deadlock in search/index.
The describe requests were launching a storm of RLocks which weren't
safe in the presence of goroutines trying to acquire write locks.

Instead, make the corpus locking the responsibility of the caller and
add Lock/Unlock/RLock/RUnlock methods to the index and move locking up
a level.

This also adds a fair bit of context.Context plumbing which was used
in earlier debugging.

Fixes camlistore/camlistore#709

Change-Id: I8d7254d1e1da541f8c080d62f5408aac807fd3b1
2016-04-22 14:57:10 -07:00
Will Norris 77ed42edf8 add canonical import paths
The import path was added to the go file that included the package
documentation if one existed.  Otherwise, I used what seemed to be the
primary file for the package.

Fixes #689

Change-Id: If51be0e86529fd6f179e80af6781e639f8550fd2
2016-03-13 19:57:14 -07:00
mpl 26cb64e825 Introduce search.QueryDescriber interface. *client.Client implements it.
There are several places where we're using a *search.Handler, when we
only want to be able to do query and describe requests. When these
places need to be moved also out of the context of camlistored, (like
when making a server app such as the publisher), the QueryDescriber can
be used, and the adequate implementation (*search.Handler,
*client.Client, or any other) is used depending on the context.

Change-Id: Ie1c5bb6f1bbba32c24a8eb2d1175af95ffcc8335
2014-07-21 22:20:24 +02:00
mpl 1ee5fd20c5 search: deletions are not modifications
1) pkg/search: documented that deletions times do not
qualify as modtimes

2) pkg/index: got rid of DeletedAt, and keyDeletes

http://camlistore.org/issue/191

Change-Id: I39578913345454d36af4599e29e7053f46577846
2013-11-29 00:29:57 +01:00
Brad Fitzpatrick 705107ad80 search/index: invert depedency. search now depends on index.
creates new package types/camtypes for misc types needed by both. might eventually go away as
search matures.

Change-Id: Ib771ead7bea39936ba478b7e5d58de997060861b
2013-11-16 15:00:30 -08:00
Brad Fitzpatrick e0c3cf9b01 search: optimize ClaimList.Less sort. duh. 2013-11-14 15:30:59 -08:00
Brad Fitzpatrick abdfd02bbb search: add camliImportRoot to set of indexed properties
Change-Id: I63a16e704569b2c834cda7cc9c84ce14bb31511d
2013-10-24 22:49:31 -07:00
Brad Fitzpatrick 17ecf8df07 search: start of general Query support. very rough draft.
Start of something that builds and 'works', but barely fleshed out at all.
Likely to change drastically yet.

Change-Id: I7fba25528a341ff2dbbc4dbb579f2450d7889320
2013-10-18 17:17:35 -07:00
Brad Fitzpatrick c8badb4404 Misc cleanups
Change-Id: I9183ca1ec7a98df210373f27ec3493c996f00f74
2013-10-05 20:46:39 -10:00
mpl d488c576fc search: support for static directory children
This change introduces a new index entry
to help with finding the children of a static directory.
It also fixes ResolvePrefixHop so that it takes
into account static directories, and not only collections.

This is the first step to support publishing static directories.

http://camlistore.org/issue/179

Change-Id: I5666e5caa6c782004054ae4c19a6b6119d4fda8b
2013-09-10 23:06:48 +02:00
Brad Fitzpatrick 0bdf20884b all: delete pkg/blobref; convert all from *blobref.BlobRef to new blob.Ref
Change-Id: Id2dfb7f19452bedf4f3c9310b36227fd8117b225
2013-08-03 19:54:30 -07:00
Brad Fitzpatrick 9468e5ba70 More docs. Every package is documented now.
misc.CountingReader moves into readerutil.

pkg/atomics is folded into pkg/types.

pkg/test/testdep is folded into pkg/test, with better name/docs.

Old cruft from pkg/webserver is deleted.

Change-Id: I3f72d8b29804254ef944995fb085837c878f79f5
2013-07-07 21:12:30 -07:00
mpl 0dfd84a7d8 search handler: return correct thumbnail dimensions
images: DecodeConfig to get the predicted width
and height after EXIF correction
search&index: add GetImageInfo and use it in search
to predict the thumbnail dimensions

http://camlistore.org/issue/115

Change-Id: I358136a2ab03ea09c8f8fd2fa0dc574921c819c5
2013-03-25 17:06:15 +01:00
Brad Fitzpatrick ace9474d95 index: index file times too, and return in index.GetFileInfo.
Change-Id: I59d91f0938c725a4cbdf5ca933cdff3529e25f5f
2013-02-18 21:31:41 -08:00
Brad Fitzpatrick 8155b3867a Fix case of MimeType to MIMEType, per Go style.
Change-Id: I6c67a61cea2c71d7fa5d52c4ae283e33e993fd0f
2013-02-18 11:16:13 -08:00
Brad Fitzpatrick 3605285924 Add pkg/types with types.Time3339; use it in search/handler JSON types
Change-Id: I79bf1484ef8b3548bca2e732f189352bcfbcd6db
2013-02-08 18:10:31 -08:00
mpl 67c5678062 Index directories with "fileinfo", and use this to
find their name in a search request.

Fixes http://code.google.com/p/camlistore/issues/detail?id=79

Change-Id: I755afd8f52dbd2f8a48ba72bed0a6b0192d1dd71
2013-01-09 16:59:04 +01:00
mpl d1e5a1d4c6 search UI: find all roots
This change allows to find all roots (permanodes with the
"camliRoot" attribute) from the search.html ui page.
To achieve that, func (x *Index) SearchPermanodesWithAttr
now uses a prefix string without the query part if the
query is "".

http://code.google.com/p/camlistore/issues/detail?id=35

Change-Id: I396fba683e6e7b2296d1f0df8009c07e3c2cd09d
2012-11-14 11:11:40 +01:00
Brad Fitzpatrick ef47a3533e index: fix deadlock, document PeerBlob and friends' lock rules, rename lk to mu 2012-11-09 10:00:47 -08:00
Brad Fitzpatrick f696007a14 index: implement EdgesTo
For finding back references in the graph.
2012-11-05 10:29:42 +01:00
Brad Fitzpatrick 4b0d2841ee Add search.EdgesTo method to search interface. No implementation yet.
Change-Id: Ie3a9eb91e218e172b195d55cb9c17cb6c8f819ce
2012-11-03 16:08:37 +01:00
Brad Fitzpatrick 1466c77198 Add 'edgeback' key to index, for going backwards.
Change-Id: I43057a6fb96c3e8d9364002288d5c7b9ad2fd034
2012-11-03 14:25:48 +01:00
Brad Fitzpatrick 3b30cbd63f add TODO
Change-Id: Icd386f58b9004b78dd52bdaf32cca257d1593495
2012-05-13 13:30:42 -07:00
Brad Fitzpatrick 1b63c44b4e Update all os.ENOENT references to ErrNotExist. Mostly docs/dead code.
Change-Id: I2bb4d5a9d9c610e8dce2309335d06046dbb1e049
2012-05-13 12:06:21 -07:00
Brad Fitzpatrick 0714a463c9 Update from r60 to [almost] Go 1.
A lot is still broken, but most stuff at least compiles now.

The directory tree has been rearranged now too.  Go libraries are now
under "pkg".  Fully qualified, they are e.g. "camlistore.org/pkg/jsonsign".

The go tool cannot yet fetch from arbitrary domains, but discussion is
happening now on which mechanism to use to allow that.

For now, put the camlistore root under $GOPATH/src.  Typically $GOPATH
is $HOME, so Camlistore should be at $HOME/src/camlistore.org.

Then you can:

$ go build ./server/camlistored

... etc

The build.pl script is currently disabled.  It'll be resurrected at
some point, but with a very different role (helping create a fake
GOPATH and running the go build command, if things are installed at
the wrong place, and/or running fileembed generators).

Many things are certainly broken.

Many things are disabled.  (MySQL, all indexing, etc).

Many things need to be moved into
camlistore.org/third_party/{code.google.com,github.com} and updated
from their r60 to Go 1 versions, where applicable.

The GoMySQL stuff should be updated to use database/sql and the ziutek
library implementing database/sql/driver.

Help wanted.

Change-Id: If71217dc5c8f0e70dbe46e9504ca5131c6eeacde
2012-02-18 21:53:06 -08:00