Commit Graph

170 Commits

Author SHA1 Message Date
Attila Tajti 0d49cffb44 pkg/client: set camliType, use correct blobref
- make sure camliType is set to "file" in UploadFile
- fileMapFromDuplicate uses the blob ref calculated from the
  file schema blob in the call to ReceiveBlob
- add test for above changes
- TestTransportSetup updated after field name change
- hasComponent fixed for case fullpath[:len(component)] == component
- hasComponent returns true if the first match is missing either separator,
  but there is another match down the path
- add special handling for Windows in hasCompontent and TestIgnoreFns

Change-Id: I96325b060e9421709bd9f684bcc9eceed7135f7b
2016-01-06 17:44:08 +01:00
Brad Fitzpatrick 66f0ddac0e pkg/client: fix directory upload deadlock, clean up HTTP handling
Unexports pkg/client.Client.TransportForConfig in the process.

Fixes camlistore/camlistore#662

Change-Id: Ifc2d6b0fee70d5db86654897d2d377b343b6d38b
2015-12-29 23:39:29 -08:00
Brad Fitzpatrick fba8b457de pkg/client: wire up http2 support, unexported Client.InsecureTLS
Also add a flag to camtool discovery:

$ camtool discovery -httpversion
HTTP/2.0

Fixes camlistore/camlistore#661

Change-Id: I6db392e01bd176a2757ebdb4a0e881eebaec9b03
2015-12-29 11:33:38 -08:00
mpl 74d9c71365 pkg/client: add UploadFile
Most of the code related to uploading a file with a client is actually
locked into the camput code. And uploading a file is something the
scanning cabinet app needs to do.

Refactoring the camput file code into pkg/client, and cleaning up
pkc/client is a lot of work. Therefore I propose with this change to
first introduce the minimum I need in pkg/client that I can use in
the scanning cabinet app.

Change-Id: Ie8690e66f0332be76a3a41fa060db410d01798e2
2015-12-18 23:05:06 +01:00
Tamás Gulácsi 9f7e1df32b Remove pkg/context
Use golang.org/x/net/context instead.

Fixes #648.

Change-Id: I676a2f25458be97610a49d6f954f2102cbd373fa
2015-12-12 23:09:02 +01:00
mpl 40be4d8da2 vendor: go4.org/jsonconfig (and errorutil)
Previously pkg/jsonconfig and pkg/errorutil

Copied from go4.org at rev d1b8a2fb2de6160036e4801aa5e4d855571078b8

Change-Id: I673ed55b0825baa2607289b6082f205100261d7a
2015-12-01 17:21:49 +01:00
mpl 57a45dcd5c pkg/jsonconfig: remove dependence on pkg/osutil
So it can be moved to go4.org

Change-Id: I1b6d427cf66935fa85c746a90914a42f57b28613
2015-12-01 00:43:39 +01:00
mpl 40c0d80b07 vendor: go4.org/wkfs
Previously pkg/wkfs

Copied from go4.org at rev 3970dd22c0e7e2397304ad626a20b72f8db41b58

Note that pkg/wkfs/gcs has not been moved yet (for dependencies
reasons).

Change-Id: I737e88a360bccfc643e1fa43541f65f2384c3157
2015-11-27 00:23:03 +01:00
mpl 12eddf9c19 vendor: go4.org/strutil go4.org/syncutil
They were internal packages (under pkg), which we are now moving to
go4.org, so we in turn need to vendor them in now.

Change-Id: I92224f731404d0bd4ca1c57492bed37cb3367ed4
2015-11-21 00:21:20 +01:00
mpl 6dfe405666 camput init: get client config from server help handler
This is particularly useful for getting clients fully configured for a
server on the same host.

Context:
https://github.com/scaleway-community/scaleway-camlistore/issues/2

Change-Id: I667dd32a80cba4e1e6f6a4ca86a0497a72047d30
2015-09-28 16:44:38 +02:00
mpl 8127040762 pkg/client: use Transport.DialTLS
We can now use DialTLS when we need a custom TLS setup, i.e. with
android, or when using self-signed certificates, or when totally
skipping certificates verification.

This allows us to get rid of the scheme rewriting hack, which simplifies
things, in particular because we have less tricky URL/host parsing to
do.

For the sake of tests and not affecting "real" code, I introduced the
fake_android build tag, so the tests can use custom functions to
simulate testing whether we're on android.

Fixes #566

Change-Id: I72ac2bb69ad2365e98dd6ca2e7016ce9c2d7c57e
2015-09-26 00:09:26 +02:00
Mathieu Lonjaret 52e370a3fa Merge "pkg/client: be stricter when rewriting URLs" 2015-09-25 12:32:27 +00:00
Gina White 56bb5d1a38 Address TODO by moving CAMLI_DEBUG checks into env
I also wonder if we want to take a more comprehensive approach to
environment variables, perhaps populating a struct at startup that other
code can consult later.  But it might be too soon for that kind of thing
in this project.

Change-Id: I65a34622bf906c1256ceb357ba983bc5acd6b887
2015-08-25 07:22:14 -07:00
mpl 21a584022f pkg/client: be stricter when rewriting URLs
url.Parse in Go 1.5 now fails with the scheme://IPv6:/foo form (trailing
colon). But it still works when the hostname is an actual hostname, or
an IPv4, which seems inconsistent.
This CL therefore not only accepts this new failure (which comes with Go
1.5) but also considers any host with a trailing colon as an invalid URL
and refuses to rewrite it.

There probably are other places in Camlistore where we should the same.

See https://github.com/golang/go/issues/12200

Change-Id: Ib9ea95a71013b5b53f2fd99415015ec916bb4d9d
2015-08-24 18:46:08 +02:00
Mathieu Lonjaret eb7db12247 Merge "pkg/client: set default port to 443 when using self-verified tls." 2015-06-02 21:50:43 +00:00
Salman Aljammaz 426496ab8d pkg/client: set default port to 443 when using self-verified tls.
Fixes #616

Change-Id: Id6c8771d06aa7ccb024459cabd9afb4433a3d011
2015-06-02 17:11:03 +03:00
Fabian Reinartz 76a445a956 pkg/server: decouple sign handler from UI, replace JSON maps with structs
Signhandler was moved from the ui to the root handler and genconfig
adjusted accordingly.
New discovery struct types to replace the JSON maps are defined in
pkg/types/camtypes. The changes are related to Issue #539.

Change-Id: Ib3f12feeaf6fa81107de2d468cab491723e590ae
2015-05-25 09:28:07 +02:00
Brad Fitzpatrick af36d78bd7 env: add IsDev and use it everywhere
Change-Id: I5547ce989c05f1f48774f24abee41c489326baa0
2015-04-02 05:55:01 -07:00
Brad Fitzpatrick 5c0e9a1fbe Rename search.IGetRecentPermanodes to search.GetRecentPermanoder
Better Go style. Plus Permanoder is funny. And IFoo looks like C#.

Change-Id: Ie418117c15a145c6c61a2f7e5fb31ae53a5d8bb6
2015-02-02 10:45:41 +01:00
Brad Fitzpatrick 9c3f196019 client: add Client.NewPathClient constructor
Change-Id: I49321e33d32f539f98ba6f1c1b405f94d1f13839
2015-01-19 15:05:29 -08:00
Brad Fitzpatrick eeb8f61dd3 client: make Fetch return os.ErrNotExist on 404, per blob.Fetcher contract
Change-Id: I8a554c6facb34f2257e3e6035030e404a051ec50
2015-01-19 15:04:54 -08:00
Salmān Aljammāz 8bee1495ab client: log when GPG signatures mismatch
Mismatching GPG keys seems to be a very common issue for newcomers.

Change-Id: If83eee5d0ba5f0c85a4d75a31ab15a5469c2965d
2015-01-12 14:06:22 +03:00
Jingguo Yao 351b4f9726 client: use index expression over rename map
Use index expression instead of loop.

Change-Id: I26b85e239a227f234fa7432bf7444b6c73e949ba
2014-11-09 10:28:35 +08:00
Brad Fitzpatrick 0e841e0f51 Rename pkg/misc to pkg/hashutil
It only had one function anyway.

Change-Id: Iec4b7868d823148c8e1d2607347fe9d3c1463403
2014-11-06 13:31:58 -03:00
Brad Fitzpatrick a16c015f32 pkg/client: make Describe use a POST request to the server
Otherwise it can't send Describe Rules.

Also in this commit: a bunch of new tests from when I thought I was going crazy
and trying to debug the search system, before I realized the problem was that the
client was doing a GET request and dropping most of my DescribeRequest.

Change-Id: I4ea9bed80f0d7d6b86814527b63acc3586ac1d06
2014-10-19 17:36:22 +02:00
mpl 65fb60de32 client: add test with multiple ignores in the config
Illustrates https://camlistore.org/r/2296

Fixes http://camlistore.org/issue/390

Change-Id: I8d98a938458239502617352f8eaf1a12f96e50d0
2014-08-26 00:46:12 +02:00
mpl e8522cca36 website: allow HTML in camli errors. use website for client error.
http://camlistore.org/issue/276

Change-Id: I16f139fe2b6f1200250db668fa415ae42ff2c171
2014-08-21 23:13:56 +02:00
Brad Fitzpatrick ec900d6bd9 pkg/client: some more wkfs usage
Change-Id: Ib4098304beb52492e1b62bfeb88962c9d4095c0d
2014-08-08 11:02:06 -07:00
mpl 975e251c36 client/upload: decode JSON response into struct instead of map
Change-Id: I63c660ae48bf7d896eed69aed84a805158709c86
2014-08-08 16:02:57 +02:00
mpl f6f1280c33 Merge "client: actually print error from server when upload fails" 2014-08-07 13:28:25 +00:00
mpl f2951298b1 client: actually print error from server when upload fails
Change-Id: I98236bc4e6680e0a3d89f2a1a1c47b77ef90e0a0
2014-08-05 22:06:08 +02:00
mpl 0188292029 client: output response on sharing error, for easier debugging
Context: http://camlistore.org/issue/462

Change-Id: Iea68138fb2b5cfeb66773a83dc84ebbcd6442f44
2014-08-04 23:37:22 +02:00
mpl 26cb64e825 Introduce search.QueryDescriber interface. *client.Client implements it.
There are several places where we're using a *search.Handler, when we
only want to be able to do query and describe requests. When these
places need to be moved also out of the context of camlistored, (like
when making a server app such as the publisher), the QueryDescriber can
be used, and the adequate implementation (*search.Handler,
*client.Client, or any other) is used depending on the context.

Change-Id: Ie1c5bb6f1bbba32c24a8eb2d1175af95ffcc8335
2014-07-21 22:20:24 +02:00
Brad Fitzpatrick cc956829f3 Rename android.go files to androidx.go
Go 1.4 (Go tip currently) defines a new GOOS=android, and a GOOS.go file is ignored on
other operating systems. But these android.go aren't actually meant to be only run on
GOOS=android, since we just build Linux binaries on Android for now.
2014-07-08 11:18:40 -07:00
Brad Fitzpatrick 985bb3482a client: allow missing config file when using camget --shared= 2014-06-23 16:57:06 -07:00
mpl a34f9e2669 publisher app
http://camlistore.org/issue/365

Change-Id: I281fdcbbe6a2bdf15607e75a21bc93b453f82c85
2014-06-15 00:03:03 +02:00
mpl 21dda2b4ef app/hello: dummy server application (hello world)
Change-Id: I6690b9459325af5a76d1de679d56701eefdd195e
2014-05-08 22:15:01 +02:00
mpl 28ac303dc7 client: add NewFromParams constructor
Also add missing CAMLI_DISABLE_CLIENT_CONFIG_FILE checks

Change-Id: I1f87d9a1a275d603109bae44aa3cf0c7c873be94
2014-05-08 01:49:52 +02:00
mpl e34c4bc828 client: replace CAMLI_CLIENT_IDENTITY with CAMLI_KEYID
And remove all mentions of CAMLI_CLIENT_IDENTITY because there isn't any
need for the both of them.

http://camlistore.org/issue/412

Change-Id: I0d6b8cff0f23738231de98171a23550d82f05fe9
2014-03-31 18:56:11 +02:00
mpl cdc4ed5ae2 camput init: ring/key related fixes
1) Removed exec call to gpg, because it automatically looks in .gnupg/,
which we don't use anymore as a default.
2) Now taking into account global --secret-keyring flag.
This flag is now in osutil.
3) New or modified funcs in osutil
4) Made sure --gpgkey works too.
5) Cleaned up error messages and hints.

Context: http://camlistore.org/issue/364
         http://camlistore.org/issue/368

Change-Id: I2e51032ed0597da656db100d72f5588b37308e1a
2014-03-28 19:55:18 +01:00
Brad Fitzpatrick bfe51f3ad5 gofmt
Change-Id: Ie6be315d02a439cbfdd870a0b0b7f16cba12a168
2014-03-20 12:29:45 -07:00
Brad Fitzpatrick bf94a73859 Get rid of SeekFetcher vs StreamingFetcher distinction and complexity.
StreamingFetcher is now just Fetcher, and its FetchStreaming is now
just Fetch.

SeekFetcher is gone. Blobs are max 16 MB anyway, so we can slurp to
memory when needed. The main thing that cared about SeekFetcher
was the GET handler, ServeBlobref, because http.ServeContent needed
one for range requests. That's rewritten in an earlier commit, using
the FakeSeeker from another earlier commit.

Lot of code got simpler as a result.

Change-Id: Ib819413e48a8f9b8d97f596d0fbf771dab211f11
2014-03-14 12:29:13 -07:00
Brad Fitzpatrick 76ba33b27a Move ListMissingDestinationBlobs from pkg/client to blobserver
Change-Id: I7489531fe2a1a5127b2212d5a5f80d4c4c614353
2014-03-06 15:04:15 -08:00
Brian Gitonga Marete d373f86d9f pkg/client: Fix ignored files matching.
Because of Go's rules regaring closures (function literals) and bound
variables, in the case in which there were multiple patterns in the
ignoredFiles configuration variable, the entire slice (vector) of
registered matching functions was being called with the
last-configured ignoredFiles pattern as the first variable. This meant
that ignoredFiles did not work if it contained more than one pattern
and in certain circumstances, such a configuration would lead to a
crash.

Change-Id: I0b991aa74d079d4ce27fc0ba373e2ee2cf9eb772
2014-03-04 17:30:50 +03:00
mpl 08923e1c00 camtool sync: warn when src and dest have different identities
http://camlistore.org/issue/281

Change-Id: I2e2d595534b14476293a031b88e312de3e0b3f4a
2014-03-01 23:47:49 +01:00
Brad Fitzpatrick 448bbfa0e2 client: better enumerate docs
Change-Id: Id46950fb578f31e17a7600bedf3108eb386826c8
2014-03-01 11:48:20 -08:00
Brad Fitzpatrick bbfcc27d85 sync: use blob.Ref.Less instead of stringify to compare, fix API wart, more tests
Change-Id: Iffd98463becbcf6d62dd666b38fb00572b34725b
2014-03-01 11:34:57 -08:00
mpl c4c4d6407a client: default to None auth if no auth in config
Defer warning until discovery fails.
Also fix SetupAuth error checks in camtool.

Change-Id: Ib554595603994e929d465e4e0de6acaf319d5b42
2014-02-27 01:37:59 +01:00
Brad Fitzpatrick 520191ecc5 client: don't access UserClientConfigPath or osutil.Homedir unless necessary
Change-Id: I618941204eb561461aab960bde576963296cea87
2014-02-23 10:59:20 -08:00
Brad Fitzpatrick 830864b715 client: don't call osutil.HomeDir during test.
Prep for future changing which makes this crash.

Change-Id: Ic02f7319deffb1d47e89f45d066fa4e62d45e499
2014-02-23 09:34:16 -08:00