Commit Graph

290 Commits

Author SHA1 Message Date
mpl d1e5a1d4c6 search UI: find all roots
This change allows to find all roots (permanodes with the
"camliRoot" attribute) from the search.html ui page.
To achieve that, func (x *Index) SearchPermanodesWithAttr
now uses a prefix string without the query part if the
query is "".

http://code.google.com/p/camlistore/issues/detail?id=35

Change-Id: I396fba683e6e7b2296d1f0df8009c07e3c2cd09d
2012-11-14 11:11:40 +01:00
Brad Fitzpatrick e2a0f194ba Merge "fix search handler test: leading slash for thumbnailSrc removed" 2012-11-12 23:28:46 +00:00
Brad Fitzpatrick 998163ec09 Merge "images: exif, rotation, flipping" 2012-11-12 23:28:29 +00:00
mpl ffcbad9528 fix search handler test:
leading slash for thumbnailSrc removed

Change-Id: If402235986d12bfbba11bbaf4f82d5cb051ec226
2012-11-12 23:20:24 +01:00
mpl fd274ecbb1 fix serverconfig tests,
broken because of discovery move from ui to root

Change-Id: I3e2e5c9280f9ea691d9143aa2bc0a971a6dce325
2012-11-12 23:04:55 +01:00
mpl 858326fec1 images: exif, rotation, flipping
This change implements rotation and flipping on images.
It can be done automatically, using the EXIF Orientation as a hint,
or optionally forced.

Change-Id: I97e887599d6a191964344e81cf8e90922313d958
2012-11-12 21:26:53 +01:00
mpl def34eff94 display thumbnail on the permanode page if content is image.
The serveDescribe search handler now also checks for the
thumbnails req parameter and uses populateJSONThumbnails.
As a result, the client side can directly use thumbnailSrc
from the json response.

http://code.google.com/p/camlistore/issues/detail?id=62

Change-Id: I4ff606f8d3ef291490f05cabdc66219744b53638
2012-11-10 00:04:44 +01:00
mpl 438ae4ecb4 recent permanodes thumbnails
Change-Id: I140766df4b58853e5bc4358f564cf1cfc807882a
2012-11-09 19:43:18 +01:00
Brad Fitzpatrick fddd9e6d2e Merge branch 'master' of ssh://camlistore.org:29418/camlistore 2012-11-09 10:00:55 -08:00
Brad Fitzpatrick ef47a3533e index: fix deadlock, document PeerBlob and friends' lock rules, rename lk to mu 2012-11-09 10:00:47 -08:00
mpl 40bd73ee07 move discovery from /ui/ to / in genconfig too
Change-Id: I1a020696e64c58f911e85070263c015bdbd2be1c
2012-11-08 15:27:17 +01:00
Brad Fitzpatrick db55ee77d9 camput: automatic blobroot path discovery 2012-11-07 22:23:45 -06:00
Brad Fitzpatrick a51cda3a34 client: move stats stuff to stats.go 2012-11-07 21:11:12 -06:00
Brad Fitzpatrick 5221a1ded9 client: document SetHTTPClient 2012-11-07 21:06:10 -06:00
Brad Fitzpatrick 281a6c40f4 client: correct case of SetHTTPClient 2012-11-07 21:03:46 -06:00
Brad Fitzpatrick 3057358cfc Index the dimensions of images. 2012-11-07 23:54:00 +01:00
Brad Fitzpatrick a41269e78e Reindex all dev-server blobs into memindex on restart.
Required some sync work (full syncs on start, blocking full syncs on
start, and also adding a dev-only hack to force a depedency from
search -> sync, to control the handler initialization order, otherwise
publish handlers would race with the sync handler and they'd create
new "blog" and "pics" permanodes and we'd end up with duplicates).
2012-11-07 22:40:17 +01:00
Brad Fitzpatrick 71d7cf9e88 Implement storage generations, init and discovery 2012-11-07 21:14:56 +01:00
Brad Fitzpatrick a6d3151645 gofmt handlers 2012-11-07 20:54:55 +01:00
Brad Fitzpatrick b887087606 Do a TODO: s/errorIsNoEnt/os.IsNotExist/g 2012-11-07 20:02:34 +01:00
Brad Fitzpatrick bccf8ed575 Move discovery up from the UI handler to the Root handler. 2012-11-07 18:57:43 +01:00
Brad Fitzpatrick 5c4d0f71f5 gofmt 2012-11-07 18:49:14 +01:00
Brad Fitzpatrick df5eee936d Make discory work with an Accept header on the root 2012-11-07 18:04:40 +01:00
Brad Fitzpatrick 39d8df7bb2 serverconfig: docs and unexport GenLowLevelConfig 2012-11-07 17:51:42 +01:00
Brad Fitzpatrick 824dbf768f genfileembed: work around a Go 1.0 vs Go 1.1 bug in fmt %q, doing it ourselves.
folder.png and node.png had problems generating on Go 1.1's %q and compiling with Go 1.0.3
2012-11-05 18:43:20 +01:00
Brad Fitzpatrick e582f155ed Thumbnails for permanodes with members (folder) and camliContent to file images 2012-11-05 18:01:03 +01:00
Brad Fitzpatrick 6f82d343ef search: push thumbnail logic into its own function 2012-11-05 17:23:42 +01:00
Brad Fitzpatrick 9b7faaca07 Push server-side thumbnails down into DescribeRequest. Use node.png. 2012-11-05 16:32:56 +01:00
Brad Fitzpatrick 03b8cd143f Start of server-side icon support for recent permanodes list. 2012-11-05 15:48:13 +01:00
Brad Fitzpatrick 832b774025 search: add EdgesTo handler and tests 2012-11-05 15:09:34 +01:00
Brad Fitzpatrick f696007a14 index: implement EdgesTo
For finding back references in the graph.
2012-11-05 10:29:42 +01:00
Brad Fitzpatrick 8c9fba0d71 Merge branch 'rotate' of github.com:bradfitz/camlistore into rotate 2012-11-04 16:34:06 +01:00
Brad Fitzpatrick a2e01c513b Start of images package and EXIF test files.
Change-Id: Ie623ddc7f6df5cbde890a7c0a47affe91780534b
2012-11-04 16:16:02 +01:00
Brad Fitzpatrick e783ad1717 Add another search handler test, for recent permanodes.
Change-Id: Iaf40cd94aba7b96c16fa1b04c2bfcebdfeea870e
2012-11-04 15:26:13 +01:00
mpl 70368abb7d rotate thumbnails
Change-Id: I87dd3008facc5e9c8007fc8ebcba13e98f4a2175
2012-11-04 14:36:31 +01:00
Brad Fitzpatrick 305b4557b9 Improve search handler tests.
Change-Id: I5639447cc95f06d0773e7f15cb601d51c7dd5031
2012-11-04 13:59:19 +01:00
Brad Fitzpatrick 546e2fc14d Send Content-Length on JSON responses
Change-Id: I7e8034b0aea8031011607527f237e24d63b88502
2012-11-04 13:50:35 +01:00
Brad Fitzpatrick ab98ebfa23 some FakeIndex docs
Change-Id: I508e5326c07c8cc7bd39068a2d7e342d0b27884c
2012-11-04 12:38:33 +01:00
Brad Fitzpatrick 933375019d Remove *testing.T from IndexDeps constructor; use Fataler interface
Change-Id: I1cfcc2fa8059dca3f4e082c1dae896974d3d0e02
2012-11-04 12:27:45 +01:00
Brad Fitzpatrick 17a3faf2df index: export NewMemoryIndex
Change-Id: Iaa6d3268ce0f6c531e37c97799deeb31c11eabdb
2012-11-04 12:18:11 +01:00
Brad Fitzpatrick 0ccf9ba3f1 some TODOs
Change-Id: Ifcc2612ea323347ba78d5cb493ebe9fabd3a10aa
2012-11-03 22:27:27 +01:00
mpl 295cbade1b Add postgres support
Also added the -short option for make presubmit
because postgres test is too slow.

Change-Id: I6be21c4d4cd67671eb44ab65333fc32c1e261bbd
2012-11-03 19:59:22 +01:00
Brad Fitzpatrick 4b0d2841ee Add search.EdgesTo method to search interface. No implementation yet.
Change-Id: Ie3a9eb91e218e172b195d55cb9c17cb6c8f819ce
2012-11-03 16:08:37 +01:00
Brad Fitzpatrick 69e155af64 Replace panics with t.Fatal
Change-Id: I7a514f4a67dbf44eff2448a423082c528e9ee365
2012-11-03 14:54:53 +01:00
Brad Fitzpatrick d2e2652eea gofmt
Change-Id: Ib9aca9d648317f1f17fce0e5e7dba5315a95c274
2012-11-03 14:54:45 +01:00
Brad Fitzpatrick 13f85afe8c Merge "Modification to allow windows build and deployement" 2012-11-03 13:25:43 +00:00
Brad Fitzpatrick 1466c77198 Add 'edgeback' key to index, for going backwards.
Change-Id: I43057a6fb96c3e8d9364002288d5c7b9ad2fd034
2012-11-03 14:25:48 +01:00
Brad Fitzpatrick 7cde834f2a Merge branch 'master' of ssh://camlistore.org:29418/camlistore 2012-11-03 14:18:26 +01:00
Brad Fitzpatrick 9b6cb04c6e pkg/index: remove pre-Go 1 trimRFC3339Subseconds
Confirmed it's no longer needed.

Change-Id: I83fdbec205476cd749c668e04b26a304c543e7b6
2012-11-02 13:36:33 +01:00
Maxime Lavigne 5a7510c384 Modification to allow windows build and deployement
Webserver were initialized with "tcp" and ":3179" by default and
listenURL assumed that it would be treated as IPv6 and replaced [::]
by localhost. Host that were listening on IPv4 0.0.0.0 didn't get
the modification.

Receive in localdisk were using link that failed on windows plateforms.

Camlistored didn't use Json Marshaling which caused problem with the
way Windows stores its paths.

Change-Id: I9f62f7d46399c3514707383efcb2752dbaf1f420
2012-10-30 23:17:58 -04:00
mpl c61feaa0bd wizard: placeholder for gallery field
The gallery field of the wizard is awkward and undocumented.
This placeholder at least gives a hint to devs (and maybe users)
on what is expected in that field.

Change-Id: Ie707dbecaf87ec98e267fcf6b987cd3b6a1d13e0
2012-10-29 11:10:42 +01:00
Brad Fitzpatrick 22495f8bfd schema: adjust rollsum splitting algorithm, make it the default
We might tweak the parameters yet, and I want to add a bunch more
tests, but this is a major relief for me.  I've been reluctant to
put too much data into my personal Camlistore instance until I
figured out how I wanted the file chunks to split.

This also simplifies (deletes) a fair bit of code.

Change-Id: I35c38cc5d39a8a43e3f62445434f8d7fd5de5d17
2012-10-29 02:03:05 +01:00
Brad Fitzpatrick 044194b746 schema: document PopulateParts, add a missing error check
Change-Id: Ic9d65fe3e56c4233f481254c6f730a8b382fb4ba
2012-10-29 00:10:31 +01:00
Brad Fitzpatrick 15ad53c545 rollsum: revert change. I was using the API wrong.
Change-Id: I5973ae82188f67493f15aa1359867c220a2b28eb
2012-10-28 13:40:29 +01:00
Brad Fitzpatrick d2b8dc2327 rollsum: decrease split frequency by 4x
Change-Id: Ibf945004abff61134fbcd07f47734ea3f0944ba0
2012-10-28 13:08:39 +01:00
Brad Fitzpatrick 82329b5882 rollsum: log stats after benchmark run
Change-Id: I2c8983d13ea80efb383ebf77316e6ad0a5da9f57
2012-10-28 13:07:42 +01:00
Brad Fitzpatrick a364983788 Merge "mongo: added user:pass style authentication" 2012-10-28 11:05:19 +00:00
mpl 231c3d7405 wizard: bugfix, TLS is now https
this should have been done along with
the other changes in 94cb8cf05a

Change-Id: I376e6616c3aeaa9d5accb65b64a43d101de25d3b
2012-10-25 18:39:55 +02:00
mpl 8138cf8fd6 mongo: added user:pass style authentication
The mongo indexer didn't have any authentication
scheme, so this change adds the user:pass@host
authentication.
The mongo indexer tests now use the root:root
credentials.

Change-Id: I7867f46e3b5f0a4a1a50f86f26d76c0bfd69e9ac
2012-10-25 00:35:19 +02:00
mpl 3e9aef409c search handler: documented the "attr" values when searching for a permanode
Change-Id: I607d1fe016078912faac9864630552cd2b496aa2
2012-10-24 17:26:46 +02:00
Brad Fitzpatrick 0645035db7 Make serverconfig tests pass again.
Change-Id: I83077f7573a81b643639c78de1a72169e82b09e9
2012-10-20 12:33:39 -07:00
Brad Fitzpatrick d0faee5726 Compile with Go tip.
Change-Id: I518c357613049e80939ccab9c30b6aaf03ebe140
2012-10-20 12:27:50 -07:00
mpl 83ce85b59f old TODOs
Change-Id: Ifa8877f594e65e081536b97f536106301f5c9836
2012-10-19 23:37:14 +02:00
Brad Fitzpatrick 44b0592135 Merge "jsonconfig: remove TODO and revert bogus change; float always works apparently." 2012-10-19 20:30:34 +00:00
Brad Fitzpatrick 6b8fc9b23a webserver: use runsit listener instead of net.Listen
Change-Id: I635d1ac5dcd89905a2fadf4b71d3bffec88389ad
2012-10-19 13:24:18 -07:00
Brad Fitzpatrick 7fba04d71a Compile with Go tip again.
See https://groups.google.com/forum/?fromgroups=#!topic/golang-dev/yULJVYAbIzg

Change-Id: I11e8dd99a9ec4a065f03b888b616c13e5ce0e92c
2012-10-19 12:51:10 -07:00
mpl 41e8215abe jsonconfig: remove TODO and revert bogus change; float always works apparently.
Change-Id: Ia80edbfd122063895ffb76d361c33b2cd70a725c
2012-10-19 19:26:15 +02:00
mpl 94cb8cf05a Be consistent between high and low level configs for https
Change-Id: Ie7e1d9c554e09a34416a2ae3ac9e10f1a2f8fce0
2012-10-18 11:58:30 +02:00
Brad Fitzpatrick c1730d6151 Merge "cammount: check that input is a directory blobref" 2012-10-18 05:47:30 +00:00
mpl 2d4f37176c cammount: check that input is a directory blobref
Change-Id: I9b33f2ae31ef902e01ddaa0504e1c960bb4e3640
2012-10-18 00:33:37 +02:00
mpl 6ac49843ab fix IPv4 assumption
Change-Id: I334e6ccb5b26a083b55750271262a3469f63e78e
2012-10-16 00:20:57 +02:00
mpl 0b15eb0cc8 handlers in wizard: swap arguments to match http handlers style
Change-Id: I78e9b218392979017682c375ce8a4b636ed3a832
2012-10-15 17:29:12 +02:00
Brad Fitzpatrick 42fb5290fd Merge "If baseURL was not user defined, this change makes sure it is dynamically defined from incoming requests. The listen flag has moved from pkg/webserver to server/camlistored and is not using runsit anymore. httputil.ServerError now only outputs the full error if we're in internal/debug mode. http://code.google.com/p/camlistore/issues/detail?id=53" 2012-10-15 14:33:43 +00:00
mpl 844551d0b2 If baseURL was not user defined, this change makes sure it is
dynamically defined from incoming requests.
The listen flag has moved from pkg/webserver to server/camlistored
and is not using runsit anymore.
httputil.ServerError now only outputs the full error if we're
in internal/debug mode.
http://code.google.com/p/camlistore/issues/detail?id=53

Change-Id: I55a1c6d43a549d9ee2022742a83aa142dc19e02a
2012-10-15 16:26:50 +02:00
mpl a534e02c2d camput: client becomes StatReceiver.
also added ReaderSize function to
guess before reading the size of
a source.

Change-Id: I659e174821696b3c58759b132b82710a49583018
2012-10-11 11:51:12 +02:00
mpl 6baedb3dc7 delete permanode: defined the keyDeleted index key,
and added check in index searches to exclude deleted
permanodes from the results

Change-Id: I7097e4a5dc893a212c91dff987ad7e505793fda9
2012-10-09 01:29:59 +02:00
Brad Fitzpatrick e557066f1d schema: more cleanup, making FileReader a ReaderAt.
testing TODOs and further cleanup TODOs remain.

Change-Id: I997153c66805cfe3220d3d735322be14b68b75dc
2012-08-23 20:09:31 -07:00
Brad Fitzpatrick faf3c90d6f schema: more docs and some FileReader work
Change-Id: Id9e754e3b6358a651a826a4cdeb00efb330e3c3a
2012-08-23 18:44:03 -07:00
mpl b355a29150 just one missing map[string]interface{} -> Map conversion
Change-Id: I616ef7260dac9187ea5157f8d47c1b05815ad723
2012-08-22 15:09:42 +02:00
Brad Fitzpatrick 1a51f44196 schema: fix test
Change-Id: Iaacc42167e40233dcb9f5f03bfcfcfe86655a2b1
2012-08-22 04:38:51 +10:00
Brad Fitzpatrick 228c0ca56f schema: change PopulateSymlinkMap to be Map.SetSymlinkTarget
Change-Id: I072ce65331048b54c90c32e951f5034060c1764c
2012-08-22 04:38:25 +10:00
Brad Fitzpatrick a8629ec5dd schema: unexport NewBytes
Change-Id: Ia6ec7c9aec33319bf277b93a189b4fa43d0540b9
2012-08-22 04:33:06 +10:00
Brad Fitzpatrick 439db110ec schema: more cleanup; ditch the Claimer type
Change-Id: I2a52564c1a99c96a2cd8bc84a791784f0a4e4b2e
2012-08-22 04:31:48 +10:00
Brad Fitzpatrick b67d753364 schema: docs and cleanup
Change-Id: Ib720b7f711d71bdeaaf33374f376f14b30a5a97c
2012-08-22 04:14:47 +10:00
Brad Fitzpatrick 11eac8721e Convert more code to use schema.Map instead of map[string]interface{}
Change-Id: I285fd1b0e730ebd32069688a71cb992c43770ade
2012-08-22 03:47:38 +10:00
Brad Fitzpatrick 3cd7f75f89 schema: introduce type Map
Change-Id: I8209930d8505a4cb74b8ea29b5b57f2cd6bd14d2
2012-08-22 03:36:36 +10:00
Brad Fitzpatrick 0a0b012eba camput,schema: cleanup, docs
Change-Id: Ifa94cb1e9e086f85222692508ab12616f4cc3211
2012-08-22 03:31:44 +10:00
Brad Fitzpatrick 8fec29c974 camput: make file permanodes consistent between runs (fix claim times)
Change-Id: Idd2bbda2372802772703eaeaf6fe2e74f9bcdd83
2012-08-21 20:43:38 +10:00
Brad Fitzpatrick b815a2b758 camput: add -filenodes option for per-file content-based planned permanodes
Change-Id: Ic03b990654d860e03b4cc710de92f7d40c392df8
2012-08-21 16:11:10 +10:00
Andrew Gerrand aa5392f510 import image/gif to handle gif files
Change-Id: If9742b89562909bd70d2c9b8ceddd394975c7db0
2012-08-05 11:01:12 +10:00
Andrew Gerrand 7874bef34e add bsds to osutil/restart_unix build rule
Change-Id: If5e7f811ad8c656bcfa48f26c12f7ea59ddef830
2012-08-05 10:55:36 +10:00
Brad Fitzpatrick a5169d1468 client: memoize SignerPublicKeyBlobref 2012-08-05 09:25:21 +10:00
Brad Fitzpatrick b3ab27e049 remove log noise 2012-08-05 09:15:17 +10:00
Brad Fitzpatrick 11842fcf33 serverconfig: let baseURL and listen be optionally separate 2012-08-04 21:42:10 +10:00
Brad Fitzpatrick 42833a76e0 serverconfig: be consistent between low- and high-level config
Use TLSCertFile and TLSKeyFile in both.
2012-08-04 21:11:52 +10:00
Brad Fitzpatrick 63418fe0fc webserver: use runsit's listen package, for running under runsit 2012-08-04 11:24:51 +10:00
Brad Fitzpatrick 493f75a72e serverconfig: allow configuring TLS cert/key 2012-08-04 11:12:39 +10:00
Brad Fitzpatrick 51e88cac65 Support for "planned permanodes" in pkg/jsonsign, pkg/schema and cmd/camput.
A planned permanode involves setting the contents of the permanode
(instead of a random string) as well as the OpenPGP signing time so
the resultant bytes of the blob (and thus its blobref) is deterministic.

This allows multiple independent devices (my laptops) to create the
same permanodes for the same files (photos backed up from my phone)
when offline (airplane) and then when they sync later, still only have
one permanode per unique file.  This means that tagging and other
metadata applied to permanodes on one laptop merge cleanly with
metadata from the other.
2012-07-28 16:32:31 -07:00
Brad Fitzpatrick 8c293e34b6 Per Go initialism style, rename Json to JSON 2012-07-28 15:42:56 -07:00
Brad Fitzpatrick 2db390d58e Add schema.NewPlannedPermanode 2012-07-28 15:41:19 -07:00