Commit Graph

4992 Commits

Author SHA1 Message Date
Mathieu Lonjaret 5d86dc153f Merge "Merge goexif with upstream package" 2014-09-05 13:42:59 +00:00
Fabian Wickborn f0d9c04bc2 Merge goexif with upstream package
This pulls the changes from the current HEAD of
https://github.com/rwcarlsen/goexif
(rev cf045e9d6ba052fd348f82394d364cca4937589a)

Changes to goexif from upstream:
- Add support for reading Nikon and Canon maker notes
- Adds parser registration to exif package
- Renamed cmd to exifstat
- Renamed exported fields and methods in goexif/tiff
- adds support for bare tiff images. bug fix and minor cosmetics
- support pulling the thumbnail
- adds thumbnail support to exif pkg
- tiff defines DataType and constants for the datatype values
- Update covnertVals and TypeCategory to use new constants for DataType
- Renamed test data dir in exif tests
- created type+constants for raw tiff tag data types

Not merged from upstream:
- ~1 MB of test JPGs in goexif/exif/samples

Minor changes in camlistore.org/pkg/* were neccessary to reflect the
name changes in the exported fields and methods.

Change-Id: I0fdcad2d7b5e01e0d4160a5eb52b8ec750d353cf
2014-09-05 08:36:42 +02:00
Aaron Boodman 3cdc4fec7a Merge "Adds less (css pre-processor) to fix missing CSS prefixes" 2014-09-05 01:27:28 +00:00
Lindsey Simon 9cb690bea8 Adds less (css pre-processor) to fix missing CSS prefixes
Change-Id: I47a90ff128abfeac00c886a3dece4ffa38c704c8
2014-09-04 17:32:01 -07:00
Aaron Boodman 2c3914dbc7 Add docs for share claim
Change-Id: I4acf68973922578e4d0b9cce0efb7a7c4b3efb8a
2014-09-04 13:46:57 -07:00
Aaron Boodman 6cca8d98d0 Merge "Add CORS header to share handler." 2014-09-04 20:26:34 +00:00
Brad Fitzpatrick ebda5dd9d7 server: gofmt ui.go
Lindsey, fix your editor. :)

Change-Id: I12ce560a7202a5aff98d18c42a49a096fd7893da
2014-09-04 08:30:59 -07:00
Brad Fitzpatrick 720ec2bd31 importer: fix start-up race
The status handler was trying to call methods on importer before
the *importer.Host had been initialized:

==================
WARNING: DATA RACE
Read by goroutine 31:
  camlistore.org/pkg/importer.(*importer).Accounts()
      /Users/bradfitz/src/camlistore.org/pkg/importer/importer.go:827 +0x344
  camlistore.org/pkg/importer.(*Host).AccountsStatus()
      /Users/bradfitz/src/camlistore.org/pkg/importer/importer.go:360 +0x1ea
  camlistore.org/pkg/server.(*StatusHandler).currentStatus()
      /Users/bradfitz/src/camlistore.org/pkg/server/status.go:166 +0x910
  camlistore.org/pkg/server.func·007()
      /Users/bradfitz/src/camlistore.org/pkg/server/status.go:73 +0x6e
Previous write by goroutine 15:
  camlistore.org/pkg/importer.(*Host).InitHandler()
      /Users/bradfitz/src/camlistore.org/pkg/importer/importer.go:402 +0x2d3
  camlistore.org/pkg/serverinit.(*Config).InstallHandlers()
      /Users/bradfitz/src/camlistore.org/pkg/serverinit/serverinit.go:583 +0x10b7
  camlistore.org/server/camlistored.Main()
      /Users/bradfitz/src/camlistore.org/server/camlistored/camlistored.go:427 +0xa94
Goroutine 31 (running) created at:
  camlistore.org/pkg/server.(*StatusHandler).InitHandler()
      /Users/bradfitz/src/camlistore.org/pkg/server/status.go:83 +0x207
  camlistore.org/pkg/serverinit.(*Config).InstallHandlers()
      /Users/bradfitz/src/camlistore.org/pkg/serverinit/serverinit.go:583 +0x10b7
  camlistore.org/server/camlistored.Main()
      /Users/bradfitz/src/camlistore.org/server/camlistored/camlistored.go:427 +0xa94
Goroutine 15 (running) created at:
  camlistore.org/server/camlistored.TestStarts()
      /Users/bradfitz/src/camlistore.org/server/camlistored/run_test.go:70 +0xc81
  testing.tRunner()
      /Users/bradfitz/go/src/pkg/testing/testing.go:427 +0x112

Change-Id: I5f4c7aaeb15f5b99e574a419c959b01e58b431af
2014-09-04 08:26:58 -07:00
Brad Fitzpatrick 108e16510f Merge "Adds less (CSS processor) v1.7.4 (Apache v2)" 2014-09-04 06:22:37 +00:00
Lindsey Simon ffb660b54d Adds less (CSS processor) v1.7.4 (Apache v2)
Change-Id: I364e46d395c0752df8c585a5aeab3aa9a289162c
2014-09-03 22:22:56 -07:00
Brad Fitzpatrick fa8edbdcd2 Merge "typo in doc for CreateHandler" 2014-09-04 01:10:28 +00:00
tnt aad1a492f1 Fatal -> Fatalf
Change-Id: I8930950b89eb1dc99c1567a0ae9cb3ef848ce67f
2014-09-04 00:43:57 +02:00
tnt 1a9aa07496 typo in doc for CreateHandler
Change-Id: I31b1f87c6ec9d24d518b2d0ebf6e0e4dd81e7ff6
2014-09-04 00:38:59 +02:00
Aaron Boodman 2acf13c702 Fix FOPA (flash of permanode aspect) when clicking permanodes with children.
We were waiting to show the container aspect until the child search session
returned. This meant that we temporarily showed the permanode aspect then
swapped to the container aspect shortly thereafter (because the container
aspect is higher priority).

The right solution to this is to change the search protocol so that clients
can know how many children a permanode has without getting the entire list
of them. But for now, it just so happens that search returns the entire
list, so we can use that instead.

Change-Id: Iaf93f689e965a1dc9ef73d834d46c57026c100cd
2014-09-03 00:30:46 -07:00
Aaron Boodman 6f91febae4 Don't trigger sad piggy when we purposely close sockets.
Change-Id: I2e3b0e894fff180a028fcffbf649ff18f69fd963
2014-09-03 00:07:08 -07:00
Aaron Boodman dbdda81230 Be defensive against getting null updates on web socket.
Change-Id: Id7a7b850ddaa57fe24d8e7ec380fb8881b27def5
2014-09-03 00:00:28 -07:00
Brad Fitzpatrick 37e38ad8b9 schema: add disabled failing ForeachChunk test
For blobkpacked.

Change-Id: Id133032aed21c9beeb17aff08d2500281ed3a5a2
2014-09-02 22:26:11 -07:00
Brad Fitzpatrick 8d7a7d9fcd Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2014-09-01 16:35:51 -07:00
Brad Fitzpatrick d63bcc04c2 blobpacked: export, document Manifest type. slight tweaks to zip format.
Change-Id: I9fbb005f0ecc6f013575255fae9e62b8397701ee
2014-09-01 16:25:37 -07:00
mpl b6448890dc Merge "test/integration: update TestWebsocketQuery" 2014-09-01 23:11:14 +00:00
mpl 638c660227 test/integration: update TestWebsocketQuery
Because 6517bbc392 introduced automatic
status update message upon connection.

Change-Id: I17383930d97b38d2a4abf7a646aa291bf84d7014
2014-09-02 00:57:23 +02:00
Brad Fitzpatrick d187f9b497 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2014-09-01 15:17:35 -07:00
Brad Fitzpatrick 59bd675e50 Add pinboard importer to unreleased 0.9 release notes
Change-Id: I29111d96f8f375a3ad3dff9c0086c96adfb0de5e
2014-09-01 15:16:24 -07:00
Brad Fitzpatrick 2368cf1c68 Merge "First version of pinboard importer." 2014-09-01 22:15:35 +00:00
Brad Fitzpatrick 76ce00022a C+A: add Gina White (Individual CLA)
Change-Id: Ie79f12a51bee2dc743b3cb9453d609c1e17fe8cd
2014-09-01 15:15:22 -07:00
Brad Fitzpatrick 5b1a327916 schema: fix test I broken when removing GetChunkOffsets
Rewrite in terms of ForeachChunk instead.

Change-Id: Id018420c47ffe116246c92b06c61a9dc9bbbee62
2014-09-01 15:11:14 -07:00
Gina White e9276c8ac8 First version of pinboard importer.
Change-Id: I2b06b20b2609981b03103b4a26c9c4633abe8473
2014-09-01 13:35:06 -07:00
mpl 9ddc3988aa Merge "serverinit: add support for memory-only indexer. Enable in devcam." 2014-09-01 18:02:53 +00:00
Aaron Boodman 698f3d0eea Add CORS header to share handler.
Change-Id: Ia12ea1f3de334e6f5b4d29c2a24291d263cc8068
2014-08-31 23:46:37 -07:00
Brad Fitzpatrick eca8ac0a3d Merge "blobserver: add memory storage, mainly for tests." 2014-08-31 03:35:01 +00:00
mpl 5036d6b456 blobserver: add memory storage, mainly for tests.
Mostly code moved from pkg/test/fetcher.go

Also got rid of blob.MemoryStorage

http://camlistore.org/issue/416

Change-Id: I8d17b80942e2fde1c869f71738bfb8a67ed8ab31
2014-08-31 01:17:52 +02:00
Brad Fitzpatrick a3ab848451 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2014-08-30 11:18:35 -07:00
Brad Fitzpatrick a563ba8d1f blobpacked: populate manifest.json chunks and verify them in tests
This switches to using a fork of archive/zip for now, until we can upstream
changes and wait a release cycle.

Plenty of TODOs remain, but making progress.

Change-Id: I3d7439dc405ba7f64b8d791952f1623a3a7e2024
2014-08-30 11:17:01 -07:00
Brad Fitzpatrick 9e0065fe8d zip: add LastDataOffset field to zip.Writer.
For blobpacked.

Change-Id: I23c325d945b0eff2ec8f47259bc527f1d4f6031f
2014-08-30 11:16:09 -07:00
Brad Fitzpatrick 2fbb26f9bb Checking in an unmodified copy of Go's archive/zip package.
The import is happening at hg 47de34bbcca1 but the last modification
of the package was at:

changeset:   20518:8ab50c14cd95
user:        Brad Fitzpatrick <bradfitz@golang.org>
date:        Mon Aug 04 16:12:55 2014 -0700
summary:     archive/zip: accept bogus trailing zeros in extras

... which will go into Go 1.4.

We don't need that fix in Camlistore, but rather we need additional
data about the state of the *zip.Writer while it's writing.  So this
is a pristine import in preparation for future changes so the changes
will be easy to see in the history, in case we want to move the
changes to the Go standard library (likely). But Go goes into 1.4
freeze in two days and it's the weekend.

Change-Id: I07117b7c5a7f58bd1e8abf6227c5fb0fa8047fa3
2014-08-30 09:55:13 -07:00
Aaron Boodman 95bf8e42a3 Get continual server status updates over websocket.
Also show channel-level websocket errors in piggy menu.

Change-Id: Iad5f00e5578f28f837cd2e88cbdab3f715a9fed1
2014-08-30 09:43:27 -07:00
Aaron Boodman 16085b9a4d Move importerRoot from status.json to discovery.
It seems to make more sense there...

Change-Id: I8450f1cb30ed331cfe15ddb49f55438c5bf9fd5b
2014-08-30 09:09:05 -07:00
Brad Fitzpatrick f66dccec08 Merge branch 'master' of https://camlistore.googlesource.com/camlistore 2014-08-30 00:13:19 -07:00
Brad Fitzpatrick 8d79f59908 blobpacked: first cut at packing, creating zip files with contiguous chunks.
See package docs in blobpacked.go for background & rationale.

Will minimize disk seeks, regardless of the underlying storage
system. I always knew this day would come, but we've been putting it
off. It's finally time to separate out logical (tons of small chunks)
storage from physical (big contiguous layout for how they'll be
accessed).

First 80% is done. Last 80% remains: mostly lots of tests and more
paranoia, and fsck things, for when we inevitably lose indexes. (but
each zip file is self-describing, so the index is redundant).

Change-Id: I13f9aa79c4d9d87b5fdcece0d99d97eba031d608
2014-08-30 00:09:10 -07:00
Aaron Boodman 40390c651a Adjust position of error icon.
Change-Id: I2b86d8b52a3c616ca3f2408973d85e70fb27a2d7
2014-08-29 20:55:52 -07:00
Aaron Boodman ef5e0dfc82 Show server errors in the piggy menu.
When there is a server error, the piggy becomes sad and periodically
shakes his head. When you expand the piggy menu you see one or more
entries explaining why he is sad.

Change-Id: If1e28dbf02c9befaa46b62838cb704220f719bc6
2014-08-29 16:08:19 -07:00
Brad Fitzpatrick f4740e77e8 Aggregate all errors with their URLS in status.json in one place.
Change-Id: If7a540fc3d1a6caa94b5f04d44de3207f15ac4cf
2014-08-29 16:03:44 -07:00
Brad Fitzpatrick 1be26ea3a4 schema: rewrite FileReader.LoadAllChunks, delete GetChunkOffsets
LoadAllChunks is now written in terms of ForeachChunk and is bounded.

GetChunkOffsets was never meant to be exported and was unused elsewhere.
It's now unused entirely.

Change-Id: I0653385fe805fda721e0b45ea0a17d9839c3ba6e
2014-08-29 15:26:36 -07:00
Brad Fitzpatrick 3388e363b9 schema: add FileReader.ForeachChunk
Will be used by the blobpacked storage server.

Also, we can then rewrite much of the legacy crufty methods on
FileReader in terms of this.

Change-Id: Ia371001f1ed502fa1da2dbc4a8e1c29e5f0c9e8c
2014-08-29 14:51:33 -07:00
Brad Fitzpatrick 6517bbc392 Send system status JSON on websockets upon connect and when it changes.
Change-Id: Ic0f211611d63b9d519ee3c5da0dd96c6b77459d5
2014-08-29 14:12:02 -07:00
Brad Fitzpatrick ee479c55a3 blobpacked: prefix meta rows with 'b:'
Change-Id: I203a0b1dac5f137415c586059c5d5e672b6359d8
2014-08-29 11:26:23 -07:00
Tamás Gulácsi 31dcddca41 Let Camlistore build on Solaris
The biggest obstacle is that syscall lacks Mkfifo on Solaris
(I've opened issue 8609 on Go for this).
Now I provide the same implementation for it as there is on Linux:
call Mknod.

github.com/cznic/fileutil needs IsEOF, so I've copied the plan9
(hole punching is stubbed out) implementation.

github.com/lib/pq: I've added solaris as a posix build tag.

Change-Id: Id667c488606a274849dc9b0af76abf1daeab5ad7
2014-08-28 14:26:11 +02:00
mpl bb49185463 serverinit: add support for memory-only indexer. Enable in devcam.
To avoid confusion, the previously named "memoryIndex" configuration
parameter, which actually meant to slurp the index into the memory
corpus, has been renamed to "copyIndexToMemory". "memoryIndex" is now
the parameter to enable a memory-only indexer.

http://camlistore.org/issue/416

Change-Id: I7026e5463fda3a1887a88e0b07d4e3431c60c8b7
2014-08-26 22:46:13 +02:00
mpl d7cc6a39b5 Merge "client: add test with multiple ignores in the config" 2014-08-26 14:02:01 +00:00
Brad Fitzpatrick 3eea26ce6c blobpacked: initial baby steps towards blob packing
Change-Id: Ied4d40513bc08311492b955eb8ad1beb5751df13
2014-08-25 21:18:18 -07:00