Commit Graph

6007 Commits

Author SHA1 Message Date
Brad Fitzpatrick b62f3b399b Merge "search: remove broken websocket status optimization." 2016-04-26 16:47:52 +00:00
Brad Fitzpatrick f7e40292cb camweb: redirect /doc/dir to /doc/dir/ with trailing slash
Also fix up non-canonical links in docs after rename of /docs/ to /doc/

Updates #732

Change-Id: I7131f57cc7f8edf150cc785f42fe48917835c361
2016-04-26 09:46:07 -07:00
Will Norris 0d77ec13cb Merge "convert protocol docs to markdown" 2016-04-26 16:39:07 +00:00
Will Norris 974f810b4b convert protocol docs to markdown
Update links to reference the website URLs rather than gerrit.  There is
some more cleanup that could be done with the docs (for example, using
tables for the request and response keys), but this is focused on just
basic rendering.

Git is treating these as file deletions and additions... apparently
there were too many changes to treat them as renames, though that's what
they really are.

Refs #720

Change-Id: Ibca17b78e1412c049843236bfbc8f2a9e368bb23
2016-04-26 09:37:01 -07:00
Brad Fitzpatrick 7d326336cf search: remove broken websocket status optimization.
It was breaking the websocket integration test. We still need to
compute the last status message even if nobody is currently connected,
because when a websocket connection does arrive, we need to send the
last commputed status message.

Change-Id: Icea0fcc69941aa6249d32f0a4ef4ec03bdfb1e20
2016-04-26 09:29:39 -07:00
mpl 135f0373d2 vendor: add golang.org/x/sys/unix
Needed by bazil.org/fuse , it should have been added with
e0a9699c63

rev 8fdfb00a6a1add0f7a145480e3729a5427a74375

Fixes #730

Change-Id: I2f9c7aa21a5ead82d7043392e4a55baf3cecffe6
2016-04-26 08:57:58 -07:00
Will Norris fc619464df website: serve godocs for /app/ packages
Change-Id: If2836840aa5523e74a734051111e0ed614ea200e
2016-04-25 22:40:17 -07:00
Mathieu Lonjaret 7df8dbb2db Merge "website: always remove old demoblob3179 container" 2016-04-26 05:21:34 +00:00
Mathieu Lonjaret d422b6e942 Merge "website: serve /doc/ URLs from top-level doc dir" 2016-04-26 05:17:54 +00:00
Will Norris 7cb074bfde website: serve /doc/ URLs from top-level doc dir
- move existing content from website/content/docs to top-level doc
  directory
- redirect /docs/ URLs to /doc/

Fixes #720

Change-Id: I92e26bbc1fdbe994089080ae3a971d85c12921ab
2016-04-25 22:13:09 -07:00
Stephen Searles fa5607c861 A+C: add Stephen Searles (Individual CLA)
Change-Id: I6b6a39545d059c30d7640bab298a2386c48e8b16
2016-04-25 18:45:53 -07:00
Brad Fitzpatrick 858504d793 Merge "server/camnetdns: set authority flag on dns response" 2016-04-26 01:08:24 +00:00
Jeromy 444b2a81f3 server/camnetdns: set authority flag on dns response
Updates #722

Change-Id: I18277a0181234e9a9e497100a5f86613fca2dba4
2016-04-25 18:07:52 -07:00
Mathieu Lonjaret e0d422768d Merge "website: refactor mainHandler to allow diff roots" 2016-04-26 01:02:23 +00:00
Will Norris 6196e6ab98 website: refactor mainHandler to allow diff roots
This helps set up serving /docs/ requests out of different file root.
Additional, this allows the actual files on disk used to satisfy
requests to contain a .md or .html file extension (basically simulating
Apache multiviews).  Finally, requests that map to directories can be
satisfied by either a index.html or README.md file.

Ref #720

Change-Id: Id6076c242edd74b99a12699cd58c4567973d0d42
2016-04-25 18:01:27 -07:00
Brad Fitzpatrick 3c5b519c1e Merge "website: redirect to https" 2016-04-26 00:40:22 +00:00
Mathieu Lonjaret 8092071278 Merge "website: do a full cloning of camlistore.org on startup" 2016-04-26 00:35:18 +00:00
mpl d9fa1ed87e website: do a full cloning of camlistore.org on startup
We used to have --depth=1 when cloning camlistore.org on startup, for
efficiency reasons.
But not having all the commits locally caused problems when pushing to
github, because it would see the remote had commits that we didn't.
So we're now doing a full cloning.

It looks like it's now taking ~4s (as opposed to ~2.5s before), and
consuming ~50MB diskspace (as opposed to 35MB before).

Plus a few bugfixes.

Change-Id: If7dbae1d3119d8b3336fb7d735f6bd0ba7606fc6
2016-04-25 17:30:17 -07:00
Jeromy d77d4b8b51 server/camnetdns: log all incoming requests
Updates #722

Change-Id: Ief6b172d8ec4e62f22332a117f379ea4998a3f76
2016-04-25 16:53:50 -07:00
Jeromy e53f91085b server/camnetdns: default to port 53 on gce
Updates #722

Change-Id: I4d435634b6c78a4d4d1392616870d50f0c458e5c
2016-04-25 16:38:04 -07:00
Jeromy 4841ce270f server/camnetdns: add cloudlaunch config
Updates #722

Change-Id: Ia7532118183b5c38dfedc3aaa437f5284301b4c9
2016-04-25 16:26:26 -07:00
Brad Fitzpatrick 15ad9bf381 Merge "server/camnetdns: start of camlistore.net DNS server" 2016-04-25 23:08:34 +00:00
Jeromy 21c5373a0b server/camnetdns: start of camlistore.net DNS server
Updates #722

Change-Id: Ie4c2d6d036f13d7fb6f5545131397f55d1d9a91c
2016-04-25 16:05:25 -07:00
Mathieu Lonjaret 7332aad3aa Merge "server/camlistored: upload gpg key blob on startup" 2016-04-25 22:43:38 +00:00
mpl 36944ca818 server/camlistored: upload gpg key blob on startup
Fixes Issue #700

Change-Id: Ica92c321666220043e9c157de38e67a043250afe
2016-04-25 15:42:04 -07:00
Brad Fitzpatrick 64ecb4bb18 Merge "importer: make RunContext.Context be a read-only accessor" 2016-04-25 22:17:48 +00:00
Brad Fitzpatrick 8a2f1f4a65 website: redirect to https
Fixes #721

Change-Id: Ib75e1023eb56afc6c15e826bc0d26ff6b3c7a859
2016-04-25 15:12:08 -07:00
Mathieu Lonjaret 41b12ba78d Merge "website: fix ssh for git push to github" 2016-04-25 21:52:22 +00:00
Brad Fitzpatrick 382794c6ba importer: make RunContext.Context be a read-only accessor
The Picasa importer was deadlocking inside the golang.org/x/oauth2
package because the Picasa code was mutating its parent's context and
accidentally creating a cycle of *oauth2.reuseTokenSource values
(double-acquisition of the reuseTokenSource's sync.Mutex was the
deadlock)

The Picasa code really just wanted to create a subcontext from the
base context, where the subcontext did the oauth2 wrapping, but then
picasa was storing that subcontext in the
camlistore.org/pkg/importer-provided mutable "RunContext" structure's
public, embedded "context.Context" field. Unfortunately, the picasa
importer code had already used that *RunContext (pre-mutation) as the
context.Context for the base HTTP client.

This change changes the camlistore.org/pkg/importer.RunContext to not
have a public embedded Context field. Instead, it only has a read-only
accessor to get at the base context, similar to Go 1.7's
net/http.Request.Context() field. This will prevent this class of
error in the future.

Fixes #698

Change-Id: I9114b64aa1bfd7ea83b36ea1ab18ed88b4a44f7e
2016-04-25 14:30:39 -07:00
mpl fe8372e117 website: fix ssh for git push to github
issue #708

Change-Id: Ib01f6c378924becb03173724c452d42907b995e8
2016-04-25 14:00:10 -07:00
Brad Fitzpatrick 40ed14d5dc importer/picasa: simplify, consolidate to single file
Removes some unnecessary indirections and types.
2016-04-25 11:46:50 -07:00
mpl 2766655a24 Test commit for github syncing
Change-Id: I7232c8735bde2a7155e8cb91370ffde5d0ef56a9
2016-04-25 09:42:19 -07:00
Mathieu Lonjaret 36d9d8d7eb Merge "website: sync commits to github" 2016-04-25 16:25:56 +00:00
mpl 43f2e48dd0 website: sync commits to github
Fixes issue #708

Change-Id: I0e081724134ab08c736b00bd2ebe81eb9028ace7
2016-04-24 22:28:16 -07:00
Brad Fitzpatrick d281c3e65a importer: add long-polling support. make Twitter updates real-time.
(uses Twitter user stream API)

No more 30 minute waits. Better for live demos. :D

Change-Id: I29860093f75bb86faee7f0b8949be7b7023f00cc
2016-04-24 11:34:33 -07:00
Brad Fitzpatrick db912bbb94 cmd/camtool: add basic describe --at flag
The TODO for more git-like time styles remains.

Change-Id: Iddba599644711e0cabf7bf85ee39150f25e934e6
2016-04-24 08:46:40 -07:00
Brad Fitzpatrick 62d3088d6a importer: make the client secret be input type=password
So it doesn't show up in screencasts.

Change-Id: I3b677fa36a77976d1f315e5848485bf5e59e72ea
2016-04-24 08:39:15 -07:00
Brad Fitzpatrick 924c895b22 server/camlistored/ui: rename confusing Pudgy menu to Camli
Change-Id: Ic7785881f2db65aff562ef329753d78aa131e68b
2016-04-23 17:42:56 -07:00
Will Norris f82dc010bc cmd/camtool: output server-config.json string
In addition to outputting the JSON config needed for cloudstorage_test,
also provide the colon-delimited string needed for server-config.json.

Also update URL for Google API Console

Fixes #715

Change-Id: I3784ad3442bcf0c99cb738a3beed7b9e5fbcac68
2016-04-23 15:04:09 -07:00
Mathieu Lonjaret 0c355a18cc Merge "misc/docker: build ignore for standalone programs" 2016-04-22 23:18:02 +00:00
mpl b43cfe22b9 misc/docker: build ignore for standalone programs
Fixes #713

Change-Id: I964939a3d5de6eb0f3ab0321fd7447774751c92d
2016-04-22 16:15:04 -07:00
Brad Fitzpatrick 359ac93fed importer: cache accounts list
Fixes non-stop queries burning CPU in the background.

Fixes #519

Change-Id: I16d7e6d83735acd8e2f96045a9d84dfe06207348
2016-04-22 16:00:25 -07:00
Brad Fitzpatrick 03253d1d3c Merge "Fix deadlock in search/index." 2016-04-22 22:01:45 +00:00
Brad Fitzpatrick e93e4f3822 Fix deadlock in search/index.
The describe requests were launching a storm of RLocks which weren't
safe in the presence of goroutines trying to acquire write locks.

Instead, make the corpus locking the responsibility of the caller and
add Lock/Unlock/RLock/RUnlock methods to the index and move locking up
a level.

This also adds a fair bit of context.Context plumbing which was used
in earlier debugging.

Fixes camlistore/camlistore#709

Change-Id: I8d7254d1e1da541f8c080d62f5408aac807fd3b1
2016-04-22 14:57:10 -07:00
mpl d2f50293cd misc/docker: rm third_party in zip-source
Change-Id: Ica0c61e164390a19944a0b3448b8468761f58402
2016-04-21 15:09:03 -07:00
mpl 07c0c9c797 misc/docker: fix third_party move for docker image
Change-Id: Id8146b3555c2758871e6d13474595165bee646a2
2016-04-21 14:58:19 -07:00
mpl 63d6e384bb path fixes for third_party move
A few more left in in misc/docker that I'll do separately

Related: 75d60962f6

Change-Id: Id5f6a3729e33aca97f8664ca8ef91afc64461891
2016-04-21 12:25:17 -07:00
mpl 8580b811cf pkg/index: make corpus RLock take a context
Also remove extra (deadlocking) RLock in pkg/search.

Updates issue #709

Change-Id: I556a1fbf9217f482b6a51e74c28a019dea8369a2
2016-04-21 11:28:24 -07:00
Brad Fitzpatrick c2e4fc7de6 blobserver/blobpacked: fix TestReindex at Go master
Fixes #710

Change-Id: I0c58900fc566812e1a48c04d3bfba5060c0c4160
2016-04-20 18:51:07 -07:00
Brad Fitzpatrick 290569f9ae blobserver/blobpacked: make TestPackLarge work with Go tip
compress/flate changed at tip. compressed zips can have different
digests now.

Updates camlistore/camlistore#710

Change-Id: Ida572ca88fe4729c57772db85f69876d28c3742f
2016-04-20 17:48:01 -07:00