Skip to main content


Showing posts from 2010

Track Built on Open Specifications

The Google Buzz team just rolled out the Track feature, which lets you subscribe to real time updates for search terms via PubSubHubbub.   It's pretty awesome, and builds on top of the work we've done for the Buzz Firehose.  A search query is just like any other URL -- say, -- and you can subscribe to it with PubSubHubbub using existing code.  In fact that URL will work in any PubSubHubbub-enabled client (like Google Reader).

Experimental Magic Signatures Draft

This morning, I put up an experimental Magic Signatures draft for comment.  This is a fairly extensive reorganization of the spec, so it would eventually become an -01 version of the Magic Signatures specification.  It is still provisional and changing and it really needs feedback (including implementation feedback).  It adds two optional pieces, and solves a bunch of unspecified things, and is organized so as to make it easier to use as a building block in specifications other than Salmon.
The major additions are: HMAC private key signaturesAbility for signer URLs to provide signer keys on GET directlyA JSON based format for signer keys The major (possibly breaking) change from the prior version is that public keys retrieved via XRD are retrieved via XRD Property elements rather than Link (since all current applications are using inline keys in any case, and this is more parallel to the new JSON format).
Looking forward to feedback on this experiment.  If it's successful, we'll …

We Built this Firehose on Open Specs

Last week, we launched the Buzz Firehose feed.  You can see it in action at the sample Buzz Mood app (code). One thing which we didn't highlight too much (though DeWitt did) is that this is completely built on non-proprietary protocols and formats: Atom, PubSubHubbub, and Activity Streams are able to deliver real time push updates easily and quickly.

We did do some special things on the supply side for efficiency.  If you GET the firehose feed you'll see that it never contains any updates, just feed metadata and the hub link.  Conceptually, it contains the last 0 seconds of updates.  This is primarily because it doesn't make sense to poll this feed, so it exists only to provide a subscription mechanism.  Internally, for efficiency we're pushing new updates to the hub with full data rather than pinging the hub and waiting for it to poll the feed.  But, we're using the standard Atom format to push the data. The hub is doing coalescing of updates so when a bunch come…

XAuth is a Lot Like Democracy

XAuth is a lot like democracy:  The worst form of user identity prefs, except for all those others that have been tried (apologies to Churchill).  I've just read Eran's rather overblown "XAuth - a Terrible, Horrible, No Good, Very Bad Idea", and I see that the same objections are being tossed around; I'm going to rebut them here to save time in the future.

Let's take this from the top.  XAuth is a proposal to let browsers remember that sites have registered themselves as a user's identity provider and let other sites know if the user has a session at that site.  In other words, it has the same information as proprietary solutions that already exist, except that it works across multiple identity providers.  It means that when you go to a new website, it doesn't have to ask you what your preferred services are, it can just look them up.  Note that this only tells the site that you have an account with Google or Yahoo or Facebook or Twitter, not what the…

Google I/O, Salmon, and the Open Web

Last week was intense, and I'm just now coming up from air from the backlog.  We launched a huge expansion to the Buzz API at Google I/O.  My contribution was to ensure that PubSubHubbub real time updates flow for the new feed URLs as well as the older ones; as part of this, we also enabled "fat pings" to the hub from Buzz.  "Fat pings" in this case means that we're doing an active push of each full update from our back end systems through to the hub, so the hub never needs to call back to the feeds to retrieve updates.  This is a more complicated approach, but reduces the overall server load and makes consistency guarantees easier for globally distributed systems.

Just before Google I/O, I went to IIW10 and talked about Salmon, LRDD, Webfinger, and OpenID Connect.  The air at IIW was thick with new specifications and updates.  I would've liked to have participated more, but I was helping to roll out PubSubHubbub for Buzz and getting ready for the "…

Salmon Protocol RFC draft Now Available

For the past couple of weeks, I've been banging on code and xml to work out a coherent draft spec for the Salmon Protocol. Although Salmon is intended to primarily be a glue protocol, tying together a bunch of disparate existing pieces into an understandable and coherent framework turns out to be a lot of work. Now starts a slog through a bunch of issues (including a backlog on Magic Signatures which need to be addressed) and updating the demo code and libraries to stay in sync with the spec.
Please join the salmon-protocol group if you're interested in working on the Salmon specification and/or writing code that implements it.
In case you're counting, Salmon references 10 other specifications in addition to Magic Signatures.

Webfinger now available for Google public profiles

If you've seen ReadWriteWeb's "Email as Identity", you know that Brad just turned on Webfinger for anyone with a public Google profile.  What this means is that, if (and only if) you've enabled your public Google profile at{username}, you can now use Webfinger to look up information you've added to your profile.  Here's the one for john.panzer for example.  By default this exposes only generic service endpoints, not actual information about the user.  But you can link it to private or public feeds, services, or basically anything you want to associate with your online identity.

A next step for Salmon is to let users add a link to their public key from their Webfinger discovered info.  Then given an email address it will be trivial to look up their preferred public key.  It'd look something like this via the discovery tool:

links { rel: "magic-public-key" type: "application/magic-public-key" href:…

Google Buzz and Salmon

You may have heard of a small launch this morning:  Google Buzz is being rolled out to all GMail users right now.

There's an obvious connection to the Salmon Protocol; DeWitt just put up a blog post giving the big picture for Buzz and open protocols on the Social Web blog, and technical details on the Google Code blog from Brian Stoler. Most importantly, there's a Labs Buzz API.  Here's the take-away:
We'd like to take this opportunity to invite developers to join us as we prepare the Google Buzz API for public launch. Our goal is to help create a more social web for everyone, so our plan for the Buzz API is a bit unconventional: we'd like to finalize this work out in the open, and we ask for your participation. By building the Google Buzz API exclusively around freely available and open protocols rather than by inventing new proprietary technologies, we believe that we can work together to build a foundation for generations of sites to come. We're ready to open…

Magic Signatures

Just wrote up a detailed Internet-Draft for the Salmon signature mechanism; since it's an outgrowth of the "magic security pixie dust" for Salmon I'm calling it Magic Signatures for now. It approximately matches the Magic Sig demo as well. Feedback welcomed!
(Updated: Just changed the text at as well, hopefully it makes a bit more sense now.)

Salmon's 'Discoverable URIs' and PKI

One issue that's cropped up in Salmon is how to identify authors.  Mostly, feed syndication protocols hand-wave about this and at most specify a field for an email address.  Atom goes one step further and provides a way to specify a URI for an author, which could be a web site or profile page.

The main things that Salmon needs are (1) a stable, user-controlled identifier that can be used to correlate messages and (2) a way to jump from that identifier to a public key used for signing salmon.  The existing atom:author/uri element works fine for #1.  Webfinger+LRDD discovery can then take over for #2.  What comes out of that process is a public key suitable for verifying the provenance (authorship) of a salmon.

Now for the details.  It turns out that the flow for Webfinger is pretty stable, so any time you have an email like identifier for an author you can just slap "acct:" in front and things will Just Work.  But I don't want to limit authorship only to acct: URIs - …

A Web Wide Public Key Infrastructure

In my last post, I talked about Magic Signatures, the evolution of Salmon's Magic Security Pixie Dust into something concrete.  There's an important bit missing, which I'll talk about now:  Public key signatures are fairly useless without a way to discover and use other people's public keys in a secure, reliable way.  We need a distributed, easily deployable public key infrastructure.  Let's build one.

The basic plan for Salmon signature verification is fairly simple:  Use Webfinger discovery on the author of the Salmon to find their Magic Signature public key, if they have one.  To find this, you first get the Webfinger XRD file, then look for a "magickey" link:
<Link rel="" href="" />
Retrieve the magickey resource, parse it into a public key in your favorite library, and use that to verify the signature of the original Salmon you started with.

A few…

Magic Signatures for Salmon

In writing the spec for Salmon we soon discovered that what we really wanted was S/MIME signatures for the Web.  In other words, given a message, let you sign it with a private key, and let receivers verify the signature using the corresponding public key.  Signing and verifying are pretty well understood, but in practice canonicalizing data and signing is hard to get right.  Making sure that the mechanism adopted is really deployable and interoperable, even in restricted environments, is a top priority for Salmon.

I'm calling this the "Magic Signature" mechanism because it's not really Salmon-specific and you can analyze it without thinking much about Salmon at all.

One of the reasons why this is hard is because of the abstraction layers that we have in place in our software.  For example, encryption algorithms operate on byte sequences, but a given XML document can have many different byte sequence serialized forms.  Even JSON isn't immune to this, though mandati…

Fork in the Road for Salmon

Happy New Year!
Over the past several weeks, I've been doing a Salmon conference roadshow, talking and listening to people about the protocol and getting feedback.
At IIW, we had a "Magic Security Pixie Dust" Salmon session which kicked around the use cases and challenges in detail.  There and elsewhere I got basically two kinds of feedback:  (a) The specification, especially the signatures, was too complicated; and (b) the specification, especially the signatures, was not comprehensive enough.
There was a suggestion to drop signatures entirely and just rely on reputation of the salmon generators, who would be on the hook for vouching for the identity of their users.  This would simplify the protocol but at the cost of giving it effectively the same security characteristics as email.  This was initially attractive and I spent some time playing with what that would look like.  Unfortunately, I am pretty sure that it would just make operation more complex and adoption more pr…