A modest proposal: RESTful OpenSocial API

Yesterday, I sent a draft proposal for a RESTful API out for comment. It pulls together a lot of the things I've been working on lately: OAuth, batching, and patching. It also has a different take on JSON vs. Atom data formats. Since OpenSocial has several different kinds of data -- activities, people, and app data to start with -- a one-size-fits-all format is likely to be a poor fit for everybody. So, there's a custom mapping, expressed through metadata, that can be different for each kind of data you deal with.

I'd Rather Batch than Fight

We need HTTP batching. Or at least some people do, in some situations. I think that a small optimization here can avoid re-inventing a lot of wheels.

To back up a bit, the problem I want to solve is just the one of minimizing round trips for REST based services such as AtomPub. Not atomic transactions, nor additional semantics; just avoiding the latency incurred by multiple round trips. The motivation is basically the same as for HTTP Pipelining.

The ideal solution would work for all types of requests (not just Atom, or JSON, or ...). Batching is orthogonal to all of these and with the use of mashups it's quite possible for a client to be using any or all of them. Batching should work really well within browsers but also be usable with any HTTP libraries. And, it should be auto-discoverable, so a client can fall back to individual requests if a server doesn't support batching.

My canonical motivating example is a smart client that is doing an update and then a retrieval of two different but interdependent resources. The client knows that the update is almost certain to trigger a need for a refresh of the second resource, and you need to ensure that the operations are performed in that order due to the dependency. It turns out to be very difficult to model this as a single REST operation, and it's really two logical operations anyway.

Proposal: Take James' multipart-MIME batch proposal, but use POST instead of PATCH. (I agree with Bill de hÓra that PATCH is a stretch; if we had a BATCH method that'd be fine, but we don't right now.) Make the semantics exactly as if the requests had been sent, in order, from the same client on the same keep-alive connection. Do not provide any atomicity guarantees. Provide an easy way for a client to determine if a server supports batching. If a server does not, then a client has a trivial for loop fallback which won't change the semantics of the request at all.

Q: Why drag in MIME? Why not just use XML/JSON/...?
A: Because you don't want to mandate a particular parser, or even a text based format. Image uploads, for example, should work fine with this scheme. It's generic. Note that we are in fact tunnelling HTTP requests here. I'd much rather be up front and open about this than try to hide the fact by smuggling it inside some other syntax.

Q: But MIME is ugly and not supported by my language!
A: We're not trying to interoperate with mail or Usenet here, just slice up a body with a separator that is a sequence of bytes (--batch-34343434) and apply some simple mapping rules for things like method and URL path.

Q: Do we really need this?
A: If you think you do, you can implement it. If you don't, you can skip it. You'll interoperate either way. Let the market decide what's best.

OpenSocial 0.7 and makeRequest

We're converging towards 1.0! There's one particular thing I want to quickly highlight: makeRequest. This goes beyond the old IG_Fetch API to allow arbitrary HTTP requests to arbitrary URLs, with full use of headers, POST data, response codes, etc. This effectively means that properly installed gadgets can talk any protocol to any server on the Internet. Now that's open.

There are controls of course. The container validates that the request is coming from a properly installed gadget, and poorly behaving gadgets can be rate limited or shut off if necessary.

You can also pass certain headers which are awfully useful. For example...

Authorization: OAuth realm="http://sp.example.com/",
             oauth_consumer_key="0685bd9184jfhq22",
             oauth_token="ad180jjd733klru7",
             oauth_signature_method="HMAC-SHA1",
             oauth_signature="wOJIO9A2W5mFwDgiDvZbTSMK%2FPY%3D",
             oauth_timestamp="137131200",
             oauth_nonce="4572616e48616d6d65724c61686176",
             oauth_version="1.0"

Which would let you do authenticated cross-domain requests.

(This assumes of course that the gadget can securely store a token for later use. Gadgets can store data securely using the OpenSocial APIs, but since the user at the browser ultimately has full control over the client side environment, this is effectively the same as a client without a very secure secret. A server side signing solution is needed if you want to to beyond more than simple scenarios involving a user looking at their own data. We're already using OAuth with an empty token to let gadgets talk to their home servers securely, so adding this won't be difficult.)