Skip to main content

Gold stars for good feed readers!

Over the past few weeks, we've been rolling out support for proper cache control on the Journals blogging platform, which will help reduce both bandwidth and database load, and also make things faster in general.  Last week we added ETag based cache control for both Atom and RSS feeds.  Today I took a quick peek at our logs to see how things are going...

The good news is that a lot of feed readers are being great citizens.  Around 33% of our feed requests get satisfied with 304 responses, meaning that clients only need to do a quick validation that they have the latest content, rather than fetching everything all over again.  Here's a quick list of feed readers, in no particular order, which are doing the right thing with our servers.  Gold stars for everybody!
  • Bloglines (http://www.bloglines.com)
  • FeedDemon/2.0
  • NewsGatorOnline/2.0
  • NetNewsWire/2.0
  • LiteFeeds/2.0
  • http://www.squeet.com
  • Firefox/1.5.0.7
  • Thunderbird/1.5.0.8
  • FacebookFeedParser/1.0 (UniversalFeedParser/4.1;) +http://facebook.com/
  • Windows-RSS-Platform/1.0
  • LiveJournal.com
  • Planet GBT +http://planetgbt.priyadi.net Planet/1.0~pre1-terasi+http://www.planetplanet.org UniversalFeedParser/3.3+http://feedparser.org/
    Google Desktop
Pretty clearly, the UniversalFeedParser library should get a lot of the credit.  It's great that the Windows RSS Platform is doing the right thing, considering the likely amount of traffic it's going to generate.

Surprisingly, we actually have a lower cache hit ratio for Atom feeds than RSS ones... mostly due to one major crawler that seems to prefer Atom feeds and never gets a 304, presumably because it's never sending an If-None-Match header:
  • Feedfetcher-Google; (+http://www.google.com/feedfetcher.html)
Hey Google: It would really help the stats if you guys could support ETags and If-None-Match.


Popular posts from this blog

Personal Web Discovery (aka Webfinger)

There's a particular discovery problem for open and distributed protocols such as OpenID, OAuth, Portable Contacts, Activity Streams, and OpenSocial.  It seems like a trivial problem, but it's one of the stumbling blocks that slows mass adoption.  We need to fix it.  So first, I'm going to name it:

The Personal Web Discovery Problem:  Given a person, how do I find out what services that person uses?
This does sound trivial, doesn't it?  And it is easy as long as you're service-centric; if you're building on top of social network X, there is no discovery problem, or at least only a trivial one that can be solved with proprietary APIs.  But what if you want to build on top of X,Y, and Z?  Well, you write code to make the user log in to each one so you can call those proprietary APIs... which means the user has to tell you their identity (and probably password) on each one... and the user has already clicked the Back button because this is complicated and annoying.

The problem with creation date metadata in PDF documents

Last night Rachel Maddow talked about an apparently fake NSA document "leaked" to her organization.  There's a lot of info there, I suggest you listen to the whole thing:

http://www.msnbc.com/rachel-maddow/watch/maddow-to-news-orgs-heads-up-for-hoaxes-985491523709

There's a lot to unpack there but it looks like somebody tried to fool MSNBC into running with a fake accusation based on faked NSA documents, apparently based on cloning the document the Intercept published back on 6/5/2017, which to all appearances was itself a real NSA document in PDF form.

I think the main thrust of this story is chilling and really important to get straight -- some person or persons unknown is sending forged PDFs to news organization(s), apparently trying to get them to run stories based on forged documents.  And I completely agree with Maddow that she was right to send up a "signal flare" to all the news organizations to look out for forgeries.  Really, really, really import…
Twister is interesting.  It's a decentralized "microblogging" system based on putting together existing protocols:  Bitcoin, distributed hash tables, and Bittorrent.  The most interesting part for me is using Bitcoin for user registration and spam control.  Federated systems handle this with federated trust, which is at least conceptually simple.  The Twister/Bitcoin mechanism looks intriguing though I don't know enough about Bitcoin to really comment.  Need to read further.