Breaking things down to Atoms

Over the last few years, more and more sites have been providing RSS and Atom feeds. This has been a huge boon both for keeping up to date with content through feed readers, and for programmatically consuming data. There are, inevitably, a few holdouts, though. Notable amongst those holdouts -and particularly relevant to me at the moment - are property listing sites. Few, if any property listing sites provide any sort of Atom or RSS feed for listings, let alone a API of any kind.

To address this, I decided to put together a service for turning other types of notification into Atom feeds. I wanted to make the service general, so it can support many different formats, but the initial target will be email, since most property sites offer email notifications. The requirements for an email to Atom gateway are fairly straightforward:

  • Incoming email should be converted to entries in an Atom feed in such a fashion as to be easily interpretable in a feed reader
  • As much of the original message and metadata as possible should be preserved in the Atom feed entry
  • It should be possible to access the original message if desired

In addition, I had a ...

Hello, developers.

Look at your app. Now back to me. Now back at your app. Now back to me.

Sadly, your app wasn't written by me. But if you used appstats, your app could be fast like mine.

Look down. Back up. Where are you? In your admin console, with the app your app could run like. What's in your hand? Back to me. It's a link from that site you love. Look again. Your site is now doing 20QPS.

Anything is possible when your app runs on App Engine and not PHP. I'm on a Python.

Anyone know a good voice actor?

Modeling relationships in App Engine

One source of difficulty for people who are used to relational databases - and certain ORMs in particular - is how to handle references and relationships on App Engine. There's two basic questions here: First, what does a relationship entail, in any database system? And second, how do we use them in App Engine?

The nature of relationships

Many ORMs expose multiple 'types' of relationships as first-class entities - one-to-one, one-to-many, and many-to-many. This obscures the fact that, in reality, they are all built on the same building block, that of references. A reference is simply a field of an entity that contains the key of another entity - for example, if a Pet references an Owner, that simply means the Pet has a field that contains the key of its owner.

All relationship types simply devolve to references. A one-to-many relationship is a reference in its simplest form - each Pet has one Owner, and therefore an Owner can have multiple pets, all pointing to it. The Owner is not modified; it relies on the individual Pets naming it as the owner. One-to-one relationships are one-to-many relationships with the additional constraint that there will be only one Pet referencing each Owner; this is ...