Nick's Blog

Blogging on App Engine, part 8: PubSubHubbub

Posted by Nick Johnson | Filed under tech, app-engine, coding, bloggart

This is part of a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.

Notice anything different? That's right, this blog has now been migrated to Bloggart - after all, if I won't run it, why should I expect anyone else to? ;)

Migrating comments to Disqus

In the previous post, I promised I'd cover migrating comments in today's post. Unfortunately, doing so proved to be both more complicated and less interesting than anticipated, so I'm going to instead provide an overview of the required steps. If you're really determined to see all the nitty-gritty, you can examine the change yourself.

Import of comments to disqus is through the disqus API. The API uses a straightforward RESTful model, with requests URL-encoded as either GET or POST requests, and responses returned as JSON strings. To make our lives easier, we'll define a straightforward wrapper function to make Disqus API calls:

def disqus_request(method, request_type=urlfetch.GET, **kwargs):
  kwargs['api_version'] = '1.1'
  if request_type == urlfetch.GET:
    url = "http://disqus.com/api/%s?%s" % (method, urllib.urlencode(kwargs))
    payload = None
  else:
    url = "http://disqus.com/api/%s ...

Blogging on App Engine, part 7: Migration

Posted by Nick Johnson | Filed under tech, app-engine, coding, bloggart

This is part of a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.

We're finally going to tackle (at least part of) that big bugbear of blogging systems: Migrating from the old system to the new one. In this post, we'll cover the necessary pre-requisites, briefly cover the theory of importing from a blogging system hosted outside App Engine, then go over a practical example of migrating from Bloog (since that's what this blog is hosted on).

Regenerating posts

Before we can write migration or import scripts, we need to improve (again) our dependency regeneration code. One thing that's probably occurred to you if you've been following this series is that there's currently no easy way to regenerate all the resources when something global changes such as the theme or the configuration. One could simply call .publish() on each blog post, but that would result in regenerating the common resources, such as the index and tags pages, over and over again - potentially hundreds of times. The same applies to migration: We could publish each new post as we process it, but this ...

Blogging on App Engine, part 6: Comments and Search

Posted by Nick Johnson | Filed under coding, app-engine, tech, bloggart

This is part of a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.

Today we're going to tackle two separate issues: Support for commenting on posts, and support for search. Commenting is fairly straightforward, so we'll deal with that first.

Commenting

Rather than implement our own comments system, we're going to take advantage of an existing 'SaaS' commenting offering, Disqus. Disqus provides simple drop in Javascript powered comment support, and has, by now, a rather impressive feature set, incorporating support for various login schemes - their own, OpenID, facebook connect, twitter, and others - as well as advanced functionality like finding and displaying 'reactions' from social sites around the web along with comments from users.

Integrating disqus support is straightforward. Since some people might not want to use it, or might want to use an alternate system, however, we're going to use a new config setting to ensure we only enable it if it's wanted. Add the following to the bottom of config.py:

# To use disqus for comments, set this to the 'short name' of the disqus forum
# created for the purpose.
disqus_forum = None ...

Blogging on App Engine, part 5: Tagging

Posted by Nick Johnson | Filed under coding, app-engine, tech, bloggart

This is part of a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.

Following on from our previous post, today we're going to deal with tagging. There are three components to adding tagging support to our blog:

Adding tags to the model and the post/edit interface.
Generating listing pages of posts with a given tag.
Adding tags and links to the listing pages on individual posts.

We'll tackle these in order. First, adding tags to the model and to the add/edit post interface. Add the following property immediately after 'body' on the BlogPost class (in models.py):

tags = db.StringListProperty()

That's it. No, really. Thanks to our use of ModelForms, our admin interface now has support for adding and editing posts with tags. Try it, if you wish. One slight caveat: The interface expects tags to be separated by newlines, rather than by commas. That's something we could address with a custom widget, at a later stage.

Next, the listing pages. Nearly all the functionality required to generate listings of posts with a given tag is identical to that required to generate ...

Blogging on App Engine, part 4: Listings

Posted by Nick Johnson | Filed under tech, app-engine, coding, bloggart

This is part of a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.

As you may have surmised from previous posts in the series, the 'static serving' approach we're using can lead to regenerating a lot of pages at once. For a long lived blog, with lots of history, regenerating the archive pages could take a significant amount of time - potentially long enough that we could run into the 30 second request deadline when updating or adding a post. Fortunately, however, we have something custom-made for the purpose: the Task Queue API. Using the Task Queue API, we can take care of the essential updates immediately - the post page itself, for example - then queue up other updates, such as the archive pages, on the task queue for later execution. Using the task queue has the extra advantage that updates can be executed in parallel.

Even better, we can make use of a new library in version 1.2.5 of the SDK, called 'deferred'. deferred is a clone of Ruby's delayed::job library, and makes it easy to enqueue function and method calls on the App ...

Blogging on App Engine, part 3: Dependencies

Posted by Nick Johnson | Filed under coding, app-engine, tech, bloggart

This is part of a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.

First, a couple of things of note. Between the last post and this one, I've snuck around behind your back and made a couple of minor changes. Don't worry, none of them are major. The most noticeable of these is that I've implemented a CSS design from the excellent site styleshout; our blog will now look halfway presentable. I've also refactored the existing admin code into a number of smaller modules; if you're browsing the source, you'll notice the code is now split between 'handlers.py' (the webapp.RequestHandlers), 'models.py' (the datastore models), and 'utils.py' (the utility functions such as those to generate content from templates).

I'm also pleased to announce that a couple of dedicated coders are following along with the series by writing their own ports of Bloggart. Sylvain is writing 'bloggartornado', a port of Bloggart to the Tornado framework, the source to which is here; a demo can be seen at http://bloggartornado.appspot.com/. Rodrigo Moraes is writing 'bloggartzeug', a port of ...

Blogging on App Engine Interlude: Editing and listing

Posted by Nick Johnson | Filed under coding, app-engine, tech, bloggart

This is part of a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.

A couple of things didn't quite make it into part 2 of the series: Listing and editing posts in the admin interface. This post is a short 'interlude' between the main posts in the series, and briefly covers the changes needed for those features.

Editing posts requires surprisingly little work, thanks to our use of the Django forms library. First, we write a decorator function that we can attach to methods that require an optional post ID, loading the relevant BlogPost object for us:

def with_post(fun):

def decorate(self, post_id=None):

post = None

if post_id:

post = BlogPost.get_by_id(int(post_id))

if not post:

self.error(404)

return

fun(self, post)

return decorate

Then, we enhance the PostHandler to take an optional post ID argument, using the decorator we just defined. Here's the new get() method:

@with_post

def get(self, post):

self.render_form(PostForm(instance=post))

If no post ID is supplied, post is None, and the form works as it used to. If a post ID is supplied, the post variable contains ...

Blogging on App Engine, part 2: Basic blogging

Posted by Nick Johnson | Filed under coding, app-engine, tech, bloggart

This is the second in a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.

In this post, we'll be handling the most basic part of blogging: Submitting new posts. For this, we're going to have to create our admin interface - which will involve a fair bit of boilerplate - as well as templates for both the admin interface and the blog posts themselves. But first, we need to make a slight change to the static serving code.

In order to publish new blog posts, we need to make sure we can generate a unique URL for the post, and for that we need a new method in the static serving interface. We'll call it 'add', and define it in static.py like so:

def add(path, body, content_type, **kwargs):

def _tx():

if StaticContent.get_by_key_name(path):

return None

return set(path, body, content_type, **kwargs)

return db.run_in_transaction(_tx)

add() is a fairly straightforward transactional wrapper for set(), which first checks if a resource with the provided URL path already exists, and only creates it if it doesn't, returning None otherwise.

Next, we need to add routing ...

Blogging on App Engine, part 1: Static serving

Posted by Nick Johnson | Filed under coding, app-engine, tech, bloggart

This is the first in a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.

As promised, today we'll be covering the static serving component of our blog-to-be. First, though, is the naming issue. There were a lot of good names proposed by readers. Unfortunately, pretty much every one of them is taken on appspot.com. In the end, I settled on a name suggested to me out-of-band: 'bloggart'. My wife, who loves to draw, has kindly promised to draw me a picture of a boastful looking monster to act as a mascot.

Now down to business. Create a new app called 'bloggart-demo' (or whatever you wish, really), and put the following in its app.yaml file:

application: bloggart-demo

version: live

runtime: python

api_version: 1

handlers:

- url: /remote_api

script: $PYTHON_LIB/google/appengine/ext/remote_api/handler.py

- url: /.*

script: static.py

Note that we're including remote_api right away. Without any sort of admin interface at this stage, it's going to be our only way of creating some initial content to test things out with. As discussed in the introductory post, we're separating out the ...

Win a Dell Netbook and $1000 of App Engine credit!

Posted by Nick Johnson | Filed under coding, app-engine, tech

Just a quick note: We recently announced a competition on the App Engine blog: Develop an App Engine app that uses Twilio, and you could win a Dell Netbook and $1000 of App Engine credit! Twilio is an excellent service that lets you write sophisticated telephony applications, entirely over HTTP.

You have until October 4th to enter. I really wish I could, but as part of the team, I'm disqualified. ;)

Migrating comments to Disqus

Regenerating posts

Commenting

Blogroll