Nick's Blog

Blogging on App Engine Interlude: Editing and listing

Posted by Nick Johnson | Filed under coding, app-engine, tech, bloggart

This is part of a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.

A couple of things didn't quite make it into part 2 of the series: Listing and editing posts in the admin interface. This post is a short 'interlude' between the main posts in the series, and briefly covers the changes needed for those features.

Editing posts requires surprisingly little work, thanks to our use of the Django forms library. First, we write a decorator function that we can attach to methods that require an optional post ID, loading the relevant BlogPost object for us:

def with_post(fun):

def decorate(self, post_id=None):

post = None

if post_id:

post = BlogPost.get_by_id(int(post_id))

if not post:

self.error(404)

return

fun(self, post)

return decorate

Then, we enhance the PostHandler to take an optional post ID argument, using the decorator we just defined. Here's the new get() method:

@with_post

def get(self, post):

self.render_form(PostForm(instance=post))

If no post ID is supplied, post is None, and the form works as it used to. If a post ID is supplied, the post variable contains ...

Blogging on App Engine, part 2: Basic blogging

Posted by Nick Johnson | Filed under coding, app-engine, tech, bloggart

This is the second in a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.

In this post, we'll be handling the most basic part of blogging: Submitting new posts. For this, we're going to have to create our admin interface - which will involve a fair bit of boilerplate - as well as templates for both the admin interface and the blog posts themselves. But first, we need to make a slight change to the static serving code.

In order to publish new blog posts, we need to make sure we can generate a unique URL for the post, and for that we need a new method in the static serving interface. We'll call it 'add', and define it in static.py like so:

def add(path, body, content_type, **kwargs):

def _tx():

if StaticContent.get_by_key_name(path):

return None

return set(path, body, content_type, **kwargs)

return db.run_in_transaction(_tx)

add() is a fairly straightforward transactional wrapper for set(), which first checks if a resource with the provided URL path already exists, and only creates it if it doesn't, returning None otherwise.

Next, we need to add routing ...

Blogging on App Engine, part 1: Static serving

Posted by Nick Johnson | Filed under coding, app-engine, tech, bloggart

This is the first in a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.

As promised, today we'll be covering the static serving component of our blog-to-be. First, though, is the naming issue. There were a lot of good names proposed by readers. Unfortunately, pretty much every one of them is taken on appspot.com. In the end, I settled on a name suggested to me out-of-band: 'bloggart'. My wife, who loves to draw, has kindly promised to draw me a picture of a boastful looking monster to act as a mascot.

Now down to business. Create a new app called 'bloggart-demo' (or whatever you wish, really), and put the following in its app.yaml file:

application: bloggart-demo

version: live

runtime: python

api_version: 1

handlers:

- url: /remote_api

script: $PYTHON_LIB/google/appengine/ext/remote_api/handler.py

- url: /.*

script: static.py

Note that we're including remote_api right away. Without any sort of admin interface at this stage, it's going to be our only way of creating some initial content to test things out with. As discussed in the introductory post, we're separating out the ...

Win a Dell Netbook and $1000 of App Engine credit!

Posted by Nick Johnson | Filed under coding, app-engine, tech

Just a quick note: We recently announced a competition on the App Engine blog: Develop an App Engine app that uses Twilio, and you could win a Dell Netbook and $1000 of App Engine credit! Twilio is an excellent service that lets you write sophisticated telephony applications, entirely over HTTP.

You have until October 4th to enter. I really wish I could, but as part of the team, I'm disqualified. ;)

Writing a blog system on App Engine

Posted by Nick Johnson | Filed under coding, app-engine, tech, bloggart

I'm going to spend the next few posts working through the process of writing a simple, robust and scalable blogging system for App Engine. This first post is going to be fairly dull, unfortunately, as it will serve to cover what our requirements and non-requirements are, and the general approach we're going to take to achieve those objectives. In other words: Lots of bullet points, and little to no code. Don't worry, it'll get more exciting soon.

First, let's outline what we (or at least, I) expect out of a blogging system:

Simple authoring. I personally don't want to juggle with WYSIWYG HTML editors in order to write blog posts - I prefer to enter markup directly. But at the same time, I don't intend to make all potential users conform to the same expectations - so we should be able to support rich text editors if they're desired.
Good isolation of code and markup. Users shouldn't have to understand the innards of our blogging software if all they want to do is change how the blog looks or is laid out.
RSS and Atom support. This should go without saying, these days ...

Handling file uploads in App Engine

Posted by Nick Johnson | Filed under coding, app-engine, cookbook, tech

This is the ninth in a series of 'Cookbook' posts describing useful strategies and functionality for writing better App Engine applications.

One issue that comes up frequently on the App Engine groups is how to handle file uploads from users. Part of the confusion arises from the fact that App Engine apps cannot write to the local filesystem. Fortunately, the Datastore comes to the rescue: We can easily write an app that accepts file uploads and stores them in the datastore, then serves them back to users.

To start, we need an HTML form users can upload files through:

<html>
<head>
<title>File Upload</title>
</head>
<body>
<form method="post" action="/">
    <input type="file" name="file" />
    <input type="submit" value="Upload" />
</form>
</body>
</html>

We'll also need a datastore model we can store the uploaded file in:

class DatastoreFile(db.Model):
data = db.BlobProperty(required=True)
mimetype = db.StringProperty(required=True)

And finally, a Request Handler that can handle the uploads:

class UploadHandler(webapp.RequestHandler):
def get(self):
    self.response.out.write(template.render("upload.html", {}))

def post(self):
    file = self.request.POST['file']
    entity = DatastoreFile(data=file.value, mimetype=file.type)
    entity.put()
    file_url = "http://%s/%d/%s ...

Read more | Comments | 30 September, 2009

Custom Datastore Properties 1: DerivedProperty

Posted by Nick Johnson | Filed under coding, app-engine, cookbook, tech

This is the eighth in a series of 'cookbook' posts describing useful strategies and functionality for writing better App Engine applications.

The idea of extended Model properties has been covered a couple of times in the past, but it's a topic that I think is worth coming back to: There's a lot you can do with the datastore by writing your own Property subclass. To illustrate, I'd like to work through a salient and widely applicable example.

A common requirement when using the Datastore is storing some form of calculated property - for example, the lower-cased version of a text string, so it can be filtered on, or the length of a list, or the sum of some elements. One can do this manually, but it's easy to forget to update the computed property in some places. Other solutions include overriding the put() method, but this doesn't get updated if you store your entity using db.put(). Given the post this is appearing in, I'm sure you can figure out what the solution is going to be: a custom property class!

What we want here is a DerivedProperty. In order to be as flexible as ...

Read more | Comments | 28 September, 2009

Advanced Bulk Loading Part 5: Bulk Loading for Java

Posted by Nick Johnson | Filed under coding, app-engine, cookbook, tech

This is the seventh in a series of 'cookbook' posts describing useful strategies and functionality for writing better App Engine applications.

When it comes to Bulk Loading, there's currently a shortage of tools for Java users, largely due to the relative newness of the Java platform for App Engine. Not all is doom and gloom, however: Users of App Engine for Java can use the Python bulkloader to load data into their Java App Engine instances! Because App Engine treats each major version of an app as a completely separate entity - but sharing the same datastore - it's possible to upload a Python version specifically for the purpose of Bulk Loading. This won't interfere with serving your Java app to your users.

To follow this guide, you won't need to understand much of the Python platform, though you will need to know a little Python. If your bulkloading needs are straightforward, you won't need to know much at all - it's essentially connect-the-dots - but if your bulkloading needs are a little more complex, you'll need to understand some of the basics of programming in Python - defining and calling functions and methods, basically. Who knows, you ...

Read more | Comments | 25 September, 2009

Advanced Bulk Loading, part 4: Bulk Exporting

Posted by Nick Johnson | Filed under coding, app-engine, cookbook, tech

This is the sixth in a series of 'cookbook' posts describing useful strategies and functionality for writing better App Engine applications.

In previous posts, we covered Advanced Bulk-loading. Now we'll cover, briefly, the reverse: advanced use of the Bulk Exporter functionality. Unfortunately, the Bulk Exporter is currently much more limited than the Bulk-Loader - though it's also much less mature, so we can expect that to change - but there are still customizations you can apply.

The simplest one is the same as what we covered in part 1 of our Bulk-loader series: Custom conversion functions. Bulk exporter classes define a list of fields and conversion functions just like the importer; the difference is that these functions are expected to convert to strings, rather than from strings. Let's start by going over the equivalent to the two loader conversion functions we defined. First, one to format dates:

def export_date(fmt):

"""Returns a converter function that outputs the supplied date-time format."""

def converter(d):

return d.strftime(fmt)
return converter

So far so good. We use it the same way we used the converter function in the importer:

class AlbumExporter(bulkloader.Exporter):

def __init__(self):

bulkloader.Exporter.__init__(self, 'Album ...

Read more | Comments | 23 September, 2009

Advanced Bulk Loading, part 3: Alternate datasources

Posted by Nick Johnson | Filed under coding, app-engine, cookbook, tech

This is the fifth in a series of 'cookbook' posts describing useful strategies and functionality for writing better App Engine applications.

The bulkloader automatically supports loading data from CSV files, but it's not restricted to that. With a little work, you can make the bulkloader load data from any datasource you can access with Python. The key to this is the generate_records method. This method accepts a filename, and is expected to yield a list of strings for each entity to be uploaded. By overriding this method, we can load from any datasource we wish - for example, a relational database such as MySQL. To make this reusable, let's define a base class we can use to load data from MySQL databases:

import MySQLdb

class MySQLLoader(bulkloader.Loader):
  def __init__(self, kind_name, query, converters):
    self.query = query
    bulkloader.Loader.__init__(kind_name, converters)

  def initialize(self, filename, loader_opts)
    self.connect_args = dict(urlparse.parse_qsl(loader_opts))

  def generate_records(self, filename):
    """Generates records from a MySQL database."""
    db = MySQLdb.connect(self.connect_args)
    cursor = db.cursor()
    cursor.execute(self.query)
    return iter(cursor.fetchone, None)

What we've done here is extended the bulkloader's Loader class to load from a MySQL database instead ...

Read more | Comments | 21 September, 2009

Newer Older

Blogroll