Webapps on App Engine, part 6: Lazy loading
Posted by Nick Johnson | Filed under python, coding, app-engine, tech, framework
This is part of a series on writing a webapp framework for App Engine Python. For details, see the introductory post here.
A major concern for many people developing for App Engine, particularly those building low-to-medium traffic sites, is instance load time. When App Engine serves the first request to a new instance of your app, it must import the request handler module you specified, which in turn imports all the other modules required to serve the request. In large apps, this can add up to quite a lot of additional overhead for loading requests, and substantially impact the experience for end users.
There are a number of things you can do to reduce loading times, including using lighter weight frameworks instead of all inclusive ones, and breaking seldom used components up into separate handlers - an approach taken by bloggart for the admin interface. One source of inefficiency stands out as a prime candidate for optimisation, though: unnecessary imports.
Many frameworks, including the built in webapp framework, require you to provide a list of handler classes that should be instantiated to serve requests, in a 'url map'. When a request comes in, the framework simply instantiates the relevant class and ...
Webapps on App Engine, Part 5: Sessions
Posted by Nick Johnson | Filed under python, coding, app-engine, tech, framework
This is part of a series on writing a webapp framework for App Engine Python. For details, see the introductory post here.
Sessions are another component that's regularly required by webapps, but isn't really a core part of a framework. In this post, we'll discuss the session mechanisms available for App Engine and how they work, and settle on a recommendation for our own lightweight framework.
The basic mechanism behind a session library is straightforward: A random session ID is generated for the user, which is embedded in an HTTP cookie and sent to the user. Meanwhile, a record is created on the server with the same ID, containing any data the webapp wants to store about this user. When the user makes a subsequent request, the session library decodes the session ID from the cookie header, and loads the corresponding session record from permanent storage.
There are three major advantages of handling sessions this way, rather than naively storing session data directly in the cookie:
- We can store data that the client shouldn't be able to modify, such as the user's access flags.
- We can store data the client shouldn't even be able ...
Webapps on App Engine, part 4: Templating
Posted by Nick Johnson | Filed under python, coding, app-engine, tech, framework
This is part of a series on writing a webapp framework for App Engine Python. For details, see the introductory post here.
In the first three posts of this series, we covered all the components of a bare bones webapp framework: routing, request/response encoding, and request handlers. Most people expect a lot more out of their web framework, however. Individual frameworks take different approaches to this, from the minimalist webapp framework, which provides the bare minimum plus some integration with other tools, to the all-inclusive django, to the 'best of breed' Pylons, which focuses on including and integrating the best libraries for each task, rather than writing their own.
For our framework, we're going to take an approach somewhere between webapp's and Pylons': While keeping our framework minimal and modular, we'll look at the best options to use for other components - specifically, templating and session handling. In this post, we'll discuss templating.
To anyone new to webapps, templates may seem somewhat unnecessary. We can simply generate the output direct from our code, right? Many CGI scripting languages used this approach, and the results are often messy. Sometimes, which page to be generated isn't clear ...
Webapps on App Engine part 3: Request handlers
Posted by Nick Johnson | Filed under python, coding, app-engine, tech, framework
This is part of a series on writing a webapp framework for App Engine Python. For details, see the introductory post here.
Now that we've covered the background on request handling, it's time to tackle request handlers. Request handlers are the core and most obvious part of a web framework. They serve to simplify the writing of your app, and remove some of the boilerplate that you end up with if you write raw WSGI applications. Before we go any further, let's see what basic request handlers look like in a range of frameworks. Then we can discuss the pros and cons of each, and settle on one for ours.
def current_datetime(request):
now = datetime.datetime.now()
html = "It is now %s." % now
return HttpResponse(html)
App Engine's webapp framework:
class MainPage(webapp.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/plain'
self.response.out.write('Hello, webapp World!')
@expose('/display/')
def display(request, uid):
url = URL.query.get(uid)
if not url:
raise NotFound()
return render_template('display.html', url=url)
def hello1():
return "Hello World"
class HelloController(BaseController):
def index(self):
# Return a rendered template
#return render('/hello.mako')
# or ...
Webapps on App Engine part 2: Request & Response handling
Posted by Nick Johnson | Filed under python, tech, app-engine, coding, framework
This is part of a series on writing a webapp framework for App Engine Python. For details, see the introductory post here.
This post is mostly background on request and response encoding/decoding. If you're already fairly familiar with how this works in CGI and in higher level frameworks, you may want to skip this and wait for the next posting, on request handlers.
If you've ever written a CGI script, you'll be well aware of how much of a pain interpreting and decoding CGI environment variables and headers can be - so much so that the first action of many is to find or write a library to handle it for you! And if you've ever coded in a CGI-inspired language such as PHP, you're probably familiar with how much of a pain managing the combination of response headers and output content can likewise be.
As a result of this, one of the most basic tools a framework offers is some form of abstraction for request and response data. Typically, this takes care of collecting and parsing HTTP headers, parsing the query string (if any), decoding POSTed form data, and so forth. Better frameworks also ...
Webapps on App Engine part 1: Routing
Posted by Nick Johnson | Filed under python, coding, app-engine, tech, framework
This is part of a series on writing a webapp framework for App Engine Python. For details, see the introductory post here.
The first part of a framework you encounter when using one is, more often than not, the routing code. With that in mind, it's what we'll be tackling first. There are several approaches to handling request routing, and we'll go on a quick tour of the libraries and approaches before we decide on one and implement it.
The built-in App Engine webapp framework takes an extremely straightforward approach: The incoming request's URL is compared to a list of regular expressions in order, and the first one that matches has the corresponding handler executed. As an enhancement, any captured groups in the regular expression are passed to the request handler as additional arguments. You can see the code that does all this here - it's extremely straightforward and easy to follow.
The webapp module does one thing that I'm not a huge fan of: It ties the request routing in with handling the requests. The same function that finds the appropriate handler for a request also takes care of parsing the request and calling ...
Writing your own webapp framework for App Engine
Posted by Nick Johnson | Filed under python, coding, app-engine, tech, framework
Welcome back! I trust you all had a good holiday period? Mine was spent back in sunny New Zealand, seeing friends and family, visiting favorite restaurants, enjoying the sunshine, and learning to paraglide.
I would have started blogging again last week, but my first week back was made both exciting and frantically busy preparing for, then attending the BT Young Scientist Exhibition, where I gave tutorials on App Engine at the Google booth. But now, back to your regularly scheduled blogging...
Sometimes it seems like everyone has written their own blogging system, and everyone has written their own framework for webapps. That's not all these two things have in common, though: They're both excellent learning projects. Since you've already done the first, why not do the second? This will be the first of a series of posts covering how to write your own Python webapp framework. The framework, while targeted at App Engine, isn't exclusive to it.
As with the blogging project, it helps to set some goals before we get started. Here's our goals for this project:
- Lightweight. With cold startup time being a significant concern for many, it's essential to avoid creating ...