App Engine Java API call hooks
Posted by Nick Johnson | Filed under tech, app-engine, coding, java
In a previous post, we discussed API call hooks for Python. It's possible to hook and modify RPC calls in Java, too. In this post, we'll demonstrate how.
All API calls in Java are handled by the class com.google.apphosting.api.ApiProxy. This class behaves similarly to the ApiProxyStubMap in the Python SDK, having makeSyncCall and makeAsyncCall functions that take care of invoking the relevant API calls. Unlike the Python runtime, however, all Java API calls are handled by a single delegate class, defined by the interface ApiProxy.Delegate. The active delegate for an App Engine app can be retrieved with ApiProxy.getDelegate, and set with ApiProxy.setDelegate.
Another difference from the Python API is that API calls are serialized into byte arrays before being passed to the ApiProxy, and responses are likewise deserialized by the caller. As a result, the makeSyncCall method takes a byte array as an argument, and returns a byte array.
Because all calls are routed to a single Delegate, the granularity of hooks supported is restricted to hooking all API calls, or none. To make things easier, we'll define a Delegate implementation that dispatches API calls to other Delegate subclasses based ...
Shiny!
Posted by Nick Johnson | Filed under app-engine, usb-key, tech
Shiny, and headed my way (hopefully) in time for .astronomy:
Python Gotchas
Posted by Nick Johnson | Filed under python, tech, app-engine, coding
A lot of App Engine developers are fairly new to Python as well, and so probably haven't encountered a few subtle 'gotchas' about the Python programming language. This post aims to sum up the ones you're most likely to encounter while programming for App Engine.
Mutable default arguments
First up is something every Pythonista learns sooner or later. What's wrong with this snippet?
def append(value, l=[]):
l.append(value)
return l
API call hooks for fun and profit
Posted by Nick Johnson | Filed under tech, app-engine, cookbook, coding, hooks
API call hooks are a technique that's reasonably well documented for App Engine - there's an article about it - but only lightly used. Today we'll cover some practical uses of API call hooks.
To start, let's define a simple logging handler. This can be useful when you're seeing some odd behaviour from the datastore, especially when you're using a library that may be modifying how it works. You can also use it to log any other API, such as the URLFetch or Task Queue APIs. For this, we just need a post-call hook:
def log_api_call(service, call, request, response):
logging.debug("Call to %s.%s", service, call)
logging.debug(request)
logging.debug(response)
To install this for the datastore, for example, we call this:
apiproxy_stub_map.apiproxy.GetPostCallHooks().Append(
'log_api_call', log_api_call, 'datastore_v3')
The arguments to Append are, in order, a name for the hook, the hook function itself, and, optionally, the service you want to hook. If you leave out the last argument, the hook is installed for all API calls.
Once a hook is installed, it remains installed as long as the runtime is loaded - including across requests, and for requests to ...
Talk Schedule
Posted by Nick Johnson | Filed under app-engine, conferences
I'm going to be spending the next week going to conferences. As a result, your regularly scheduled posts are likely to be absent next week; it's going to be hectic enough as it is. In the meantime, here's my talk schedule for the next while.
November 2: Presenting on App Engine Python at Stack Overflow Dev Day Amsterdam.
November 4: Presenting on App Engine Java at Øredev in Malmo, Sweden.
November 18: Python Ireland talk on App Engine at Scruffy Murphy's, Dublin, Ireland.
November 30 - December 4: At .Astronomy 2009 in Leiden, Netherlands, helping people put together Astronomy apps on App Engine
January 24 - January 30: LovelySnow Sprint in the Austrian Alps, another App Engine hackathon.
Server-side JavaScript with Rhino
Posted by Nick Johnson | Filed under tech, app-engine, cookbook, coding, java
I've only made limited use of the Java runtime for App Engine so far: The two runtimes are largely equivalent in terms of features, and my personal preference is for Python. There are, however, a few really cool things that you can only do on the Java runtime. One of them is embedding an interpreter for a scripting language.
Rhino is a JavaScript interpreter implemented in Java. It supports sandboxing, meaning you can create an environment in which it's safe to execute user-provided code, and it allows you to expose arbitrary Java objects to the JavaScript runtime, meaning you can easily provide interfaces with which the JavaScript code can carry out whatever actions it's designed to carry out.
Rhino also supports several rather cool additional features. You can set callback functions that count the number of operations executed by the JavaScript code, and terminate it if it takes too long, you can serialize a context or individual objects and deserialize them later, and you can even use continuations to pause execution when waiting for data, and pick it up again later - possibly even in another request! Hopefully we'll show off some of these features in future ...
Blogging on App Engine, part 10: Recap
Posted by Nick Johnson | Filed under coding, app-engine, tech, bloggart
This is part of a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.
Over the last 9 posts, we've covered building a fully functional blogging system for App Engine from scratch, and I've even migrated this blog to it. In the process, we've covered many important components of App Engine, including the new deferred library (and through it, Task Queues), important design principles, such as pre-generation of content, and interesting technologies such as PubSubHubbub, CSEs, Disqus, and sitemaps.
There's also been significant community involvement, for which I am very grateful. Amongst the contributors were:
- Moraes, who ported bloggart to werkzeug, calling it bloggartzeug
- Sylvain, who ported bloggart to tornado, calling it bloggartornado
- andialbrecht, who contributed - and continues to contribute - enhancements and bugfixes for bloggart
- Everyone who contributed a name suggestion on the first post, and everyone who has provided feedback during the series
- Everyone who filed bug reports and feature requests in the issue tracker
To give some sort of objective assessment of how well Bloggart measures up, we need to compare it to our original goals, outlined in the very first post:
- Simple ...
Blogging on App engine, part 9: Sitemaps and verification
Posted by Nick Johnson | Filed under tech, app-engine, coding, bloggart
This is part of a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.
Today we're going to cover basic sitemap support and verifying your site with Google.
Sitemaps
Sitemaps are a recent innovation that aim to make it easier for search engines to find and index your site. The format is a very straightforward XML file. Several optional attributes can be present, such as the last-modified date and update frequency; for this first attempt we're not going to use any of them, and just provide a basic listing of URLs. Future enhancements could provide more sitemap information, and break the sitemap into multiple files for extensibility.
For the purposes of generating a complete sitemap, we have a significant advantage: Our static serving infrastructure provides us with a convenient means of getting a list of all valid URLs. Not all URLs should be indexed, however, so we should make it possible to specify what content should be indexed. Add a new property to the StaticContent model in static.py:
indexed = db.BooleanProperty(required=True, default=True)
We'll need to enhance our set() method to take this ...
Blogging on App Engine, part 8: PubSubHubbub
Posted by Nick Johnson | Filed under tech, app-engine, coding, bloggart
This is part of a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.
Notice anything different? That's right, this blog has now been migrated to Bloggart - after all, if I won't run it, why should I expect anyone else to? ;)
Migrating comments to Disqus
In the previous post, I promised I'd cover migrating comments in today's post. Unfortunately, doing so proved to be both more complicated and less interesting than anticipated, so I'm going to instead provide an overview of the required steps. If you're really determined to see all the nitty-gritty, you can examine the change yourself.
Import of comments to disqus is through the disqus API. The API uses a straightforward RESTful model, with requests URL-encoded as either GET or POST requests, and responses returned as JSON strings. To make our lives easier, we'll define a straightforward wrapper function to make Disqus API calls:
def disqus_request(method, request_type=urlfetch.GET, **kwargs):
kwargs['api_version'] = '1.1'
if request_type == urlfetch.GET:
url = "http://disqus.com/api/%s?%s" % (method, urllib.urlencode(kwargs))
payload = None
else:
url = "http://disqus.com/api/%s ...
Blogging on App Engine, part 7: Migration
Posted by Nick Johnson | Filed under tech, app-engine, coding, bloggart
This is part of a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.
We're finally going to tackle (at least part of) that big bugbear of blogging systems: Migrating from the old system to the new one. In this post, we'll cover the necessary pre-requisites, briefly cover the theory of importing from a blogging system hosted outside App Engine, then go over a practical example of migrating from Bloog (since that's what this blog is hosted on).
Regenerating posts
Before we can write migration or import scripts, we need to improve (again) our dependency regeneration code. One thing that's probably occurred to you if you've been following this series is that there's currently no easy way to regenerate all the resources when something global changes such as the theme or the configuration. One could simply call .publish() on each blog post, but that would result in regenerating the common resources, such as the index and tags pages, over and over again - potentially hundreds of times. The same applies to migration: We could publish each new post as we process it, but this ...
Newer Older