Shameless plug
Posted by Nick Johnson | Filed under travel, blogging
Just a quick reminder that I've started my tour of NZ, and I'll be blogging about it at http://laidbacktouring.blogspot.co.nz/ - so check it out and subscribe, if you're so inclined.
This is the last time I plug the blog here, I promise*.
* I reserve the right to renege on this promise. It _is_ my blog, after all.
Hiatus
Posted by Nick Johnson | Filed under travel, blogging, app-engine
I've been blogging regularly or semi-regularly about App Engine on this blog since mid 2008, and I've been on the App Engine team for even longer, and I've decided it's time for a well-earned break.
With that in mind, starting January 30th, I'm taking eight weeks' leave from my job at Google, during which I'll be cycling the length of New Zealand! Not just on a regular bike, either, but one one of these. With any luck, I'll come back from my break revitalised and ready to bring more new and interesting things to the App Engine community.
As a result, I won't be updating this blog during my vacation. Nor am I likely to answer (m)any questions on Stack Overflow or respond promptly to email directed my way. My most capable and excellent colleagues, Amy Unruh, Ikai Lan and Johan Euphrosine will continue to take good care of the community in my absence.
I'll be blogging and vlogging my experiences riding around New Zealand while I'm gone on my new blog, Laid Back Touring, so for anyone interested in following my exploits, that is the place to look ...
There's nothing quite like...
Posted by Nick Johnson | Filed under python, app-engine, theresnothingquitelike
Witness the product of an idle hour or two at work, and time spent pondering the meaninglessness of the cliche "There's nothing quite like...":
Powered by App Engine. Naturally.
Damn Cool Algorithms: Fountain Codes
Posted by Nick Johnson | Filed under damn-cool-algorithms, fountain-codes, lt-codes, bittorrent
That's right, it's time for another episode of the frustratingly infrequent Damn Cool Algorithms series! If you're not familiar with it, you might want to check out some of the previous posts.
Today's subject is Fountain Codes, otherwise known as "rateless codes". A fountain code is a way to take some data - a file, for example - and transform it into an effectively unlimited number of encoded chunks, such that you can reassemble the original file given any subset of those chunks, as long as you have a little more than the size of the original file. In other words, it lets you create a 'fountain' of encoded data; a receiver can reassemble the file by catching enough 'droplets', regardless of which ones they get and which ones they miss.
What makes this so remarkable is that it allows you to send a file over a lossy connection - such as, say, the internet - in a way that doesn't rely on you knowing the rate of packet loss, and doesn't require the receivers to communicate anything back to you about which packets they missed. You can see how this would be useful in a number of ...
Introducing iZac, the Android Bartender
Posted by Nick Johnson | Filed under iZac, android, barbot, adk, arduino
In a previous post, I alluded to a larger project I was working on in the physical/electronics/maker world, and now after much development, I'm pleased to present iZac, the Android Bartender!
A while ago I was inspired by Drink Making Unit 2.0 by Evil Mad Scientist Labs, and decided I wanted to build my own barbot. In this post I'll go over the high level details of what I did and why I did it.
One problem with any drink-dispensing machine is the difficulty of sourcing parts for interacting with food liquids. Pumps are expensive or impractical, valves are likewise problematic and difficult to obtain. My original intention was to use syringe pumps - effectively a DC motor hooked up to a threaded rod that pushes a syringe plunger in and out - and that was the reason behind my making the motor driver shield I wrote about previously. Unfortunately, syringe pumps turned out to be too fiddly and unreliable to use for the barbot, so I had to look for an alternate solution.
I ended up using a similar approach to that used by the DMU2.0, employing a battery powered aquarium pump to pressurize the ...
Migrating to Python 2.7, part 2: Webapp and templates
Posted by Nick Johnson | Filed under python, webapp2, python-27, jinja2, google-app-engine
In part 1 of the migration series, we discussed changes to your app to take advantage of the 2.7 runtime's support for multithreading. Today, we'll start looking at the changes to the included libraries, and how we'll have to modify our app to use them.
webapp2
First up is the replacement of App Engine's lightweight webapp framework with webapp2. Webapp2 is an external open-source project, which says it "follows the simplicity of webapp, but improves it in some ways: it adds better URI routing and exception handling, a full featured response object and a more flexible dispatching mechanism". Webapp2 does an excellent job of maintaining compatibility with webapp, which is why it was chosen as a replacement, but it also adds a number of useful improvements on it.
To see what's changed in concrete terms, let's assume we have a very straightforward app that looks something like this:
import os from google.appengine.ext import webapp from google.appengine.ext.webapp import template class BaseHandler(webapp.RequestHandler): def render_template(self, filename, **template_args): path = os.path.join(os.path.dirname(__file__), 'templates', filename) self.response.out.write(template.render(path, template_args)) class IndexHandler(BaseHandler ...
Grargh!
Posted by Nick Johnson | Filed under dog-ate-my-homework
I had about half of this week's post - part 2 in the "migrating to Python 2.7" series - written when my browser ate it. I foolishly hadn't made a backup, and I lack both the time and the energy to rewrite it right now.
Sorry to all of you who were waiting, and you can expect to see it mid next week, after GDD.
Migrating to Python 2.7, part 1: Threadsafe
Posted by Nick Johnson | Filed under python, threading, python-27, google-app-engine
With the recent experimental release of Python 2.7 support, many people are starting to move their apps from the 2.5 runtime to 2.7. In this series of posts, I'll go over the various considerations for migrating your app in detail, starting with the most immediately obvious - and arguably biggest impact - of them all: threadsafe and multithreaded Python apps.
As you're probably aware, the 2.7 runtime supports making your Python app multithreaded, meaning a single instance of the app may service multiple user requests at the same time. Due to the Global Interpreter Lock, a multithreaded Python app still has limited concurrency, but since most of the wallclock time of a typical App Engine app is spent waiting for RPCs - during which the GIL is not held - the parallelism, and corresponding improvements in utilization, can be substantial.
Moving to Python 2.7 and enabling multithreading is pretty straightforward. First, you have to update your app.yaml. Suppose the start of your app.yaml looks something like this:
application: myapp version: main runtime: python api_version: 1
To switch to Python 2.7, you change it to this (changes are highlighted):
application: myapp version: main runtime: python27 ...
Building a stackable motor driver shield for Arduino
Posted by Nick Johnson | Filed under busdriver-shield, i2c, avr, arduino
Hi, Hack a Day readers! If you're just looking for the source and schematics, they're here. Otherwise, read on!
Recently I've been working with embedded electronics a bit for a project I'm working on, using an Arduino. For the project, I need to control a number of low-power DC motors, and while there are plenty of solutions out there for driving one or two (or even three) like this, none of them let you drive as many motors as I need, and none of them are stackable, as they all require exclusive use of some IO lines. I could get several standalone driver boards like these, but I didn't much like the prospect of the ratsnest of wiring that would ensue. So, I decided to develop my own.
Before I go on, a brief aside about motor drivers. Motors can require a lot of power, so powering them directly from a microcontroller's IO pins is not practical. The obvious solution is to use a simple transistor, but in many cases, including mine, it's necessary to be able to drive the motor both forward and backwards, which isn't possible using a simple transistor ...
Using the Channel API on App Engine for instant traffic analysis
Posted by Nick Johnson | Filed under python, channels, app-engine, javascript, prospective-search
In last week's post, we introduced Clio, a system for getting live insight into your site's traffic patterns, and we described how the Prospective Search API lets us filter the site's traffic to get just the records we care about.
This week, we'll cover the other part of the system: delivering results in real-time. For this, we'll be using the Channel API to stream new log entries to admin users in real-time. As with last week's post, where there's differences between our demo implementation and what you'd use in a real-world system, I'll point those out.
The admin interface
First up, we need to provide a simple admin interface to which we'll stream results. Here's the handler for that:
class IndexHandler(webapp.RequestHandler):
"""Serve up the Clio admin interface."""
def get(self):
client_id = os.urandom(16).encode('hex')
channel_key = channel.create_channel(client_id)
template_path = os.path.join(os.path.dirname(__file__),
'templates', 'index.html')
self.response.out.write(template.render(template_path, {
'config': config,
'client_id': client_id,
'channel_key': channel_key,
}))
The only thing of significance we do here relates to the Channel API. First, we generate a random client ID by getting some ...
Older