Using OpenID authentication on App Engine
Posted by Nick Johnson | Filed under python, openid, app-engine, coding, clickpass
With the release of SDK 1.3.4, preliminary support is available for native OpenID authentication in App Engine. Today, we'll demonstrate how to use the new OpenID support in your app.
Edit: There's now an official article on OpenID on App Engine!
The first step in setting up OpenID authentication is to change your app's authentication settings. Log in to the admin console, select your app, and go to "Application Settings". There, you can pull down the "Authentication Options" box, and select "(Experimental) Federated Login".
Once you've enabled OpenID authentication for your app, a few things change:
- URLs generated by create_login_url without a federated_identity parameter specified will redirect to the OpenID login page for Google Accounts.
- URLs that are protected by "login: required" in app.yaml or web.xml will result in a redirect to the path "/_ah/login_required", with a "continue" parameter of the page originally fetched. This allows you to provide your own openid login page.
- URLs generated by create_login_url with a federated_identity provider will redirect to the specified provider.
In order to make best use of this functionality, here's what we'll do:
- Provide an OpenID login page on /_ah/login_required ...
Exploring the new mapper API
Posted by Nick Johnson | Filed under app-engine, google-io, coding, mapreduce
One of the new features announced at this year's Google I/O is the new mapper library. This library makes it easy to perform bulk operations on your data, such as updating it, deleting it, or transforming/filtering/processing it in some fashion, using the 'map' (and soon, 'reduce') pattern. I'm happy to say that I'm deprecating my own bulkupdate library in favor of it.
The mapper API isn't just limited to mapping over datastore entities, either. You can map over lines in a text file in the blobstore, or over the contents of a zip file in the blobstore. It's even possible to write your own data sources - something we'll cover in a later post. Today, though, I'd like to dissect the demo that was presented at I/O. The demo uses a number of the mapper framework's more sophisticated features, so it's a good one to use to get an idea for how the framework works.
For basic usage, the Getting Started page in the mapper docs is the place to go. If you're interested in seeing something more complex in practice, read on...
The demo at I ...
Google I/O day 2: The Android push messaging API
Posted by Nick Johnson | Filed under push-messaging, android, google-io, app-engine
As I mentioned in my earlier post about the keynote, one of the exciting new releases at I/O is the new Push messaging API for Android. With it, you can push notifications from any webapp to an Android device in real-time - no polling, no messy hacks.
Push messaging is particularly valuable on the Android, because of battery life concerns. Even a single app polling every 5 minutes can cause a substantial drain on the battery - mostly wasted on checking for changes that don't exist. Several Android APIs already use push for sync - including Contacts, CAlendar, and the GMail app. Implementing it yourself, and getting it right, is tough, though - and that's where the new API comes in.
Under the hood, this is implemented as a background service. It starts whenever the network is available, and maintains a connection with the server for new messages. In order to prevent NATs and other network hardware from destroying the connection, and to detect that if it happens, it uses heartbeat messages - but it's been extensively tested and optimized to minimise the frequency of these as much as possible.
To use it, an app on the Android device registers with ...
Google I/O day 2: BigQuery and Prediction APIs
Posted by Nick Johnson | Filed under google-io, bigquery, prediction-api, google-storage
First up this morning on the App Engine track is the BigQuery and Prediction APIs talk.
BigQuery
First up is BigQuery. BigQuery is a new API that lets you make use of Google's infrastructure for performing queries and analysis over large collections of read only data. It's designed to scale to massive datasets, and integrates well with App Engine and other platforms.
To use it, you start by uploading your data to the new Google Storage service. Then, you import it into BigQuery tables, and you can run queries on those tables. Despite the fact that it handles billions of rows of data, there's no need to explicitly define indexes, or to shard your data.
The syntax used to query should be familiar: It's based on SQL, and is extremely flexible. Using the example of the database of all Wikipedia revisions, getting the 5 most edited titles is as simple as:
SELECT TOP(title, 5), COUNT(*) FROM [bigquery.test.001/tables/wikipedia] WHERE wp_namespace = 0;
The speed has to be seen to be believed - response times from under a second to a few seconds for hundreds of millions to tens of billions of rows - seemingly regardless ...
I/O Day 2
Posted by Nick Johnson | Filed under app-engine, google-io, android
It's day 2 at I/O!
The keynote was packed with Android stuff today - see the official blog post for the awesome details.
One in particular stands out for App Engine users: the new Cloud to Device messaging service. This makes it possible for an app in the cloud to send an Intent directly to an Android device, asynchronously. That means that you can send a notification to an android device, and have the device receive it immediately: no polling, no hacks.
Obviously, App Engine is a shoo-in for the cloud portion of this. Previously, I blogged about authenticating against App Engine from your Android app - look out for a blog post soon on how to do the reverse, and send notifications from your App Engine app back to your Android device.
The keynote's over, but there's still plenty of new App Engine content to come in the talks today. I'll be writing up short posts on the new APIs and features during the day.
I won't get to everything - I can only be in one talk at a time, and I have Office Hours to staff as well - but I'll attempt to cover ...
Help me name my distributed publishing system
Posted by Nick Johnson | Filed under censorship-resistant, dht, coding
A few days ago, I posted an outline for a distributed publishing system. I asked for name suggestions, and people have provided more than a few!
Here's the shortlist. Please vote on the one you think works best, or suggest your own.
I/O Day 1
Posted by Nick Johnson | Filed under app-engine, google-io, enterprise-app-engine
Piles of awesome new stuff at today's Keynote, including WebM - based on the on2 codec, and the Chrome Web Store. In this post, I'll focus on what's new and interesting in App Engine, though.
The big news for App Engine, of course, is the release of App Engine for Business. App Engine for Business lets businesses build and maintain apps using the App Engine architecture, but within a controlled environment just for their corporate apps. See the blog post for full details, including centralized administration, SLA, a new billing model, and coming soon, support for SQL databases!
Also, Bloggart was briefly featured on the big screen. Woohoo!
I'm sure some of you have been waiting for something exactly like this, and I'll be blogging about some of the details in the near future. If there's something you'd really like to see me write about, let me know in the comments.
I'll be updating this post with more news from I/O as the day goes on. If you're here at the event, don't forget to stop by the Office Hours and pay me a visit.
Edit: I've spent most ...
More I/O prep
Posted by Nick Johnson | Filed under app-engine, google-io
Once again, I'm afraid I have to tender my apologies for the lack of a decent blog post today, as I find myself suddenly swamped with last-minute I/O prep.
On Wednesday and Friday, though, I'll be providing recaps and details about some of the new App Engine features we're announcing at I/O, for those of you that couldn't make it. After that, I'll be going into more detail with howto posts and other material that goes into depth about some of the new stuff.
In the meantime, tell me what you're most excited about! Obviously it's hard to ask about the features that haven't been announced yet, but for those that are already out, let me know what you're interested in hearing about in the comments!
Behind the scenes with remote_api
Posted by Nick Johnson | Filed under python, remote_api, app-engine, internals
I've discussed remote_api in passing many times before on this blog, but never gone into detail about how it works, and the options you have for customizing it. Today, we'll remedy that, by taking a close look at its operation.
You may be wondering why anyone would want to customize remote_api - it seems like a fairly straightforward service, right? There are two main reasons you might want to do some degree of customization:
- You're providing a software-as-a-service solution, and need to provide remote_api access to your customers, but want to limit what they can do.
- You want to expose an API of your own via remote_api.
The first of these use-cases is particularly apt in the face of this nasty hack, which makes it possible to download a Python app's source if both the remote_api and deferred handlers are installed (and the user is an admin). You may want to use both of these libraries, but still keep your source to yourself. The second use-case is more complicated, and we'll only touch on it in passing.
How remote_api works
remote_api has two components, the client (otherwise known as the 'stub') and the server (otherwise known ...
No post today - preparing for I/O
Posted by Nick Johnson | Filed under app-engine, google-io
Unfortunately, a combination of preparation for I/O and jetlag leave me with little time to write a proper post today. For all of you who are going to come to I/O, though: It's going to be really, really awesome. For those of you that aren't: Sorry! At least you'll get to see the presentations on the web. :)
In the meantime, consider this an open thread for suggestions for future posts. Do you have something - App Engine related or otherwise - that you'd like to see me write about? Something I've written about in the past that you'd like to see me expand on? Let me know in the comments and I'll see what I can do.
Older