This is the fifth in a series of posts providing a day-by-day playlist to help break up the Google I/O session videos - specifically the App Engine ones - into manageable chunks for those that haven't seen them.
Today's video is Data migration in App Engine, Matthew Blain's talk on new improvements to the bulkloader. I've blogged about the new bulkloader previously, but Matthew's talk goes into a lot more detail.
Matt starts talking about the new configuration format at 6:38, if you want to skip the intro.
With the full videos of the App Engine sessions at I/O now released, those who missed I/O, or, like me, didn't have time to catch all the sessions can now catch up on what they missed out on. The list is quite intimidating, though - there's a lot of content there, and most of us have a limited amount of time to spend watching each day.
With that in mind, over the next couple of weeks, I'm going to be posting session videos, one or two each day, in an order that's hopefully useful and makes sense to people. I'll also explicitly note who'll find each video of interest, and what other videos, if any, you should watch first.
First in the lineup is Guido's Appstats talk. This talk will be of interest to both Python and App Engine developers, at all levels of skill. There's some Python-specific code in the talk, but it mostly applies to both languages.
What's that, you say? You already know about appstats? Even if you do, there's almost certainly something you didn't know in Guido's talk. If you're already familiar ...
With the release of SDK 1.3.4, preliminary support is available for native OpenID authentication in App Engine. Today, we'll demonstrate how to use the new OpenID support in your app.
Edit: There's now an official article on OpenID on App Engine!
The first step in setting up OpenID authentication is to change your app's authentication settings. Log in to the admin console, select your app, and go to "Application Settings". There, you can pull down the "Authentication Options" box, and select "(Experimental) Federated Login".
Once you've enabled OpenID authentication for your app, a few things change:
- URLs generated by create_login_url without a federated_identity parameter specified will redirect to the OpenID login page for Google Accounts.
- URLs that are protected by "login: required" in app.yaml or web.xml will result in a redirect to the path "/_ah/login_required", with a "continue" parameter of the page originally fetched. This allows you to provide your own openid login page.
- URLs generated by create_login_url with a federated_identity provider will redirect to the specified provider.
In order to make best use of this functionality, here's what we'll do:
- Provide an OpenID login page on /_ah/login_required ...
One of the new features announced at this year's Google I/O is the new mapper library. This library makes it easy to perform bulk operations on your data, such as updating it, deleting it, or transforming/filtering/processing it in some fashion, using the 'map' (and soon, 'reduce') pattern. I'm happy to say that I'm deprecating my own bulkupdate library in favor of it.
The mapper API isn't just limited to mapping over datastore entities, either. You can map over lines in a text file in the blobstore, or over the contents of a zip file in the blobstore. It's even possible to write your own data sources - something we'll cover in a later post. Today, though, I'd like to dissect the demo that was presented at I/O. The demo uses a number of the mapper framework's more sophisticated features, so it's a good one to use to get an idea for how the framework works.
For basic usage, the Getting Started page in the mapper docs is the place to go. If you're interested in seeing something more complex in practice, read on...
The demo at I ...
As I mentioned in my earlier post about the keynote, one of the exciting new releases at I/O is the new Push messaging API for Android. With it, you can push notifications from any webapp to an Android device in real-time - no polling, no messy hacks.
Push messaging is particularly valuable on the Android, because of battery life concerns. Even a single app polling every 5 minutes can cause a substantial drain on the battery - mostly wasted on checking for changes that don't exist. Several Android APIs already use push for sync - including Contacts, CAlendar, and the GMail app. Implementing it yourself, and getting it right, is tough, though - and that's where the new API comes in.
Under the hood, this is implemented as a background service. It starts whenever the network is available, and maintains a connection with the server for new messages. In order to prevent NATs and other network hardware from destroying the connection, and to detect that if it happens, it uses heartbeat messages - but it's been extensively tested and optimized to minimise the frequency of these as much as possible.
To use it, an app on the Android device registers with ...
It's day 2 at I/O!
The keynote was packed with Android stuff today - see the official blog post for the awesome details.
One in particular stands out for App Engine users: the new Cloud to Device messaging service. This makes it possible for an app in the cloud to send an Intent directly to an Android device, asynchronously. That means that you can send a notification to an android device, and have the device receive it immediately: no polling, no hacks.
Obviously, App Engine is a shoo-in for the cloud portion of this. Previously, I blogged about authenticating against App Engine from your Android app - look out for a blog post soon on how to do the reverse, and send notifications from your App Engine app back to your Android device.
The keynote's over, but there's still plenty of new App Engine content to come in the talks today. I'll be writing up short posts on the new APIs and features during the day.
I won't get to everything - I can only be in one talk at a time, and I have Office Hours to staff as well - but I'll attempt to cover ...
The big news for App Engine, of course, is the release of App Engine for Business. App Engine for Business lets businesses build and maintain apps using the App Engine architecture, but within a controlled environment just for their corporate apps. See the blog post for full details, including centralized administration, SLA, a new billing model, and coming soon, support for SQL databases!
Also, Bloggart was briefly featured on the big screen. Woohoo!
I'm sure some of you have been waiting for something exactly like this, and I'll be blogging about some of the details in the near future. If there's something you'd really like to see me write about, let me know in the comments.
I'll be updating this post with more news from I/O as the day goes on. If you're here at the event, don't forget to stop by the Office Hours and pay me a visit.
Edit: I've spent most ...
Once again, I'm afraid I have to tender my apologies for the lack of a decent blog post today, as I find myself suddenly swamped with last-minute I/O prep.
On Wednesday and Friday, though, I'll be providing recaps and details about some of the new App Engine features we're announcing at I/O, for those of you that couldn't make it. After that, I'll be going into more detail with howto posts and other material that goes into depth about some of the new stuff.
In the meantime, tell me what you're most excited about! Obviously it's hard to ask about the features that haven't been announced yet, but for those that are already out, let me know what you're interested in hearing about in the comments!
I've discussed remote_api in passing many times before on this blog, but never gone into detail about how it works, and the options you have for customizing it. Today, we'll remedy that, by taking a close look at its operation.
You may be wondering why anyone would want to customize remote_api - it seems like a fairly straightforward service, right? There are two main reasons you might want to do some degree of customization:
- You're providing a software-as-a-service solution, and need to provide remote_api access to your customers, but want to limit what they can do.
- You want to expose an API of your own via remote_api.
The first of these use-cases is particularly apt in the face of this nasty hack, which makes it possible to download a Python app's source if both the remote_api and deferred handlers are installed (and the user is an admin). You may want to use both of these libraries, but still keep your source to yourself. The second use-case is more complicated, and we'll only touch on it in passing.
How remote_api works
remote_api has two components, the client (otherwise known as the 'stub') and the server (otherwise known ...
Unfortunately, a combination of preparation for I/O and jetlag leave me with little time to write a proper post today. For all of you who are going to come to I/O, though: It's going to be really, really awesome. For those of you that aren't: Sorry! At least you'll get to see the presentations on the web. :)
In the meantime, consider this an open thread for suggestions for future posts. Do you have something - App Engine related or otherwise - that you'd like to see me write about? Something I've written about in the past that you'd like to see me expand on? Let me know in the comments and I'll see what I can do.Newer Older