Google I/O day 2: BigQuery and Prediction APIs

First up this morning on the App Engine track is the BigQuery and Prediction APIs talk.


First up is BigQuery. BigQuery is a new API that lets you make use of Google's infrastructure for performing queries and analysis over large collections of read only data. It's designed to scale to massive datasets, and integrates well with App Engine and other platforms.

To use it, you start by uploading your data to the new Google Storage service. Then, you import it into BigQuery tables, and you can run queries on those tables. Despite the fact that it handles billions of rows of data, there's no need to explicitly define indexes, or to shard your data.

The syntax used to query should be familiar: It's based on SQL, and is extremely flexible. Using the example of the database of all Wikipedia revisions, getting the 5 most edited titles is as simple as:

SELECT TOP(title, 5), COUNT(*) FROM [bigquery.test.001/tables/wikipedia] WHERE wp_namespace = 0;

The speed has to be seen to be believed - response times from under a second to a few seconds for hundreds of millions to tens of billions of rows - seemingly regardless ...

I/O Day 2

It's day 2 at I/O!

The keynote was packed with Android stuff today - see the official blog post for the awesome details.

One in particular stands out for App Engine users: the new Cloud to Device messaging service. This makes it possible for an app in the cloud to send an Intent directly to an Android device, asynchronously. That means that you can send a notification to an android device, and have the device receive it immediately: no polling, no hacks.

Obviously, App Engine is a shoo-in for the cloud portion of this. Previously, I blogged about authenticating against App Engine from your Android app - look out for a blog post soon on how to do the reverse, and send notifications from your App Engine app back to your Android device.

The keynote's over, but there's still plenty of new App Engine content to come in the talks today. I'll be writing up short posts on the new APIs and features during the day.

I won't get to everything - I can only be in one talk at a time, and I have Office Hours to staff as well - but I'll attempt to cover ...

I/O Day 1

Piles of awesome new stuff at today's Keynote, including WebM - based on the on2 codec, and the Chrome Web Store. In this post, I'll focus on what's new and interesting in App Engine, though.

The big news for App Engine, of course, is the release of App Engine for Business. App Engine for Business lets businesses build and maintain apps using the App Engine architecture, but within a controlled environment just for their corporate apps. See the blog post for full details, including centralized administration, SLA, a new billing model, and coming soon, support for SQL databases!

Also, Bloggart was briefly featured on the big screen. Woohoo!

I'm sure some of you have been waiting for something exactly like this, and I'll be blogging about some of the details in the near future. If there's something you'd really like to see me write about, let me know in the comments.

I'll be updating this post with more news from I/O as the day goes on. If you're here at the event, don't forget to stop by the Office Hours and pay me a visit.

Edit: I've spent most ...

More I/O prep

Once again, I'm afraid I have to tender my apologies for the lack of a decent blog post today, as I find myself suddenly swamped with last-minute I/O prep.

On Wednesday and Friday, though, I'll be providing recaps and details about some of the new App Engine features we're announcing at I/O, for those of you that couldn't make it. After that, I'll be going into more detail with howto posts and other material that goes into depth about some of the new stuff.

In the meantime, tell me what you're most excited about! Obviously it's hard to ask about the features that haven't been announced yet, but for those that are already out, let me know what you're interested in hearing about in the comments!

No post today - preparing for I/O

Unfortunately, a combination of preparation for I/O and jetlag leave me with little time to write a proper post today. For all of you who are going to come to I/O, though: It's going to be really, really awesome. For those of you that aren't: Sorry! At least you'll get to see the presentations on the web. :)

In the meantime, consider this an open thread for suggestions for future posts. Do you have something - App Engine related or otherwise - that you'd like to see me write about? Something I've written about in the past that you'd like to see me expand on? Let me know in the comments and I'll see what I can do.