Implementing a dropbox service with the Blobstore API (part 2)
Posted by Nick Johnson | Filed under python, app-engine, blobstore, datastore, filehangar
In part 1 of this series, we demonstrated what's necessary to build a very basic 'dropbox' type service for App Engine. Today, we're going to enhance that by adding support for 'rich' upload controls.
Various types of rich upload controls have sprung up in recent years in order to work around the weaknesses of the HTML standard file input element, which only allows selection of one file at a time, and doesn't support any form of progress notification. The most common widgets are written in Flash, but there are a variety of solutions available. With the ongoing browser adoption of HTML5, additional options are opening up, too!
Today we're going to use an excellent component called Plupload. Plupload consists of a Javascript component with a set of interchangeable backends. Backends include Flash, HTML5, Gears, old-fashioned HTML forms, and more. When you configure Plupload, you can specify which backends you want it to try, in which order, and it will stop when it finds one that works in the user's browser.
Different backends have different capabilities, and the ones you need will depend on your use-case. Check out the feature matrix on the Plupload homepage to get an idea of what's available.
Unfortunately, Plupload currently requires some small modifications to work with the App Engine blobstore. I've made a modified version that works on App Engine available here (.zip download). In future, this will hopefully not be necessary, with improvements to both the Blobstore service and to plupload. With this modified copy, you can use plupload pretty much exactly as documented on the site. Plupload supports a variety of features, but today we're going to focus on getting basic operation, uploading a single file, working. In future posts, we'll handle support for uploading multiple files at once, and a more sophisticated UI.
Download and unzip Plupload from the link above, and copy the contents of the 'js' subdirectory into 'static/plupload' in your app. Next, we'll modify the upload form to use plupload:
<html>
<head>
<title>File Hangar: Upload file</title>
<script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js"></script>
<script type="text/javascript" src="/static/plupload/gears_init.js"></script>
<script type="text/javascript" src="/static/plupload/plupload.full.min.js"></script>
<script type="text/javascript">
$(function() {
var uploader = new plupload.Uploader({
runtimes: 'gears,html5,flash,html4',
browse_button: 'pickfiles',
container: 'container',
url: '{{form_url}}',
use_query_string: false,
multipart: true,
flash_swf_url: '/static/plupload/plupload.flash.swf',
});
uploader.bind('FilesAdded', function(up, files) {
$.each(files, function(i, file) {
$('#filelist').append(
'<div id="' + file.id + '">' +
'File: ' + file.name + ' (' + plupload.formatSize(file.size) + ') <b></b>' +
'</div>'
);
});
});
uploader.bind('UploadProgress', function(up, file) {
$('#' + file.id + ' b').html(file.percent + '%');
});
uploader.bind('FileUploaded', function(up, file, response) {
window.location = response.response;
});
uploader.bind('Error', function(up, err) {
alert("Upload error: " + err.message);
});
uploader.bind('QueueChanged', function(up) {
uploader.start();
});
uploader.init();
});
</script>
</head>
<body>
<p style="float: right"><a href="{{logout_url}}">Log Out</a></p>
<h1>Upload a file to App Engine File Hangar</h1>
<form>
<div id="container">
<div id="filelist"></div>
<a id="pickfiles" href="#">[Upload files]</a>
</div>
</form>
</body>
</html>
First, note that we're including a couple of new javascript files: gears_init.js, which initializes the gears runtime for us (if it's available), and plupload itself. We don't have to include the extensions - plupload will find them for itself. Next, take a look at the first part of the embedded Javascript, where we initialize the uploader object using a dict of settings. Significant amongst these are 'runtimes', where we specify a list of runtimes and the order we want it to try them in, 'url', which specifies the URL to upload to, 'use_query_string', a custom modification to prevent plupload attaching a query string to the URL, and 'multipart', which instructs plupload to use multipart encoding for the form.
Next, we bind to several events generated by plupload. When a file is added, we add some UI elements. Upload progress notifications cause us to modify those elements to show the percent complete. When the file finishes uploading, we extract the URL from the body of the response, and send the user's browser there. Finally, we automatically start the upload when the queue is modified by adding a file to it.
The HTML changes are fairly straightforward. We still use a form, but plupload generates most of its contents. Inside the form, we have a container element and a link to pick files, both of which have IDs that we passed to the plupload constructor.
Now we need to consider the server-side modifications needed to support this. These are fairly simple, but a little up-front explanation helps: As you know, the Blobstore API requires that we only send redirects in response to an upload. Most of the plupload backends respect redirects, automatically fetching the referenced page. What we want, though, is the URL of the final page, so we can redirect the user's browser to it. In order to accomplish this, we do the following:
- Modify the file upload handler to return a redirect to /file/{id}/success
- Add a handler for /file/{id}/success, which returns the url for /file/id
- Cause the Javascript to fetch the body of the response (which is the contents of /file/{id}/success - eg, a URL), and redirect the user's browser to it.
Here it is in practice, with changes again highlighted in bold:
class FileUploadHandler(blobstore_handlers.BlobstoreUploadHandler):
def post(self):
blob_info = self.get_uploads()[0]
if not users.get_current_user():
blob_info.delete()
self.redirect(users.create_login_url("/"))
return
file_info = FileInfo(blob=blob_info.key(),
uploaded_by=users.get_current_user())
db.put(file_info)
self.redirect("/file/%d/success" % (file_info.key().id(),))
class AjaxSuccessHandler(BaseHandler):
def get(self, file_id):
self.response.headers['Content-Type'] = 'text/plain'
self.response.out.write('%s/file/%s' % (self.request.host_url, file_id))
# ...
application = webapp.WSGIApplication([
('/', FileUploadFormHandler),
('/upload', FileUploadHandler),
('/file/([0-9]+)', FileInfoHandler),
('/file/([0-9]+)/download', FileDownloadHandler),
('/file/([0-9]+)/success', AjaxSuccessHandler),
])
With that done, you should now be able to upload files with (slightly) improved interactivity! Once again, the source is available here. In the next post, we'll cover improving the UI and supporting multiple file upload.
Before we go, a brief word about backend support and browsers: All the backends in the modified Plupload should work with App Engine, though the browserplus one does not follow redirects, and hence cannot return the body of the response _or_ the URL being redirected to. Silverlight ought to work, but is untested, as I don't have a Windows machine handy. With the available backends, every browser ought to support something better than HTML4, though in my own testing, Chrome on mac failed to display progress, even though it works just fine with several runtimes.
Previous Post Next Post