Blogging on App Engine, part 6: Comments and Search
Posted by Nick Johnson | Filed under coding, app-engine, tech, bloggart
This is part of a series of articles on writing a blogging system on App Engine. An overview of what we're building is here.
Today we're going to tackle two separate issues: Support for commenting on posts, and support for search. Commenting is fairly straightforward, so we'll deal with that first.
Commenting
Rather than implement our own comments system, we're going to take advantage of an existing 'SaaS' commenting offering, Disqus. Disqus provides simple drop in Javascript powered comment support, and has, by now, a rather impressive feature set, incorporating support for various login schemes - their own, OpenID, facebook connect, twitter, and others - as well as advanced functionality like finding and displaying 'reactions' from social sites around the web along with comments from users.
Integrating disqus support is straightforward. Since some people might not want to use it, or might want to use an alternate system, however, we're going to use a new config setting to ensure we only enable it if it's wanted. Add the following to the bottom of config.py:
# To use disqus for comments, set this to the 'short name' of the disqus forum
# created for the purpose.
disqus_forum = None
Next, we need to add the Disqus javascript to the post pages. Open up post.html and add the following just before the final endblock directive:
{% if config.use_disqus %}
<h3 id="comments">Comments</h3>
<div id="disqus_thread"></div>
{% if devel %}
<script type="text/javascript">
disqus_developer = 1;
</script>
{% endif %}
<script type="text/javascript" src="http://disqus.com/forums/{{config.disqus_form}}/embed.js"></script
<noscript><a href="http://disqus.com/forums/{{config.disqus_forum}}/?url=ref">View the discussion thread.</a></noscript>
<a href="http://disqus.com">blog comments powered by <span>Disqus</span></a>
{% endif %}
That's all that's required to add disqus comment support (besides signing up for Disqus for your site, which we'll cover in a moment). Note how we conditionally set the 'disqus_developer' Javascript variable - this permits testing Disqus support locally, without having to sign up for an account for 'localhost'.
Disqus does offer one useful additional feature: Comment count support on listing pages. Implementing this is nearly as simple - open up listing.html, and add the following straight after the existing 'read more' link:
<a href="{{post.path}}">Read more</a> |
{% if config.use_disqus %}
<a href="{{post.path}}#disqus_thread">Comments</a> |
{% endif %}
<span>{{post.published|date:"d F, Y"}}</span>
And add the following block just before the final endblock directive:
{% if config.use_disqus %}
<script type="text/javascript">
//<![CDATA[
(function() {
var links = document.getElementsByTagName('a');
var query = '?';
for(var i = 0; i < links.length; i++) {
if(links[i].href.indexOf('#disqus_thread') >= 0) {
query += 'url' + i + '=' + encodeURIComponent(links[i].href) + '&';
}
}
document.write('<script charset="utf-8" type="text/javascript" src="http://disqus.com/forums/bloggart-demo/get_num_replies.js' + query + '"></' + 'script>');
})();
//]]>
</script>
{% endif %}
Now, listing pages will automatically show comment counts associated with each post.
Finally, in order to actually use disqus, you need to sign your site up. It's free, you simply need to go here, sign up for a disqus account if you don't have one already, and follow the instructions. When presented with the option, you can enable whatever set of additional features you feel comfortable with.
Note that if you've just added comment support to your blog, you'll need to edit any existing posts to trigger regeneration of them before you see the Disqus comments. We'll discuss how to do this regeneration when templates change in a more automated fashion in a later post.
Search
For search, we're again going to use an external service. The reasons for this are the same as for using external comments, only more so: Implementing search ourselves would be an involved undertaking, and the end result is unlikely to be as good as a third-party implementation we could have simply plugged right in and used. In this case, we're going to use a Google CSE, or Custom Search Engine.
The standard procedure for setting up a CSE requires you to sign up for one, specify all your details, and so forth, then copy a unique code into your site's markup. However, there is another option, called a linked CSE, which allows us to use a Custom Search Engine without any pre-configuration at all. This has major benefits for us, because we want to make it easy for other people to install Bloggart without having to go through the entire signup and configuration process of setting up their own CSE. Another feature of CSEs we'll take advantage of is the ability to host search results inside our own site, which significantly improves the look and feel.
Unfortunately, adding CSE support is going to be more complex than it may appear at first glance. So far, our static serving technique has served us well (if you'll forgive the pun), but here we're presented with a conundrum: The CSE definition file and the search results page don't depend on blog posts, so we can't use our existing dependency regeneration system to build them. They're not entirely static, either - they need to contain values taken from config.py - so we can't simply include them in the app as static resources. As a result, we're going to have to develop a new system for generating these sort of static resources.
Post-deploy tasks
What we'll do is develop a simple system of 'post deploy' hooks. These hooks get run the first time you access the admin page of your Bloggart instance after deploying a new version. Create a new file called post_deploy.py, and start off by defining the following function:
def run_deploy_task():
"""Attempts to run the per-version deploy task."""
task_name = 'deploy-%s' % os.environ['CURRENT_VERSION_ID'].replace('.', '-')
try:
deferred.defer(try_post_deploy, _name=task_name, _countdown=10)
except (taskqueue.TaskAlreadyExistsError, taskqueue.TombstonedTaskError), e:
pass
Here we're generating a task name from the current version number, and using the deferred library to enqueue a task using that name. The Task Queue API guarantees that a task with a given name is unique, and that even after it's run, it can't be reused for at least a week, and we'll make use of this to ensure that the post-deploy task doesn't run every time we load the page. If the task already exists, or has already run, an exception will be thrown; we simply catch and ignore the exception.
Next, define try_post_deploy:
def try_post_deploy():
"""Runs post_deploy() iff it has not been run for this version yet."""
version_info = models.VersionInfo.get_by_key_name(
os.environ['CURRENT_VERSION_ID'])
if not version_info:
post_deploy()
This function will be run from a task queue task. Since a deployed version of Bloggart may stick around for a lot longer than a week, relying on the task name alone isn't entirely satisfactory; we'll use a datastore model to indicate if the upgrade process has already been run. The model is defined in models.py:
class VersionInfo(db.Model):
bloggart_major = db.IntegerProperty(required=True)
bloggart_minor = db.IntegerProperty(required=True)
bloggart_rev = db.IntegerProperty(required=True)
@property
def bloggart_version(self):
return (self.bloggart_major, self.bloggart_minor, self.bloggart_rev)
If a VersionInfo entity matching the currently deployed version isn't found, then, we finally run the post_deploy function:
def post_deploy():
"""Carries out post-deploy functions, such as rendering static pages."""
q = models.VersionInfo.all()
q.order('-bloggart_major')
q.order('-bloggart_minor')
q.order('-bloggart_rev')
previous_version = q.get()
for task in post_deploy_tasks:
task(previous_version)
new_version = models.VersionInfo(
key_name=os.environ['CURRENT_VERSION_ID'],
bloggart_major = BLOGGART_VERSION[0],
bloggart_minor = BLOGGART_VERSION[1],
bloggart_rev = BLOGGART_VERSION[2])
new_version.put()
This function is divided into three parts. The first part looks for an existing VersionInfo entity indicating the most recent version that was installed before the current one. This will be useful in future, in case we need to run tasks to handle migration between versions - for example, if we change the data model. Next, we iterate through a list of post_deploy_tasks, and run each one, passing in the previous version. Finally, we create a new VersionInfo entity for the current bloggart version and app version, and store it to the datastore, ensuring the post_deploy process won't be run again for this deployment.
post_deploy_tasks is a simple list of functions; let's write one to generate the static pages, since that's why we started all this in the first place:
def generate_static_pages(pages):
def generate(previous_version):
for path, template in pages:
rendered = utils.render_template(template)
static.set(path, rendered, config.html_mime_type)
return generate
We're using Python's support for nested functions and closures again here, wrapping a simple function that generates a list of templates and saves them to the appropriate path. Here's how it's used:
post_deploy_tasks = []
post_deploy_tasks.append(generate_static_pages([
('/search', 'search.html'),
('/cse.xml', 'cse.xml'),
]))
Back to search
Now that we've figured out how to generate our static pages, it's time to get back to dealing with what we're generating. First, we should add a search box to the blog. Edit base.html, and add the following straight after the header-image div:
<div id="header-image"></div>
<form id="quick-search" action="/search" method="get">
<p>
<label for="q">Search:</label>
<input type="hidden" name="cref" value="http://{{config.host}}/cse.xml" />
<input type="hidden" name="cof" value="FORID:11" />
<input type="hidden" name="ie" value="UTF-8" />
<input type="text" name="q" size="31" />
<input type="image" name="sa" value="Search" src="/static/{{config.theme}}/images/search.gif" alt="Search" />
</p>
</form>
<script type="text/javascript" src="http://www.google.com/coop/cse/brand?form=quick-search&lang=en"></script>
This form is mostly prescribed by the CSE documentation here. Note that the form submits to a page on our site with the URL '/search', and the 'cref' form field refers to a 'cse.xml'. cse.xml is the xml definition file for our Custom Search Engine. The format for it is described here. Ours is fairly straightforward:
<?xml version="1.0" encoding="UTF-8" ?>
<GoogleCustomizations>
<CustomSearchEngine volunteers="false" visible="false" encoding="utf-8">
<Title>{{config.blog_name}}</Title>
<Description>{{config.slogan}}</Description>
<Context>
<BackgroundLabels>
<Label name="cse_include" mode="FILTER" />
<Label name="cse_exclude" mode="ELIMINATE" />
</BackgroundLabels>
</Context>
<LookAndFeel nonprofit="false" />
</CustomSearchEngine>
<Annotations>
<Annotation about="http://{{config.host}}/*">
<Label name="cse_include" />
</Annotation>
</Annotations>
</GoogleCustomizations>
We won't go into detail about how this works here - if you're interested, check out the specification and documentation, linked above.
Finally, the search results page. Create search.html, and enter the following:
{% extends "base.html" %}
{% block title %}Search results - {{config.blog_name}}{% endblock %}
{% block body %}
<div id="cse-search-results"></div>
<script type="text/javascript">
var googleSearchIframeName = "cse-search-results";
var googleSearchFormName = "cse-search-box";
var googleSearchFrameWidth = 649;
var googleSearchDomain = "www.google.com";
var googleSearchPath = "/cse";
</script>
<script type="text/javascript" src="http://www.google.com/afsonline/show_afs_search.js"></script>
{% endblock %}
As you can see, we're simply inheriting the basic site style, and inserting the script snippet required to have Google insert our search results inline, in the page.
With that, we're done - you should now be able to enter search terms in the search box anywhere on the site, and have the results displayed inline, embedded in the search results page of our site. Remember that if your site is new and has not yet been indexed by Google, the results will likely be empty - that will change as you start linking to your blog from elsewhere, of course.
As always, you can see the blog so far at
Edit: Note that the original version of the disqus support failed to substitute the forum's name (the {{config.disqus_forum}} variable). The code in the article and in the latest version of the repository has been updated, but the code shown in the repository linked above remains the old version.
In the next post, we'll show how to migrate content from an existing blog.
Previous Post Next Post