Monday, January 3, 2011

Static Resources and Cache-Busting on Google App Engine Python

Hello and happy new year!

In a couple of previous posts I discussed using Google Closure JavaScript Compiler in your build process to minimise the size of your JavaScript resources. Arguably a bigger win for the end user experience is to ensure the caching of all of your website static resources (JavaScript, stylesheets and images) is properly optimised. In this post I want to share how I think I have achieved this Google App Engine (Python).

Ideally, we would like the end user's browser to use its cache for every static resource, on every request, until that resource changes because we have deployed a new application version. Adding the (relatively new) default_expiration and expiration directives to your app.yaml file will achieve the first part of this requirement:

- url: /images
  static_dir: images
  expiration: "99d"

Behind the scenes, these directives add additional headers to the HTTP response, letting the browser know that they can safely serve these files from cache on subsequent requests.

But how do we tell the end user browser when our static resources have changed?

The solution is called cache-busting, and there are a number of similiar approaches. These involve changing the request URI for the static resources. This could mean appending a version identifier to the individual JS, CSS and image filenames. When the browser sees the request for a nominally different file it has no reason to pull the cached copy, so it requests the file again, with the specified caching headers in place again to prevent it being requested again:

First Request
URI:\images\banner_v1.jpg (requested)

Second Request
URI:\images\banner_v1.jpg (not requested - cached)

Third Request
URI:\images\banner_v2.jpg (updated file - re-requested)

Fourth Request
URI:\images\banner_v2.jpg (not requested - cached)

This technique works, but how you consistently achieve it without introducing errors, confusing source control with trivial changes or simply absorbing too much development time is worth a discussion. Making manual changes to resource file names, the app.yaml static resource directives and the HTML for every application deployment is not very practical.

My approach to removing the development pain associated with cache busting works as follows: Instead of mangling the actual filename of the resource, I change the URI path. This means filenames themselves do not need to change. I also setup static file directives in the app.yaml such that the intermediate path variance (due to changing version) is ignored:

- url: /images/.*/(.*)
  static_files: static/img/\1
  upload: static/img/(.*)

Using a directives such as this, the position or naming of the actual static resource in the source code never changes. For example, the following URI's all serve the same static file: » /static/img/banner.jpg » /static/img/banner.jpg

Within the HTML (or any other resource specifying a URI) the source code contains a known token:

img src="/images/xx-replace-with-version-id-zz/banner.jpg"

My build script, written in Apache Ant, replaces the known token in the prepared build files with the version identifier. The original source files (whose known token works fine in development) are not updated:

<copy overwrite="true" todir="${}">
    <fileset dir="${path.assets.src}">
      <include name="**/*.html" />
      <include name="**/*.css" />
    <filterset begintoken="xx-" endtoken="-zz">
      <filter token="replace-with-version-id" value="${}" />

The build script goes on to deploy the application either to its online test or production versions and then uses the same version identifier to tag newly deployed source code in my source control.

I know some developers prefer using a dynamically generated version identifier (using os.environ['CURRENT_VERSION_ID']) within their HTML or other URI carrying source. Since I already use a build script for JS compilation and Python linting, my approach of replacing a known token makes more sense to me - It removes complexity from my code and allows a full correspondence with the version identifier between the deployed version and the source control records.

I'd be interested in hearing if someone has an idea for improving this process or thinks another approach has distinct advantages I have not considered. If you have questions, leave them here as well.


  1. That it was shocking at that moment a chrome steel activity sit back and watch that marketed with the exact same charge in the form of look at via several other labels. That parallel amongst the van and watch can be comical, plus the numbers are generally almost exactly the same every. The fact is that, your amazing interpretation of any observe that had been blisteringly swiftly, not to mention mega gorgeous, achieved the country collapse. The actual designer watches for sure would not! It absolutely was hence attractive and additionally chic. He did this a wristwatch destined to get the iconic reflection for the make replica omega. Whereas absolutely everyone idea a watch seemed to be beautiful, exclusively a small number of families truly appreciated the benefits in the nation style, additionally, the extraordinary company at the rear of the software. Fast in advance to 2015. typically the pieces have fashion inside your prior to when and ceramic certainly is the hippest materials close to The industry is usually full of wrist watches constructed from naff, aluminum or inflated material. All the running watches for a long time offers tried out a variety of items together with prefer a great many others lives in ceramic for a few purposes despite the fact that entirely ceramic pieces continue unheard of. Considering that the rewards of the very very first follow, they need put together in the unique structure making a lot of designs and additionally closes which will spreads out most of the access throughout the market place when staying genuine to their primary vision- an extravagance outdoor activity watch for an incredibly scary consumer. The principle look at was additional in order to include a tourbillion, some chronograph, your perpetual work schedule, an important awesome side effect, and even more!. That may find a watch wholly blanketed with stones any time which may be ones own matter. Gemstone to your preference or perhaps not necessarily, a enjoy is invariably ready look at an item fresh, bold plus daring. Speaking of extremes, I prefer the follow that located 48mm huge, yet the fact is that Ariel definitely assessed them replica watches uk, therefore i california not likely. Moreover, I can merely suppose that bracelets supplied a rest through the convention with what was you can get, relating to old anklet bracelets that can be purchased. In addition to, when you have obtained that gold tweaked suitably, it ought to be comfortable at any kind of wrist.

  2. This comment has been removed by the author.

  3. Quel est notre produit? - La meilleure marque de luxe au monde regarde des kopie horloges de la plus haute qualité! Si vous voulez avoir ces Kopie Horloges IWC de luxe, vous voulez porter ces montres sur votre poignet, mais arrêtez-vous en raison de leur prix élevé. Alors s'il vous plaît arrêtez vos pas, c'est votre endroit le plus correct..

  4. nice and realy good work now it is amazingamiduos crack

  5. replika rolex klockor, som kombinerar elegant stil och spetsteknik, en rad stilar av replika rolex submariner klockor, går pekaren mellan din exklusiva smakstil.

  6. Hi! Im so happy that you shared this stuff. I was really having a hard time understanding static resources.
    Chris | click here