Sunday, December 6, 2009

Supporting Timezones in Google App Engine Pt. 2 - Writing UtcDateTimeProperty

I am covering how I added Timezone support to my web application My Web Brain in a series of posts.  Hopefully someone will find the discussion useful or even better contribute ways to achieve the same effect.
This is Part 2 of the series, and covers writing a custom model property for the datastore to smooth out some inconsistences of the Google App Engine datastore.

As we covered in Part 0: The Starting Point, the datestore does not store the timezone of DateTimeProperty properties defined in a model. If a datetime object has a timezone defined that is not UTC, it will be converted prior to storage to UTC time, but all DateTimeProperty values will be naive - without timezone information - when they are retrieved. That means:
  • If you set a naive datetime as the value, it will be retrieved as a naive datetime
  • If you set a datetime with a timezone of UTC, it will be retrieved as a naive datetime
  • If you set a datetime with a timezone different from UTC, it will be converted to UTC prior to storage and retrieved naive (in UTC).
One recommendation I saw proposed implementing a custom model property to make this behaviour more consistent. The provided code was not quite generic enough so I rewrote the same approach using some of the Google help pages.

The primary effect we are looking for is to ensure that no naive datetime values are retrieved from the datastore properties of this type. Handling naive datetimes that are timezone sensitive repeatedly throughout your application is both redundant and dangerous, so we are trying to avoid this and make the assumption that all datestore datetimes are UTC explicit rather than implicit.

A smaller, less consequential effect is to ensure all datetime values passed to the original model property have a UTC timezone set if none would otherwise be present. This could be consdered wasteful since naive datetimes set on DateTimeProperties are treated as UTC anyway. The purpose is to make that assumption explicit and to future proof the code against future changes in the way the provided datastore properties may work in the future.

The code for my UtcDateTimeProperty is below:

import pytz
from google.appengine.ext import db

class UtcDateTimeProperty(db.DateTimeProperty):

'''Marks DateTimeProperty values returned from the datastore as UTC. Ensures
all values destined for the datastore are converted to UTC if marked with an
alternate Timezone.

Inspired by

def get_value_for_datastore(self, model_instance):
'''Returns the value for writing to the datastore. If value is None,
return None, else ensure date is converted to UTC. Note Google App
Engine already does this. Called by datastore

date = super(UtcDateTimeProperty,
if date:
  if date.tzinfo:
    return date.astimezone(pytz.utc)
    return date.replace(tzinfo=pytz.utc)
  return None

def make_value_from_datastore(self, value):
'''Returns the value retrieved from the datastore. Ensures all dates
are properly marked as UTC if not None'''

if value is None:
  return None
  return value.replace(tzinfo=pytz.utc)

The code and comments above should relatively readable. As stated the code ensures all datetimes saved to the datestore have an explicit timezone set and when retrieved also have an explicit timezone.

Using the custom property is simple, since it inherits the methods and important behaviour from the DateTimeProperty class:

from model_ext import UtcDateTimeProperty
from google.appengine.ext import db

class MyModelObject(db.model):

  my_tz_date = UtcDateTimeProperty(required=false)

You could go further and create a true TzDateTimeProperty class that stores and applies a provided timezone. In the case of my application this was not required, so you will not see the code of that custom property class here.

As usual, any questions, corrections or comments are very welcome.  In the next  part of this series I will look at the practical implementation of timezone support throughout your application.

1 comment: