Writing a simple memoization decorator in Python

I was generating some reports recently that involved accessing expensive object methods whose results were known to not change on subsequent calls; However, instead of using local variables, I sketched-out this quick memoization decorator to save method responses as variables on the object (using a leading ‘_’ followed by the method-name as the variable name):

def cache_method_results(fn):
    def _view(self, *args, **kwargs):
        var_name = '_{n}'.format(n=fn.__name__)

        if var_name in self.__dict__:  # Return the copy we have
            return self.__dict__[var_name]

        else:  # Run the function and save its result
            self.__dict__[var_name] = fn(self, *args, **kwargs)
            return self.__dict__[var_name]

    return _view

You might use it like this:

class Foo(object):
    def some_expensive_operation(self):
        ...calculate something big and unchanging...
        return results

f = Foo()
print(f.some_expensive_operation())  # This first call will run the calculation
print(f.some_expensive_operation())  # but this one will used the cached result instead

It’s not rocket science, but these little tricks add to the fun of using Python.

jsmacro 0.2.3

The latest jsmacro (v0.2.3) adds support for “else” clauses to “if”, “ifdef”, and “ifndef” statements. Combine this with the command-line variable definition support and you can now do fun things like this:

//@ifdef IE6_BUILD
 ...custom IE6 code here
 ...code for other browsers here

Of course, this goes against the idea that your JavaScript would remain usable for development without needing to be processed, but it’s just an example. Longer term, I hope to have a different approach available that will allow conditional code substitution so that browser specific optimizations won’t get in the way of an easy development/test/debug process.

jsmacro — an oddly named JavaScript preprocessor

For awhile now I’ve wanted a JavaScript preprocessor to conditionally include debug and testing code when needed. It’s always registered as merely a “nice to have”, so I hadn’t sought one out. However, I had a little time over the weekend and wanted to play with the idea, so here it is: jsmacro (on GitHub.)

[Note that before writing this I did seek out existing implementations, and found js-preprocess to be the most interesting; However, I needed something that would work as part of an existing build chain, so authoring the tool in Python instead of JavaScript made more sense.]

Currently, jsmacro is poorly named, as I didn’t write the macro system that was in my head. Instead, it’s currently a basic preprocessor supporting only DEFINE and IF statements, which happened to be all I needed at the time. Usage works like this:

Input JavaScript

  //@define DEBUG 0

  var foo = function() {
    //@if DEBUG

    print "Hi";

Pass the above JavaScript through jsmacro from the command line like this: ./jsmacro.py -f infile.js > outfile.js (assuming the files are all in the same directory), and you get the following:

Output JavaScript

  var foo = function() {

    print "Hi";

The tool has registered the variable ‘DEBUG’ as 0 (i.e., false), so the conditional include statements omit the alert() calls. If DEBUG had been set to 1 (i.e., true), the alert() statements would remain (though all jsmacro instructions would be removed either way.)

One of the tricky things about doing macros or preprocessing in JavaScript is that I wanted the code to be valid JavaScript before the tool is run (which is why C-preprocessors won’t work.) The idea is that you develop as you normally would, but wrap your debug and testing code in conditional jsmacro statements so that they are automatically removed as part of your build process.

There’s nothing fancy about the current implementation (it’s a crude state machine that scans line-by-line, top-to-bottom looking for regex patterns and deciding whether to output the line of not.) Crude as it may be though, it completely solved a problem for me, and hopefully it will help you out as well.

Announcing tinyLinkFeed — RSS feeds for links posted to Twitter

I’ve come to enjoy the link-sharing that happens on Twitter, but a few months back I found myself with less time to regularly scan my Twitter stream.

Since I don’t always need real-time links, devising a mechanism to time-shift, and read at my leisure, seemed appropriate. Furthermore, since I don’t mind following friend’s del.icio.us feeds with a feed reader, converting tweets-with-links into RSS feeds seemed like a natural fit. After some brainstorming with @karllong and a bit of hacking, tinyLinkFeed was born!

tinyLinkFeed is designed to aggregate microblogging streams (like Twitter), and expose RSS feeds for posts containing URLs. To make the feeds a bit more useful, tinyLinkFeed also resolves shortened URLs, so you know what you’re linking to. (The URL un-shortening is done by another webservice I wrote at the same time, called tinyexpander; but I’ll save those details for another post.)

Using tinyLinkFeed is pretty straight-forward. For example, my normal Twitter page can be found at http://twitter.com/smartt, so my tinyLinkFeed page lives at http://www.tinylinkfeed.com/twitter/smartt. To follow the links I post to Twitter, you’d point your RSS reader to http://www.tinylinkfeed.com/feed/twitter/smartt.xml. For something perhaps more useful, you might try Tim O’Reilly’s link feed: http://www.tinylinkfeed.com/feed/twitter/timoreilly.xml (which is actually one of the main reasons I wrote tinyLinkFeed.)

The application came together fairly quick (it’s written in Python and runs on Google App Engine), though I haven’t had time to expand it since launch. I have a few ideas on what to add next, but I’m setting up getSatisfaction so you can leave ideas as well.

Adding profiles to tinyLinkFeed’s aggregator isn’t openly available yet, so if you’d like a profile added, drop a message to @smartt or @tinylinkfeed. I hope you find it useful!

How-To: Dynamic WWW-Authentication (.htaccess style) on Google App Engine

Sometimes classic Basic Access Authentication is the right approach to password protecting a webpage. It’s not secure from sniffing, but functional if you’re just trying to ward off the casual surfer in the wrong spot. (For example, restricting access to your cat pictures, not your missile silo codes.)

Basic authentication is often added to sites (or directories) using a .htaccess file and something like this:

AuthUserFile /home/foo/.htpasswd
AuthName "Private Area"
AuthType Basic

<Limit GET>
require valid-user

…but you can also do basic authentication on-the-fly by reading/writing HTTP Headers. To ask the browser for a user/password, you can raise a 401 error, and write a “www-Authenticate” header containing something like ‘Basic realm=”Secure Area”‘. To read the user/password, look for an Authorization header, grab it’s value, Base 64 decode it, and you should have a string in the form of “user:password”.

Here’s how you might handle it with Google App Engine. (Well, really you might use a decorator.. but this example is easier to explain.)

class AuthTest(webapp.RequestHandler):
  def get(self):

    # Wrapping in a huge try/except isn't the best approach. This is just 
    # an example for how you might do this.
      # Parse the header to extract a user/password combo.
      # We're expecting something like "Basic XZxgZRTpbjpvcGVuIHYlc4FkZQ=="
      auth_header = self.request.headers['Authorization']

      # Isolate the encoded user/passwd and decode it
      auth_parts = auth_header.split(' ')
      user_pass_parts = base64.b64decode(auth_parts[1]).split(':')
      user_arg = user_pass_parts[0]
      pass_arg = user_pass_parts[1]
      checkAuth(user_arg, pass_arg) # have this call raise an exception if it fails

      self.response.out.write(template.render('templates/foo.html', {}))

    except Exception, e:
      logging.debug("AuthTest Exception: %s" % (e))

      # Here's how you set the headers requesting the browser to prompt
      # for a user/password:
      self.response.set_status(401, message="Authorization Required")
      self.response.headers['WWW-Authenticate'] = 'Basic realm="Secure Area"'

      # Rendering a 401 Error page is a good way to go...
      self.response.out.write(template.render('templates/error/401.html', {}))

That’s all there is to it.

If you want to programatically write an Authorization header (as in, sending authentication credentials to another site, like the Twitter API’s, for example) you’ll do something like this:

request = urllib2.Request(url)
request.add_header('Authorization', "Basic %s" % (base64.b64encode("%s:%s" % (user, password))))


Conway’s Game of Life in Nodebox

I was reading Ben Fry‘s thesis, Organic Information Design yesterday, came across the section on Conway’s Game of Life, and thought it would make a nice NodeBox demo.

Here it is: conway-life.py

Nodebox screenshot

There’s not much to it, but it does show a software pattern I’ve been using frequently with NodeBox. Many of the NodeBox examples make heavy use of non-namespaced, global variables. I suppose it makes simple code easy to read for those new to programming, but it’s a habit you’ll want to break before your code starts getting more complex.

What I’ve found helpful is to create a World/Universe/Controller/Stage object that drives the rendering. Instead of using multiple globals in draw(), the controller object keeps the main parameters as local properties, and instantiates any needed objects in it’s __init__(). This approach prevents global variables names from clashing, and allows for creative reuse of rendering components.