Writing a simple memoization decorator in Python

I was generating some reports recently that involved accessing expensive object methods whose results were known to not change on subsequent calls; However, instead of using local variables, I sketched-out this quick memoization decorator to save method responses as variables on the object (using a leading ‘_’ followed by the method-name as the variable name):

def cache_method_results(fn):
    def _view(self, *args, **kwargs):
        var_name = '_{n}'.format(n=fn.__name__)

        if var_name in self.__dict__:  # Return the copy we have
            return self.__dict__[var_name]

        else:  # Run the function and save its result
            self.__dict__[var_name] = fn(self, *args, **kwargs)
            return self.__dict__[var_name]

    return _view

You might use it like this:

class Foo(object):
    def some_expensive_operation(self):
        ...calculate something big and unchanging...
        return results

f = Foo()
print(f.some_expensive_operation())  # This first call will run the calculation
print(f.some_expensive_operation())  # but this one will used the cached result instead

It’s not rocket science, but these little tricks add to the fun of using Python.

“Fun example of programming language sc…

“Fun example of programming language scope” is only “fun” for a certain type of geek; But I like programming examples that help explain how your code is interpreted, particularly if the lesson can help prevent a certain class of bug.

Now that you’re expecting a scope puzzle, what will the following JavaScript print? (Ignoring the line numbers, of course, which are here to aid in discussion.)

 1:  var foo = 1;
 3:  function bar1() {
 4:   print("A: " + foo );
 5:  }
 7:  function bar2() {
 8:   print("B: " + foo );
 9:   var foo = 2;
10:   print("C: " + foo );
11:  }
13:  function bar3() {
14:    print("D: " + foo );
15:    eval("var foo = 2;")
16:    print("E: " + foo );
17:  }
19:  bar1();
20:  bar2();
21:  bar3();


No peeking…



A: 1
B: undefined
C: 2
D: 1
E: 2


“A” is easy. Since foo is a “free variable” (i.e., not defined within function bar1), the interpreter goes up the scope chain, and finds the global foo, defined on line 1.

For bar2, “C: 2” is obvious — it’s “B: undefined” that lets you in on the magic under the hood. You sort of expect to see “B: 1” (or a compiler error.) However, JavaScript interpreters scan-ahead, searching for variable definitions (e.g., var statements) when parsing a code block. The interpreter sees/re-writes bar2 like this:

 1:  function bar2() {
 2:    var foo;
 3:    print("B: " + foo );
 4:    foo = 2;
 5:    print("C: " + foo );
 6:  }

With that definition, “B: undefined” makes perfect sense.

To short-circuit the magic, bar3 uses eval() to do it’s trickery. At line 14, foo still points to the global foo, much like in bar1; However, the eval statement on line 15 modifies the local scope, introducing a new, local foo. By line 16, “E: 2” is using the newly introduced foo.


The lesson: Even though JavaScript allows you to declare variables at any point within a block, putting your var statements at the beginning of the block can help eliminate scope confusion around whether an inner- or outer-closure contains the correct value.


Bonus Question

Is JavaScript’s var a let or let*? It’s easy to find out using the following:

 1:  var x = 2, y = 3, z = x + y;
 2:  print(z);

Is this legal? Will it print ‘5’?

[Update: 2010/09/21: If you liked this, you’ll also enjoy “JavaScript Scoping and Hoisting“.]

jsmacro 0.2.3

The latest jsmacro (v0.2.3) adds support for “else” clauses to “if”, “ifdef”, and “ifndef” statements. Combine this with the command-line variable definition support and you can now do fun things like this:

//@ifdef IE6_BUILD
 ...custom IE6 code here
 ...code for other browsers here

Of course, this goes against the idea that your JavaScript would remain usable for development without needing to be processed, but it’s just an example. Longer term, I hope to have a different approach available that will allow conditional code substitution so that browser specific optimizations won’t get in the way of an easy development/test/debug process.

jsmacro 0.2.2

jsmacro 0.2 was a full rewrite (because version 0.2’s are always a full rewrite.) It’s now a little closer to what I was originally thinking. Instead of a line-by-line state machine, the parser now uses regex, and dynamically calls macro-handling methods based on the name of the macro. That’s a little vague, but in practice it means that extending the macro language is easier, and it may be possible to do it on-the-fly (as in, writing new macro implementations within the JavaScript source file that’s being parsed — which is a geeky goal I’m going for.)

Other new additions:

  • Test files are now picked-up automatically when named correctly. This makes it painless to add more tests.
  • Added support for setting DEFINE flags from the command-line. Handy if you automate builds for different environments (like IE6 vs. the rest of the world.)
  • Added support for #ifdef and #ifndef

The next big hurdles will be how to handle else statements, and coming up with a reason to implement some type of #inline capability.

jsmacro — an oddly named JavaScript preprocessor

For awhile now I’ve wanted a JavaScript preprocessor to conditionally include debug and testing code when needed. It’s always registered as merely a “nice to have”, so I hadn’t sought one out. However, I had a little time over the weekend and wanted to play with the idea, so here it is: jsmacro (on GitHub.)

[Note that before writing this I did seek out existing implementations, and found js-preprocess to be the most interesting; However, I needed something that would work as part of an existing build chain, so authoring the tool in Python instead of JavaScript made more sense.]

Currently, jsmacro is poorly named, as I didn’t write the macro system that was in my head. Instead, it’s currently a basic preprocessor supporting only DEFINE and IF statements, which happened to be all I needed at the time. Usage works like this:

Input JavaScript

  //@define DEBUG 0

  var foo = function() {
    //@if DEBUG

    print "Hi";

Pass the above JavaScript through jsmacro from the command line like this: ./jsmacro.py -f infile.js > outfile.js (assuming the files are all in the same directory), and you get the following:

Output JavaScript

  var foo = function() {

    print "Hi";

The tool has registered the variable ‘DEBUG’ as 0 (i.e., false), so the conditional include statements omit the alert() calls. If DEBUG had been set to 1 (i.e., true), the alert() statements would remain (though all jsmacro instructions would be removed either way.)

One of the tricky things about doing macros or preprocessing in JavaScript is that I wanted the code to be valid JavaScript before the tool is run (which is why C-preprocessors won’t work.) The idea is that you develop as you normally would, but wrap your debug and testing code in conditional jsmacro statements so that they are automatically removed as part of your build process.

There’s nothing fancy about the current implementation (it’s a crude state machine that scans line-by-line, top-to-bottom looking for regex patterns and deciding whether to output the line of not.) Crude as it may be though, it completely solved a problem for me, and hopefully it will help you out as well.

Detecting whether your code is running on App Engine servers, or in development

For one of my projects running on Google App Engine, I’m using Google Analytics to track traffic, but I didn’t want the tracking code showing up in my local development environment. Fortunately, detecting where your code is running is easy.

Take a look at this sample:

import os

  is_dev = os.environ['SERVER_SOFTWARE'].startswith('Dev')
  is_dev = False
is_prod = not(is_dev)

If the is_dev and is_prod variables are exposed to your templates, you can do something like this:

{% if is_prod %}
 {% include "analytics.html" %}
{% endif %}

Hope this helps!