The newest version of the lecture notes includes a section on high(er) performance Python. Read that rather than this short post.

Original post

Python code can be very slow or very fast. For loops are slower than list comprehensions, which are much much slower than numpy calls or built in python functions. The latter two use optimized Fortran and C libraries. So the first rule of thumb is, whenever you find yourself writing a for loop or list comprehension, check if there is a built-in Python or numpy function that does the same thing.

The above rule always holds since built in functions will lead to simpler code (remember the importance of simplicity). However, there are often other optimizations that lead to slightly harder to read or more complicated code. To address this, first consider the following quote, credited to Donald Knuth, "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil." Then solution is to profile your code first, to determine where the slow spots are, and then only re-write the parts that are slowing your code down.

To profile scientific code, you need a line-by-line readout of the time taken in different function calls in your code. This can be had by using the line_profiler in conjunction with the kernprof script. To install, simply type

pip install line_profiler

Then, at the top of a function that you want to profile (you can only profile functions), put @profile. For example:

def myfun(x):
    y = 2 * x
    return y

Then you need some way to call myfun from the command line. This could be for example a script, which could be as simple as:

from mymodule import myfun


Then, at the command line, type -l

If you are using anaconda and have installed line_profiler, then will be in your PATH, and the above line will work. The above line will produce the file This is the profiler output. You need to use the module line_profiler to read it. To do this, type

python -m line_profiler  |  less

You should see a line-by-line breakdown of time taken to run your code.

blog comments powered by Disqus


12 May 2013


Subscribe to RSS Feed