Tips on Python

What are generators, and what are they for?

A generator is in some ways like an object with a next() function. Here are two ways to generate a sequence of Fibonnaci numbers.

class FibGenerator():
    def __init__(self):
        self.a = 0
        self.b = 1
    def next(self):
        res = self.b
        self.a,self.b = self.b,self.a+self.b
        return res

def fib():
    a,b = 0,1
    while True:
        yield b
        a,b = b,a+b

g1 = FibGenerator()
g2 = fib()
for i in range(10):
    print g1.next(), g2.next()

Under the hood, Python sees that the fib() function contains the yield keyword, so it realizes that fib() is a generator, so it gives it a next() method. Each time you call g2.next(), Python re-enters the fib() function and keeps on executing until it hits yield, whereupon it yields up that value. Next time you call g2.next() the generator will start executing again immediately after the yield statement.

Why bother with generators?

Why bother with generators when the same thing could just as well be done with an object? Here is an example where generators are more elegant. I want to generate a sequence (U₁, 1-U₁, U₂, 1-U₂,…) where the U_i are independent random variables.

class AlternatingGenerator():
    def __init__(self):
        self.lastx = None
    def next(self):
        if self.lastx is None:
            self.lastx = random.random()
            res = self.lastx
        else:
            res = 1-self.lastx
            self.lastx = None
        return res

def ralternating():
    while True:
        x = random.random()
        yield x
        yield 1-x

The object-based solution requires you to turn your program logic inside-out. The object has a toggle (whether or not self.lastx is None) and it uses this toggle state to remember whether it should generate a U_new or whether it should return 1-U_old. The generator on the other hand is written with `natural' program logic, and Python does the work of keeping track of the state.

The end of a generator

The two generators you've seen so far are capable of generating an infinite sequence of values. Python also supports generators for finite sequences. When you `fall off the end', it raises an exception.

def splitwords(words):
    for w in words.split(' '):
        yield w

>>> g = splitwords('Network performance is fun')
>>> print g.next()
Network
>>> print g.next()
performance
>>> print g.next()
is
>>> print g.next()
fun
>>> print g.next()
Traceback (most recent call last):
  File "", line 1, in 
StopIteration
>>>

This sort of generator is often used for for loops, and in list comprehensions (see below). The loop stops when the generator comes to its end.

def inputwords(prompt):
    try:
        while True:
            s = raw_input(prompt)
            if s=='Q': break
            elif len(s)==0: print 'Type Q to stop'
            else: yield s
    except EOFError:
        pass

>>> res = [i for i in inputwords(': ')]
: see
: Jane
: 
Type Q to stop
: run
: Q
>>> print res
['see', 'Jane', 'run']
>>>

Tuples and lists

A tuple is a fixed-length list, e.g. a pair. You indicate a tuple by putting several elements together with brackets around and commas in between (although actually the brackets are optional).

>>> p = (5,2)
>>> q = 5,2
>>> print p,'and',q
(5, 2) and (5, 2)
>>>

You can use a tuple on the left hand side of = to assign several values at the same time.

>>> a,b,c = 'hello there world'.split(' ')
>>> print a,'---',b,'---',c
hello --- there --- world
>>>

A list is variable-length and modifiable, whereas a tuple is fixed-length and immutable.

>>> x = ['I','can','fly']
>>> x.append('on a plane')
>>> print x
['I', 'can', 'fly', 'on a plane']
>>> 
>>> x = ('i','can','fly')
>>> x.append('on a plane')
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'tuple' object has no attribute 'append'
>>>

List comprehensions

A list comprehension is a syntactic shorthand for constructing a list by applying a function to elements of another list. Here are three ways to capitalize all words in a list; the last of these is called a list comprehension.

x = ['i','can','fly','on','a','plane']
y = []
for i in range(0,len(x)):
    word = x[i]
    word = word.upper()
    y.append(word)

y = []
for word in x:
    y.append(word.upper())

y = [word.upper() for word in x]

You can also put in an if clause to select only certain elements:

vowels = ('a','e','i','o','u')
y = [word.upper() for word in x if word.startswith(vowels)]