Proofs and Induction

A significant issue in designing proper computer procedures to solve problems is that of proving that they always work. There is a very large body of empirical evidence that just writing a program and testing it (even testing it on a very large number of examples) is not sufficient: formal proof is necessary. Examples given in this course will not all have a very obviously computerish feeling--I hope that when the DS&A course follows it you will see some of the applications of notation introduced here.

The first piece if notation to introduce here is the symbol `` $\Rightarrow$'' which I will pronounce ``implies''. The idea is that if you know that the thing on the left is true then you can deduce that the one on the right is. A major use of this is in the following construction:

If you know both A and $A \Rightarrow B$
Then you may deduce B

whatever the statements A and B are. Eg:

All cats are animals
and Arthur is a cat
therefore
Arthur is an animal

To bring the example better into line with the original formalism it is perhaps helpful to see ``All cats are animals'' as shorthand for standing for all possible statements of the form ``x is a cat $\Rightarrow$x is an animal'', with arbitrary words (including in particular including ``Arthur'') substituted for the marker x.

What you find is that the more you strain to make real-world examples precise the more murky they appear. Some of this is a direct reflection of the flexibility and imprecision of natural language. (A pretty example of the imprecision of English here is the statement `everyone loves someone'. Does this mean that every person has there own little favourite(s)? Or that there is some shared someone which everyone admires to the point of loving?) Often things that start off mathematical in style fit into the structure of formal proofs more easily.

The notation `` $A \Rightarrow B$'' can be read as ``If A then B'' or ``B is true if A is'' or ``whenever A is true it happens that Bis true''. So far we have use this in cases where we have some way of knowing that A is true. Another use of the same rule is when we can tell (from some other source) that B is false. It is then valid to deduce that A must be false also. For instance given that all cats are animals, knowing that the statement ``the moon is an animal'' is false assures us that the moon is not a cat. Wow! If you were a class of mediaeval philosophers you would know these rules as modus ponens and modus tolens, and we could talk about deduction using them as syllogisms. I hope the rules look like natural common sense, but it is worth noting that in common speech (especially of a political nature!) the rules are very often not adhered to. As an exercise try to spot examples where people try to carry forward an argument but break the formal rules of logic. Eg modified from a Thurber story

People who eat carrots and have long ears cause earthquakes
and there was just an earthquake
therefore
It was caused by the rabbits
therefore We wolves are justified in taking them into protective custody
...to protect ourselves

Some people would consider that the above line of argument illustrates various other bad forms of deduction, such as starting the argument off with an assertion that is false, and slightly adjusting the interpretation of ``protect'' part way through (etc).

On occasions it will also be useful to write things the other way around as in `` $B \Leftarrow A$'', which can perhaps unexpectedly be read as ``A only if B''. To see how, observe that A being true while B is false contradicts the implication. Finally $A \Leftrightarrow B$ is a shorthand for having both $A \Rightarrow B$and $A \Leftarrow B$, and to prove it you normally need to prove both the if and the only if parts--often separate proofs will be required for these two parts. A very common confusion is to mix up whether you are in the process of proving the $\Rightarrow$ or $\Leftarrow$ part.

Often the sorts of things you want to prove in computer science (and mathematics) will be general statements true for all cases of some condition. One approach to this is known as proof by induction. It comes in a number of variants, so I will deal with the simplest one first:

Suppose I want to prove that some result is true for all integer values of n, ie n=1, n=2, n=3, ...then I can start by proving the base case which will be the first one, n=1 (or n=0, since computer scientists often start counting at zero), and then showing that if the result is true for some particular value of n then it must also be true for the next step up (n+1).

As an example, consider the Tower of Hanoi problem--you have a set of graduated discs and three pegs. A larger disc may never be piled on top of a smaller one, and the discs live on the pegs. In a single move you may move one disc from the top of the pile on one peg to the top of the pile on another peg, but you must keep to the rule that large discs may never be placed above small ones. The result to be proved is that however many discs there are it is possible (starting with all discs piled in order on one peg) to make a sequence of moves that end up with the discs on another nominated peg.

The proof by induction goes as follows

Base case:

With one disc it is easy - just move it!

Induction step:

Suppose we can achieve the desired effect with k discs, and now we have k+1. Calls the discs X, Y and Z and suppose all discs start on X and are to be moved to Y. Observe that if we just ignore the bottom (largest) disc the remaining k discs are subject to the same rules of the Hanoi game, so by our induction hypothesis there is a sequence of moves that ends up with all of them on peg Z. From this state it is possible to move the largest disc from X to Y. For the remainder of the sequence again the largest disc can be ignored, and again moving the k discs from Z to Y can be done. In the end all discs are on peg Y in the desired order, so we have shown how to generate a sequence of steps that move k+1 discs.

The two parts above are sufficient to prove the general result. If one writes an abbreviation H_n for the statement ``The Hanoi puzzle can be solved if there are n discs'' then we proved H<sub>1</sub> directly, and then set up a chain $H_{1} \Rightarrow H_{2}$, then $H_{2} \Rightarrow H_{3}$, $H_{3} \Rightarrow H_{4}$ and so on for ever. So you see that the inductive step in the proof is just a way of building up (all at once) a chain of simple `` $\Rightarrow$'' deductions.

It is vital in proofs by induction to have a base case. In many examples that you will come across it will be incredibly trivial, but it is still important to write it down.

In the above example the inductive step took the view that when proving stage n it was legal to assume that stage n-1 was true². An alternative variant on the induction idea is that when proving stage n you assume that the result has already been proved for all values less than n, not just the special value n-1.

A lion may be asleep somewhere in a desert, can we find it or discover that it is absent?

Base case:

If the desert has area less than 10 square metres then we can find the lion [since this example is slightly a joke I am going to assume that this remark is ``obviously'' true].

Inductive step:

Divide the desert into two sub-deserts each of equal area. Call these N and S (or E and W if you like). Observe that their area is in each case smaller than that of the whole desert. Thus by an induction hypothesis we can find the lion (or its absence) in each sub-desert. If we find the lion in one part we report success, if it is not found in either sub-desert it is utterly absent.

This proof needs one more crucial component--an argument that if we take any desert at all and keep dividing it in two we will always end up with a region of sand that has area less than 10 square metres. The base case here is in fact a bit questionable, especially if you imagine a large desert always being split strictly North/South so that the 10 square metre base case is in fact a very very long but very very thin strip of territory. Such concerns do not damage the structure of the inductive proof, but should serve to warn you that all the fine details must be in place for a proof to be valid.

I will show one more example of a proof that uses a form of induction. The result I want to prove is that any arithmetic expression that uses just addition and multiplication and that has only even numbers written in it will evaluate to an even value. This will be structurally much closer to many of the proofs needed for DS&A methods. The inductive proof will be in terms of the number of operators in an expression.

Base case:

With no operators at all the expression is just a number, and by the statement of the problem that is an even one.

Inductive step:

If there is at least one operator then the whole expression is of the form $A \oplus B$where $\oplus$ will be either an addition or multiplication operator, and A and B are sub-expressions. I will treat things like 2+4+6 as (2+4)+6 here. There are then two cases to consider

The leading operator is +

Each of A and B will have fewer operators than the whole expression, hence by induction each will evaluate to an even number. The sum of two even numbers is even, hence the whole expression is even as required.

The leading operator is *

Similarly, since the product of two even numbers is even.