The average length of the code is not the only important point in designing a “good” code. Consider the following example adapted from [10]. Suppose our source alphabet consists of four letters a1, a2, a3, and a4, with probabilities P(a1)= 1/ 2 , P(a2) = 1 /4 , and P(a3) = P(a4) = 1 /8 . The entropy for this source is 1.75 bits/symbol. Consider the codes for this source in Table
Letters | Probability | Code 1 | Code 2 | Code 3 | Code 4 |
a1 | 05 | 0 | 0 | 0 | 0 |
a2 | 025 | 0 | 1 | 10 | 01 |
a3 | 0125 | 1 | 00 | 110 | 011 |
a4 | 0125 | 10 | 11 | 111 | 0111 |
Average length | 1.125 | 1.25 | 1.75 | 1.875 | |
alphabet
The average length l for for each code is given by
