A Brief Introduction to Information Theory

Although the idea of a quantitative measure of information has been around for a while, the person who pulled everything together into what is now called information theory was Claude Elwood Shannon. An electrical engineer at Bell Labs. Shannon defined a quantity called self-information. Suppose we have an event A, which is a set of … Read more