ECE 6520 - Information Theory

Concept of information and uncertainty; source and channel models; entropy and its properties; relative entropy; mutual information; Shannon's source coding theorem; the Asymptotic Equipartitioning Property (AEP); concepts of source codes; Huffman code; arithmetic coding; variable to fixed source codes; typical sequences; rate distortion theory; channel coding; Shannon's channel coding theorem.