ECE 5620

Information Theory and Generative Modeling

Course Description

Course information provided by the 2026-2027 Catalog.

A graduate-level introduction to information theory, data compression, and generative modeling. An introduction to information measures: entropy, mutual information, relative entropy, differential entropy, and their properties. Lossless compression and its connection to prediction and generative modeling. The Minimum Description Length (MDL) principle in model selection. Practical lossless compression using arithmetic coding. The rate-distortion theorem and its connection to lossy compression standards such as JPEG, mp3, and AAC as well as generative modeling techniques such as autoencoders and variational inference. The Nonlinear Transform Coding framework. Practical methods for lossy compression such as Trellis-Coded Quantization (TCQ) and entropy-constrained dithered quantization.

Prerequisites ECE 4110 or equivalent.

Enrollment Priority Intended for: M.Eng. students and advanced undergraduates with assignments focused on the development of practical data compression algorithms.

Last 4 Terms Offered 2024FA, 2021SP, 2019SP, 2018SP

View Enrollment Information