Channel Capacity When to Use Use this skill when working on channel-capacity problems in information theory. Decision Tree 1. Mutual Information - I(X;Y) = H(X) + H(Y) - H(X,Y) - I(X;Y) = H(X) - H(X|Y) = H(Y) - H(Y|X) - Symmetric: I(X;Y) = I(Y;X) - 2. Channel Model - Input X, output Y, channel P(Y|X) - Channel matrix: rows = inputs, columns = outputs - Element (i,j) = P(Y=j | X=i) 3. Channel Capacity - C = max {p(x)} I(X;Y) - Maximize over input distribution - Achieved by capacity-achieving distribution 4. Common Channels | Channel | Capacity | |---------|----------| | Binary Symmetric (BSC)…