Entropy When to Use Use this skill when working on entropy problems in information theory. Decision Tree 1. Shannon Entropy - H(X) = -sum p(x) log2 p(x) - Maximum for uniform distribution: H max = log2(n) - Minimum = 0 for deterministic (one outcome certain) - for discrete 2. Entropy Properties - Non-negative: H(X) = 0 - Concave in p - Chain rule: H(X,Y) = H(X) + H(Y|X) - 3. Joint and Conditional Entropy - H(X,Y) = -sum sum p(x,y) log2 p(x,y) - H(Y|X) = H(X,Y) - H(X) - H(Y|X) <= H(Y) with equality iff independent 4. Differential Entropy (Continuous) - h(X) = -integral f(x) log f(x) dx - Can b…