Mutual Information, Visually Explained

Описание: In this video, we build an intuitive understanding of mutual information from three complementary perspectives: missing randomness, uncertainty reduction, and distance from independence.

Timestamps

00:00 Missing randomness
03:48 Uncertainty reduction
06:44 Distance from independence

Topics
Mutual information
Information theory
Entropy
Joint entropy
Conditional entropy
KL divergence
Shannon entropy
Probability
Joint distribution
Conditional probability
Independence