mutual information

mutual information (usually uncountable, plural mutual informations)

1. (information theory) A measure of the entropic (informational) correlation between two random variables.

   Mutual information ${\displaystyle I(X;Y)}$ between two random variables ${\displaystyle X}$ and ${\displaystyle Y}$ is what is left over when their mutual conditional entropies ${\displaystyle H(Y|X)}$ and ${\displaystyle H(X|Y)}$ are subtracted from their joint entropy ${\displaystyle H(X,Y)}$. It can be given by the formula ${\displaystyle I(X;Y)=-\sum _{x}\sum _{y}p_{X,Y}(x,y)\log _{b}{p_{X,Y}(x,y) \over p_{X|Y}(x|y)p_{Y|X}(y|x)}}$.

   • 2018, Clarence Green, James Lambert, “Position vectors, homologous chromosomes and gamma rays: Promoting disciplinary literacy through Secondary Phrase Lists”, in English for Specific Purposes, →DOI, page 6:

     From these lists, all combinations of the four major parts of speech were extracted and each phrase was checked for the frequency, dispersion and range criteria respectively. Those that passed the criteria thresholds then had their mutual information scores computed using the tool Collocate (Barlow, 2004) and those failing to meet the threshold were removed.

• total correlation on Wikipedia.Wikipedia
• interaction information on Wikipedia.Wikipedia