Mercurial > hg > camir-aes2014
annotate toolboxes/FullBNT-1.0.7/KPMtools/cross_entropy.m @ 0:e9a9cd732c1e tip
first hg version after svn
| author | wolffd |
|---|---|
| date | Tue, 10 Feb 2015 15:05:51 +0000 |
| parents | |
| children |
| rev | line source |
|---|---|
| wolffd@0 | 1 function kl = cross_entropy(p, q, symmetric) |
| wolffd@0 | 2 % CROSS_ENTROPY Compute the Kullback-Leibler divergence between two discrete prob. distributions |
| wolffd@0 | 3 % kl = cross_entropy(p, q, symmetric) |
| wolffd@0 | 4 % |
| wolffd@0 | 5 % If symmetric = 1, we compute the symmetric version. Default: symmetric = 0; |
| wolffd@0 | 6 |
| wolffd@0 | 7 tiny = exp(-700); |
| wolffd@0 | 8 if nargin < 3, symmetric = 0; end |
| wolffd@0 | 9 p = p(:); |
| wolffd@0 | 10 q = q(:); |
| wolffd@0 | 11 if symmetric |
| wolffd@0 | 12 kl = (sum(p .* log((p+tiny)./(q+tiny))) + sum(q .* log((q+tiny)./(p+tiny))))/2; |
| wolffd@0 | 13 else |
| wolffd@0 | 14 kl = sum(p .* log((p+tiny)./(q+tiny))); |
| wolffd@0 | 15 end |