lecture note 3
`
Since is very small, from the right-hand side of the inequality we have
Definition(From the textbook): the relative entropy between two probability mass functions p(x) and q(x) is defined as
.(divergence)
if ||p-q|| we have
the number of typical
Question :
to prove
by l'HÃ´pital's rule, we have
the number of the typical type-class and the number of the typical element
the size of largest set class
Expected # bits (the typical part + the atypical part)
the typical set decrease, as the epsilon decrease.
p <1/2 and n =10 in the following figures
Fig1: the relation between #of heads and probability,
Fig2: between # of heads and size of T(k,n)
BSCT
(a) ,code with expected rate
(b) code with and expected rate <
Comments (0)
You don't have permission to comment on this page.