lecture note 3
`
Since is very small, from the right-hand side of the inequality we have
Definition(From the textbook): the relative entropy between two probability mass functions p(x) and q(x) is defined as
.(divergence)
if ||p-q|| we have
the number of typical
Question :
to prove
by l'Hôpital's rule, we have
the number of the typical type-class and the number of the typical element
the size of largest set class
Expected # bits (the typical part + the atypical part)
Comments (0)
You don't have permission to comment on this page.