• If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

• You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!

View

# lecture notes 3

View current version     Page history
Saved by
on January 18, 2009 at 7:59:33 pm

lecture note 3

`

Since  is very small, from the right-hand side of the inequality we have

Definition(From the textbook): the relative entropy between two probability mass functions p(x) and q(x) is defined as

.(divergence)

if ||p-q||   we have

the number of  typical

Question :

to prove

by l'HÃ´pital's rule, we have

the number of the typical type-class  and the number of the typical element

the size of largest set class

Expected # bits  (the typical part + the atypical part)

the typical set decrease, as the epsilon decrease.

p <1/2 and n =10 in the following figures

Fig1: the relation between #of heads and probability,

Fig2: between # of heads and size of T(k,n)

BSCT

(a) ,code with expected rate

(b)  code with and expected rate <