| 
View
 

lecture notes 3

This version was saved 15 years, 11 months ago View current version     Page history
Saved by Tong
on January 18, 2009 at 7:58:14 pm
 

lecture note 3

      Formula

`      Formula

Since Formula is very small, from the right-hand side of the inequality we have Formula 

          Formula

 

Definition(From the textbook): the relative entropy between two probability mass functions p(x) and q(x) is defined as

Formula.(divergence)

 

 

if ||p-q||  Formula we have Formula 

 

the number of  typical Formula 

               Formula

               Formula

Question :                Formula

               to prove   Formula

 

by l'Hôpital's rule, we haveFormula   

                                  Formula  

 

the number of the typical type-class Formula and the number of the typical element Formula

the size of largest set class Formula  

Expected # bits  (the typical part + the atypical part)

 Formula

the typical set decrease, as the epsilon decrease.

 

p <1/2 and n =10 in the following figures

Fig1: the relation between #of heads and probability,

Fig2: between # of heads and size of T(k,n)

 

 

BSCT

(a) ,code with expected rate

(b)  code with and expected rate <

 

 

 

Comments (0)

You don't have permission to comment on this page.