• If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

• You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!

View

# Spring 2009 IERG5154 Information Theory (redirected from Spring 2009)

last edited by 12 years, 5 months ago

# Welcome to the IEG 5154 Information Theory Course Wiki!

This wiki is dedicated to the course IEG 5154, Information Theory, of the Department of Information Engineering of The Chinese University of Hong Kong. The content is to be collaboratively created and edited by the students, tutor, and instructor of this course. This wiki is a primary resource for this class.

## Lectures/Problem Sets/Homeworks/...:

### Author

1-3 Homework 1

4-6 Homework 2

18 Jan

5 Jan

5 Jan

9 Jan

12 Jan

16 Jan

16 Jan

19 Jan

6 Feb

22 Jan

7 Feb

10 Feb

16 Feb

23 Jan

4 Feb

14 Feb

23 Feb

23 Feb

2 Mar

6 Mar

6 Mar

9 Mar

20 Mar

23 Mar

27 Mar

9 Mar

27 Mar

27 Mar

17 Apr

17 Apr

Basic probability theory

A simple (source) coding theorem

A simple (source) coding theorem

A simple (source) coding theorem

A simple (source) coding theorem

Entropy: Definitions and Properties

Entropy: Definitions and Properties

Entropy: Definitions and Properties

Fano's inequality

Source/Channel Coding Theorems

Source Coding Theorem: Achievability

Channel Coding Theorem

Source coding Theorem and Symbol-by-symbol Source Coding

Based on PS1-3. Due 5th Feb.

Symbol-by-symbol Source Coding

Symbol-by-symbol Source Coding

Processes with memory

Memory/Universal (arithmetic) codes/Weak typicality/Differential entropy

Arithmetic Codes and Weak Typicality

Differential Entropy

Typicality of Differential Entropy/Error Estimation/Some Channel Models

Coding theorems and coding theory

A note on Concatenated code

Channel Feedback

Source channel seperation

Water Filling Method

Based on PS4-6. Due 23rd Mar.

Quantization/Rate distortion theory

Quantization

Rate distortion

(The same as Scribe Notes 7.2? But shouldn't Scribe Notes 7.2 be about

Lloyd's algorithm etc?)

Sid

Sid

Lingxiao

Yan Kei

Tong

Sid

Silas

Yuchen

Harry

Sid

Ting Kam

Liu Ke

Ting Kam

Sid

Michael

Yuchen

Ken

Sid

Lingxiao

Silas

Yuchen

Sid

Tong

Silas

Liu Ke

Liu Ke

Tong

Lingxiao

Sid

Sid

Lingxiao

Homework 3 7 Apr Due 20th Apr. Sid

## Important Announcements:

### Details

1

Jan 7

Change of Lecture Venue

Michael

The venues for Monday and Friday lectures will be ELB 202 and ERB 703 respectively.

Details have been updated here.

2. Jan 15 HW1 released on Jan 23d Sid HW1 will be released next week. Two weeks to submit.
3. Jan 15 Classes 9th/13th Feb handled by TA Michael Sid

I'll be out of town. The TA will do a refresher of material already covered, and also discuss

related topics in a bit more detail (possibly some source coding algorithms).

4. Jan 18

23rd Jan class preponed to

22nd Jan

Sid

Special pre-CNY arrangement. Class on Thursday, 1630-1800, instead of on Friday just before CNY. Room 1106, EB-II

5.

Jan 18

Notes on Basic Probability

6. Jan 23 Problem Set 1 Michael Problem set 1 has been graded. Please come to my office to obtain your work.
7. Jan 23 Homework 1 Michael The due date of Homework 1 is Feb 5th.
8. Jan 23 Change of Class Michael The class on Fri, 6th Feb, will again have to be preponed to Thurs, 5th Feb (due to IE board meeting Prof. Jaggi has to attend). Exact time/class TBD in class on Monday, 2nd Feb.
9. Feb 2 Change of Class Michael The class on Fri, 6th Feb, will be preponed to Thurs, 5th Feb. The time will be 1530-1630. In addition, the class on Mon, 16th Feb, will be postponed to Thurs, 19th Feb. The time will be 1530-1630. Both classes will be in SHB 833.
10. Feb 5 Class Cancelation Michael The class on Mon, 16th Feb, will be canceled because Prof. Jaggi will be out of town.
11.  Feb 20 Class Cancelation Michael The class on Fri, 27th Feb, will be canceled.
12. Feb 20 Class Extension Michael The Friday classes starting from 6th Mar will be extended by 30 minutes (15:30-17:30).
13. Mar 9 Homework 2 Sid Homework 2 released Mon Mar 9th, due Monday Mar 23rd. Friday Mar 27th.
14. Apr 8 Make-up Class Michael There will be a make-up class at 11:30am-1pm on Apr 20th. The classroom will be ELB 203.
15. Apr 17 Final Exam Michael The final exam will commence at 12:00, Apr 30th and end at 12:00, May 1st. Questions will be posted in the wiki when the exam begin and please email the scanned solution by the deadline. Students can come to Prof. Jaggi or Michael's office to raise any questions about the paper at 17:30-18:30 on Apr 30th. Last-minute questions can be raised through email at 11:00-12:00 on May 1st (delay may happen). Class wiki and textbooks are opened.

## Scribe Notes Schedule:

 Week Monday Friday 1 Xia Lingxiao (Jan 5th) Mak Yan Kei (Jan 9th) 2 Liang Tong (Jan 12th) Silas Fong (Jan 16th) 3 Mei Yuchen (Jan 19th) Cheng Fan (Jan 22nd) 4 Wong Ting Kam (Feb 2nd) Wong Ting Kam (Feb 5th) 5 Liu Ke (Feb 9th) Mei Yuchen (Feb 13th) 6 Cancelled Leung Cheung Yu (Feb 20th) 7 Xia Lingxiao (Feb 23th) Cancelled 8 Silas Fong (March 2nd) Mei Yuchen (March 6th) 9 Liang Tong (March 9th) Wong Ting Kam + Leung Cheung Yu (March 13th) 10 Cheng Fan (March 16th) Liu Ke (March 20th) 11 Liang Tong (March 23rd) Xia Lingxiao (March 27th) 12 Liu Ke (March 30th)

## Free help:

1. Learn how to use PBwiki: The PBwiki Manual

2. If you prefer video, watch a recording of our popular webinar, PBwiki 101: Your Guide to Wiki Basics.