This wiki is dedicated to the course IEG 5154, Information Theory, of the Department of Information Engineering of The Chinese University of Hong Kong. The content is to be collaboratively created and edited by the students, tutor, and instructor of this course. This wiki is a primary resource for this class.
For details of this course, please click here.
Item |
Date |
Topic |
Author |
1.0 Problem Set 1 1.1 Scribe Notes 1.1 1.2 Scribe Notes 1.2 1.3 Scribe Notes 1.3
2.0 Problem Set 2 2.1 Scribe-Notes 2.1 2.2 Scribe Notes 2.2 2.3 Scribe Notes 2.3
3.0 Problem Set 3 3.1 Scribe Notes 3.1 3.2 Scribe Notes 3.2 3.3 Scribe Notes 3.3
1-3 Homework 1 4.0 Problem Set 4 4.2 Scribe Notes 4.1 4.2 Scribe Notes 4.2
5.0 Problem Set 5 5.1 Scribe Notes 5.1 5.2 Scribe Notes 5-2 5.3 Scribe Notes 5.3
6.0 Problem Set 6 6.1 Scribe Notes 6.1 6.3 Scribe Notes 6.2 6.3 Scribe Notes 6.3 6.4 Scribe Notes 6.4 6.5 Scribe Notes 6.5 4-6 Homework 2
7.0 Problem Set 7 7.1 Scribe Notes 7.1 7.2 Scribe Notes 7.2 7.3 Scribe Notes 7.3 |
18 Jan
5 Jan 5 Jan 9 Jan 12 Jan
16 Jan 16 Jan 19 Jan 6 Feb
22 Jan 7 Feb 10 Feb 16 Feb
23 Jan 4 Feb 14 Feb
23 Feb 23 Feb 2 Mar 6 Mar
6 Mar 9 Mar 20 Mar
23 Mar 27 Mar
9 Mar
27 Mar 27 Mar 17 Apr 17 Apr |
Basic probability theory
A simple (source) coding theorem A simple (source) coding theorem A simple (source) coding theorem A simple (source) coding theorem
Entropy: Definitions and Properties Entropy: Definitions and Properties Entropy: Definitions and Properties Fano's inequality
Source/Channel Coding Theorems Source Coding Theorem: Achievability Channel Coding Theorem Source coding Theorem and Symbol-by-symbol Source Coding
Based on PS1-3. Due 5th Feb. Symbol-by-symbol Source Coding Symbol-by-symbol Source Coding Processes with memory
Memory/Universal (arithmetic) codes/Weak typicality/Differential entropy Arithmetic Codes and Weak Typicality Differential Entropy Typicality of Differential Entropy/Error Estimation/Some Channel Models
Coding theorems and coding theory
A note on Concatenated code Channel Feedback Source channel seperation
Water Filling Method
Based on PS4-6. Due 23rd Mar.
Quantization/Rate distortion theory Quantization Rate distortion (The same as Scribe Notes 7.2? But shouldn't Scribe Notes 7.2 be about Lloyd's algorithm etc?) |
Sid
Sid Lingxiao Yan Kei Tong
Sid Silas Yuchen Harry
Sid Ting Kam Liu Ke Ting Kam
Sid Michael Yuchen Ken
Sid Lingxiao Silas Yuchen
Sid Tong Silas Liu Ke Liu Ke Tong Lingxiao
Sid
Sid Lingxiao |
Homework 3 | 7 Apr | Due 20th Apr. | Sid |
Item |
Date |
Topic |
Author |
Details
|
1
|
Jan 7 |
Change of Lecture Venue
|
Michael |
The venues for Monday and Friday lectures will be ELB 202 and ERB 703 respectively. Details have been updated here. |
2. | Jan 15 | HW1 released on Jan 23d | Sid | HW1 will be released next week. Two weeks to submit. |
3. | Jan 15 | Classes 9th/13th Feb handled by TA Michael | Sid |
I'll be out of town. The TA will do a refresher of material already covered, and also discuss related topics in a bit more detail (possibly some source coding algorithms). |
4. | Jan 18 |
23rd Jan class preponed to 22nd Jan |
Sid |
Special pre-CNY arrangement. Class on Thursday, 1630-1800, instead of on Friday just before CNY. Room 1106, EB-II |
5.
|
Jan 18 |
added |
Sid | Given widespread confusion about basic probability terminlogoy, added notes here - please try the exercises therein. |
6. | Jan 23 | Problem Set 1 | Michael | Problem set 1 has been graded. Please come to my office to obtain your work. |
7. | Jan 23 | Homework 1 | Michael | The due date of Homework 1 is Feb 5th. |
8. | Jan 23 | Change of Class | Michael | The class on Fri, 6th Feb, will again have to be preponed to Thurs, 5th Feb (due to IE board meeting Prof. Jaggi has to attend). Exact time/class TBD in class on Monday, 2nd Feb. |
9. | Feb 2 | Change of Class | Michael | The class on Fri, 6th Feb, will be preponed to Thurs, 5th Feb. The time will be 1530-1630. In addition, the class on Mon, 16th Feb, will be postponed to Thurs, 19th Feb. The time will be 1530-1630. Both classes will be in SHB 833. |
10. | Feb 5 | Class Cancelation | Michael | The class on Mon, 16th Feb, will be canceled because Prof. Jaggi will be out of town. |
11. | Feb 20 | Class Cancelation | Michael | The class on Fri, 27th Feb, will be canceled. |
12. | Feb 20 | Class Extension | Michael | The Friday classes starting from 6th Mar will be extended by 30 minutes (15:30-17:30). |
13. | Mar 9 | Homework 2 | Sid | Homework 2 released Mon Mar 9th, due Monday Mar 23rd. Friday Mar 27th. |
14. | Apr 8 | Make-up Class | Michael | There will be a make-up class at 11:30am-1pm on Apr 20th. The classroom will be ELB 203. |
15. | Apr 17 | Final Exam | Michael | The final exam will commence at 12:00, Apr 30th and end at 12:00, May 1st. Questions will be posted in the wiki when the exam begin and please email the scanned solution by the deadline. Students can come to Prof. Jaggi or Michael's office to raise any questions about the paper at 17:30-18:30 on Apr 30th. Last-minute questions can be raised through email at 11:00-12:00 on May 1st (delay may happen). Class wiki and textbooks are opened. |
Week | Monday | Friday |
1 | Xia Lingxiao (Jan 5th) | Mak Yan Kei (Jan 9th) |
2 | Liang Tong (Jan 12th) | Silas Fong (Jan 16th) |
3 | Mei Yuchen (Jan 19th) | Cheng Fan (Jan 22nd) |
4 | Wong Ting Kam (Feb 2nd) | Wong Ting Kam (Feb 5th) |
5 | Liu Ke (Feb 9th) | Mei Yuchen (Feb 13th) |
6 | Cancelled | Leung Cheung Yu (Feb 20th) |
7 | Xia Lingxiao (Feb 23th) | Cancelled |
8 | Silas Fong (March 2nd) | Mei Yuchen (March 6th) |
9 | Liang Tong (March 9th) | Wong Ting Kam + Leung Cheung Yu (March 13th) |
10 | Cheng Fan (March 16th) | Liu Ke (March 20th) |
11 | Liang Tong (March 23rd) | Xia Lingxiao (March 27th) |
12 | Liu Ke (March 30th) |
1. Learn how to use PBwiki: The PBwiki Manual
2. If you prefer video, watch a recording of our popular webinar, PBwiki 101: Your Guide to Wiki Basics.
3. Need more help? Sign up for a Free introductory webinar