Estimating Error Probabilities

Statistics and Simulation (Part 2) 2015


In this part, we consider a practical problem from coding theory and communications. What is the error probability when a digital message is transmitted over a communications channel? The answer builds on two pieces of fundamental theory. From probability theory we need binomial distributions, and from statistics, we need estimation techniques.

All the classroom sessions revolve around one concrete example, with a specific code and a specific channel. We illuminate this problem using both statistical analysis, probability theory, and simulation. After you have seen the theory in practice, we hope you will find it relatively easy to learn further details from textbooks and video.

Week 4-5: 19 January - 4 February
Exercises and learning methods

The classroom sessions will focus on the exercises as posted under each session in the table below.

At the end of the two-week period, you will need to submit the mandatory coursework. You will be able to reuse much of the material from the session exercises.

You will have to manage your own time and progress, using the available material, including video lectures, exercise sheets, textbook, google, classroom help etc. as best you can, to suit your preferred learning style.

Mandatory Coursework

Mandatory coursework is due Monday 2 February 8:15 am 2015.

  1. m-files for the Hamming code: encoder, decoder (These are quick and dirty implementations of a simple code with simple decoder.)
  2. m-files for the [31,11] BCH code: encoder, decoder (These use the comm's toolbox, and are merely wrappers using specific code parameters.)
  3. Assignment for the mandatory coursework.
  4. Additional exercises This is the complete exercises for this part 2014, and it includes more exercises than the session-wise exercise sheets of this year.
Session 1 (Wednesday 21 January 2015) Videos Time Slides Reading
Exercises The Binary Symmetric Channel (A Bernouilli Trial) MPEG4 / OGG 5:53 PDF
Words on the Channel MPEG4 / OGG 4:40 PDF
Random Binary Vectors (Matlab Demo) MPEG4 / OGG 5:45 summary help rand in Matlab
Session 2 (Thursday 22 January 2015) Videos Time Slides Reading
Exercises The Binomial Distribution (Error Words on the BSC) MPEG4 / OGG 7:59 clean
Frisvold and Moe pp. 100-104.
Expected Value for the Binomial Distribution MPEG4 / OGG 4:16 PDF
Variance for the Binomial Distribution MPEG4 / OGG 5:04 PDF
The Binomial Distribution in Matlab (Probability Distribution Function) MPEG4 / OGG 5:52 summary Matlab help: pdf, plot, bar, figure, hold
Comparing Probability Distributions (The Binomial Distribution in Matlab II) MPEG4 / OGG 6:34 summary
Cummulative Distribution Function (The Binomial Distribution in Matlab III) MPEG4 / OGG 7:51 summary Frisvold and Moe pp. (55), 56-59, Ā«vanlige forkortelserĀ» p. 61; help cdf in Matlab
Session 3 (Friday 23 January 2015) Videos Time Slides Reading


This session builds on Session 1, but can be done before Session 2 with little difficulty.

The Hamming Code (A little coding theory) MPEG4 / OGG 8:55 clean
Finding Error Probabilities (Monte Carlo Simulation) MPEG4 / OGG 8:35 clean
Shiflet and Shiflet pp. 358-360(+)
Decoding Error Probabilities (A Case for Estimation) MPEG4 / OGG 9:03 clean
Session 4 (Wednesday 28 January 2015) Videos Time Slides Reading


Estimating Error Probabilities (Point Estimation) MPEG4 / OGG 6:31 PDF Frisvold and Moe pp. 145-147.
Binomial Probabilities (solving the exercise from the previous video) MPEG4 / OGG 6:08 PDF
The Distribution of the Error Rate (The Normal Distribution) MPEG4 / OGG 7:51 clean
Frisvold and Moe pp. 120+, 132.
Session 5 (Thursday 29 January 2015) Videos Time Slides Reading


Confidence Intervals (Interval Estimation) MPEG4 / OGG 7:35 PDF Frisvold and Moe pp. 147-148, 163-165.
Estimating Binomial Proportions (The Confidence Interval)
This video and the next have a certain overlap and can be viewed in arbitrary order.
MPEG4 / OGG 9:59 PDF Frisvold and Moe pp. 167-169.
For further depth, see separate page on estimation. The videos on that page will be appear at different stages of the module. For some students it may be useful to jump ahead.
Session 6 (Friday 30 January 2015) Videos Time Slides Reading
Review of previous material. Questions and answers. Channels with Memory (Statistical Dependence) MPEG4 / OGG 5:25 PDF Frisvold and Moe pp. 36-38(+).
Session 7 (Wednesday 4 February 2015) Videos Time Slides Reading

This session will review the first project, and we will generalise some of the ideas to estimation of the mean.


Point Estimation (Introduction to Estimation and Statistical Inference) MPEG4 / OGG 4:43 PDF
The Sample Mean (Point Estimation by Example) MPEG4 / OGG 7:36 PDF
The Standard Error (The Random Nature of Estimators) MPEG4 / OGG 5:10 clean
Frisvold and Moe page 147
Interval Estimation (What is the Confidence Interval?) MPEG4 / OGG 6:17 PDF Frisvold and Moe pp. 147-148, 163-165.
Error Margin (Estimating the Mean with Known Variance) MPEG4 / OGG 11:17 clean
Frisvold and Moe pp. 149-150.

Hans Georg Schaathun / Siebe van Albada