Kalman Filter, Live
Lesson 1 · Random variables

Random variables: mean, variance and standard deviation

Slide 1 said every reading is the truth plus noise. Now we ask how to describe noise we cannot predict. Think of rolling a die: you never know the next roll, but you know the pattern, each face about one time in six. Sensor readings work the same way. Each reading is one draw of a random variable, and two numbers describe its pattern. The mean says where the readings centre, and the variance says how far they spread. These two numbers are the language the Kalman filter speaks.

Lesson 1 · slide 2Random variables
μ = (z₁ + … + zₙ) / n    s² = average of (zᵢ − μ)², an estimate of σ²    s = √s²
Key ideas
  1. A random variable is a quantity whose next value we cannot know in advance. Each sensor reading is one draw, called a sample. We call the readings z₁, z₂, …, zₙ, and zᵢ means any one of them.
  2. The mean μ is the centre: the average of the samples.
  3. The variance σ² is the true spread of the noise. From n readings we compute the sample variance s²: the average squared distance from the mean. It estimates σ².
  4. The standard deviation σ = √σ² (from samples, s = √s²) is the spread in the original units (metres, degrees).

A world without noise

In a perfect world we would not need any of this. The temperature would be exactly 30°C and a sensor would report the exact truth every time. The real world is noisy. A GPS, an encoder or an accelerometer returns a value close to the truth, never exactly the truth.

Random variables

To describe that honestly, we use a random variable: a quantity that takes a different value each time we look, following some pattern. A die is one. You cannot say what the next roll will be, but you know every face comes up about one time in six.

A sensor reading is the same. On slide 1 we wrote reading = truth + noise. The truth is fixed. The noise is a random variable, so the reading is one too. Each reading we take is one sample of it.

roll a die 6 times: 3, 6, 1, 3, 5, 2    six samples of one random variable
read a thermometer 4 times: 19, 20, 21, 24°    four samples of another

The mean: where readings centre

Call the readings z₁, z₂, …, zₙ, where n is how many we took and zᵢ means any one of them. The mean, written μ (the Greek letter "mu"), is the plain average of the samples: add them up and divide by how many there are.

readings 19, 20, 21, 24  →  μ = (19 + 20 + 21 + 24) / 4 = 84 / 4 = 21°

Strictly, the average of your readings is the sample mean: our estimate of the true mean of the noisy readings. We use μ for both; the context tells which. For a good sensor the mean of many readings lands close to the truth, because the + and − errors cancel.

A thermometer in a 22° room. Each dot is one reading, from first (left) to fortieth. The blue line is the mean of all readings so far: it wobbles at first, then settles near 22.

The variance: how far they spread

Two sensors can have the same mean and still be very different. One gives 9, 10, 11. The other gives 6, 10, 14. Both average 10, but you would trust the first one more.

The variance, written σ² ("sigma squared"), measures that spread. From readings we estimate it like this: for each sample, take its distance from the mean, square it, then average the squares. We call the result s², the sample variance:

s² = [ (z₁ − μ)² + (z₂ − μ)² + … + (zₙ − μ)² ] / n

Why square? Without squaring, the distances above and below the mean would cancel to zero. Squaring makes every distance count as positive, and it punishes big misses more than small ones.

Keep two names apart. The true spread σ² is the variance of the noise itself: a fixed property of the sensor. The sample variance s² is what you compute from your n readings with the recipe above. It is only an estimate of σ², and it gets closer as n grows. Back to the four readings with μ = 21:

distances: 19 − 21 = −2,   20 − 21 = −1,   21 − 21 = 0,   24 − 21 = +3
squares: 4, 1, 0, 9  →  s² = (4 + 1 + 0 + 9) / 4 = 14 / 4 = 3.5, our estimate of σ²
Same mean, different spread. Sensor A (9, 10, 11) has s² = 0.67: its readings bunch tightly. Sensor B (6, 10, 14) has s² = 10.67: its readings spread widely. Both numbers are worked out below.

The standard deviation: spread in real units

If the readings are in metres, the variance is in metres squared, which is hard to picture. So we also use the standard deviation σ = √σ². From readings we compute s = √s² in the same way. Both are back in metres, and they read like "a typical reading is about σ away from the mean".

s² = 3.5  →  s = √3.5 ≈ 1.87°    a typical reading is about 1.9° from 21°
200 readings drop in one by one. Most of them land within about 1° of 22°.

A note for later: some books divide by n − 1 instead of n when they estimate a variance from samples. For large n the two barely differ. In this course we divide by n, which is what np.var does.

A bigger example: ten readings

A cheap thermometer gives ten readings: 22, 24, 21, 19, 23, 20, 25, 18, 22 and 26°.

μ = 220 / 10 = 22°
distances: 0, +2, −1, −3, +1, −2, +3, −4, 0, +4
squares: 0, 4, 1, 9, 1, 4, 9, 16, 0, 16  →  sum 60
s² = 60 / 10 = 6    s = √6 ≈ 2.45°

The recipe never changes: average, subtract, square, average again. For a thousand readings you would let the computer do it, which is what np.mean and np.var do in the lab.

Why the filter cares

The Kalman filter keeps two numbers for everything it knows: a best guess (a mean) and how unsure it is (a variance). A drone's tilt estimate, a car's position, a phone's heading: each one is a mean with a variance attached. Get these two numbers right and the rest of the filter follows.

In the lab

The code takes n readings of a 22° room with a thermometer whose noise has σ = 1, so its true spread is σ² = 1² = 1. It computes the sample mean and the sample variance with np.mean and np.var, and draws the readings as dots. It draws two hill-shaped curves: one from your samples, one from the true noise. Think of each curve as a picture whose centre is the mean and whose width is the spread; slide 3 explains the shape. With only 5 readings the lab gives a sample mean of 22.39° and a sample variance of 0.40, far below the true spread of 1. With n = 200 they come out at 21.88° and 0.96, close to 22 and 1.

Next: pile up many readings and they form a hill, tall in the middle and thin at the edges. Slide 3 names that hill, the Gaussian bell curve, and shows what it tells us.

Examples

Example 1 · Five thermometer readings

A thermometer reads 21, 23, 22, 20 and 24°.

Show solutionHide solution
μ = (21 + 23 + 22 + 20 + 24) / 5 = 110 / 5 = 22°
distances from μ: −1, +1, 0, −2, +2  →  squares: 1, 1, 0, 4, 4
s² = (1 + 1 + 0 + 4 + 4) / 5 = 10 / 5 = 2    s = √2 ≈ 1.41°

A typical reading sits about 1.4° from the mean.

Example 2 · Four GPS readings

A parked robot's GPS reads 98, 102, 101 and 99 m.

Show solutionHide solution
μ = 400 / 4 = 100 m
distances: −2, +2, +1, −1  →  squares: 4, 4, 1, 1
s² = 10 / 4 = 2.5 m²    s = √2.5 ≈ 1.58 m

The variance is in m², the standard deviation in m.

Example 3 · Same mean, different spread

Sensor A reads 9, 10, 11. Sensor B reads 6, 10, 14.

Show solutionHide solution
A: μ = 10,   s² = (1 + 0 + 1) / 3 ≈ 0.67,   s ≈ 0.82
B: μ = 10,   s² = (16 + 0 + 16) / 3 ≈ 10.67,   s ≈ 3.27

Same mean. B's variance is 16 times bigger and its standard deviation 4 times bigger. Slide 4 turns this into a rule for which sensor to trust.

Example 4

A sensor reads 10, 10, 10, 10. What is its variance? What would that mean for a real sensor?

Show solutionHide solution

μ = 10 and every distance is 0, so s² = 0.

A variance of 0 means no noise at all: a perfect sensor. Real sensors never get there. If you see it, the sensor is often stuck, or its output is rounded so coarsely that the noise is hidden.

Example 5

A laser reads 2, 4, 4, 4 and 6 m. Find μ, s² and s, with units.

Show solutionHide solution

μ = 20 / 5 = 4 m. Distances: −2, 0, 0, 0, +2. Squares: 4, 0, 0, 0, 4.

s² = 8 / 5 = 1.6 m² and s = √1.6 ≈ 1.26 m.

Exercise Take more readings. Set n = 200 and watch the sample mean settle near 22° and the sample variance near 1 (σ² = 1² = 1). Edit the code and press Run.
slide_2.py Loading Python…
1
Output · Ctrl+Enter runs
Press Run to execute this slide's code. Ctrl+Enter works too.

Enjoying lesson 1? Get the full course: all 4 lessons through the matrix filter and the EKF, your progress saved lesson by lesson, every solution and 5 extra problems per lesson. $20 once, yours for life.

Questions and comments

Stuck on something, or found a mistake? Ask here and I'll answer. You sign in with GitHub.

The full course

From noisy sensors to the EKF, by running it

Lessons 1 and 2 are free. Finish all four and you will be able to:

You get: all 4 lessons with Python labs · every worked example and solution · 5 extra problems per lesson · your progress tracked · Steppe, your AI tutor inside every lesson, day or night · every future update.

$20 once · lifetime access · 14-day money-back guarantee
Continue to checkout →

Already a member? Log in to the course