Kalman Filter, Live
Lesson 1 · The Gaussian

The Gaussian bell curve and the 68-95 rule

Slide 2 described noise with two numbers, the mean and the variance. Now look at the shape the readings make. Pour sand through a funnel and it piles into a hill: tall in the middle, thin at the edges. Sensor errors pile up the same way, because small errors are common and big ones are rare. That hill is the Gaussian, or bell curve. The mean says where its peak sits and the variance says how wide it is, and those two numbers fix the whole curve.

Lesson 1 · slide 3The Gaussian
x ~ N(μ, σ²)    sum: N(μ₁ + μ₂, σ₁² + σ₂²)    scale: Var(a x) = a² Var(x)
Key ideas
  1. A Gaussian (normal distribution) N(μ, σ²) is fixed by two numbers: its mean μ (the centre) and its variance σ² (the width).
  2. About 68% of samples land within ±1σ of the mean, and about 95% within ±2σ.
  3. Add two independent Gaussians (one tells you nothing about the other) and you get a Gaussian: the means add and the variances add.
  4. Multiply by a number a and the variance is multiplied by a²: Var(a x) = a² Var(x), where Var(x) means the variance of x.

The bell curve

A distribution says how likely each value of a random variable is. The Gaussian is the most important one for the Kalman filter. We write N(μ, σ²), where N stands for "normal". When a variable x follows this bell we write x ~ N(μ, σ²). Read the ~ as "follows" or "is distributed as".

Readings of a thermometer drop in one by one. They pile up thick near 20 and thin out at the edges, filling the bell.

Its curve has this formula:

p(x) = 1 / √(2πσ²) · exp( −(x − μ)² / (2σ²) )

Here e = 2.718… is a fixed number, like π, and exp(t) means e to the power t. p(x) is the height of the curve at x: the higher, the more often readings land near x.

It looks scary. Don't be afraid of it: we will almost never plug numbers into it. What matters is what hides inside. Only two numbers appear, μ and σ². The mean μ slides the bell left and right. The variance σ² stretches it wide or squeezes it thin. To model a sensor's noise, that is all we need: where its readings centre and how much they scatter.

N(20, 4): peak at 20°,   variance 4, so σ = √4 = 2°
N(25, 4): the same bell, slid 5° to the right
N(20, 1): the same centre, squeezed thinner (σ = 1°)
Bell A is N(20, 1), bell B is N(25, 4). B has a bigger mean, so it sits further right. It also has a bigger variance, so it is wider and lower.

The 68-95 rule

Here is a rule of thumb you will use all the time. About 68% of samples fall within one standard deviation of the mean (between μ − σ and μ + σ). About 95% fall within two (between μ − 2σ and μ + 2σ). A reading more than 3σ away is very rare, so it often means something is broken.

So when a datasheet (the maker's spec sheet) says a GPS has σ = 3 m, you know: about 2 readings in 3 land within 3 m of the truth, and almost all within 6 m.

robot at 30 m, GPS σ = 3 m
68%: 30 − 3 to 30 + 3, so 27 m to 33 m    95%: 30 − 6 to 30 + 6, so 24 m to 36 m

Adding Gaussians

The second reason Gaussians are so handy: they combine with almost no effort. Write Var(x) for the variance of x. Two random variables are independent when one tells you nothing about the other: knowing the error in x gives no hint about the error in y. If x and y are independent, then

mean(x + y) = mean(x) + mean(y)     Var(x + y) = Var(x) + Var(y)

And the sum is again a Gaussian. Notice that it is the variances that add, not the standard deviations. Uncertainty piles up: every independent error you add makes the result less certain.

x ~ N(2, 1),   y ~ N(3, 2)  →  x + y ~ N(2 + 3, 1 + 2) = N(5, 3)

A Gaussian can also describe our own uncertainty. N(30, 4) means "we think the robot is near 30 m, give or take 2 m" (σ = √4 = 2).

A robot at N(30, 4) (best guess 30 m, variance 4) takes a step of N(5, 1) (about 5 m, variance 1). Its new position sits at 30 + 5 = 35 and the bell gets a little wider: variance 4 + 1 = 5.

Scaling a Gaussian

Multiply a random variable by a number a, and its mean is multiplied by a, but its variance is multiplied by a²:

Var(a x) = a² Var(x)     so the standard deviation is multiplied by |a| (a without its minus sign)

That makes sense in units. Convert metres to centimetres (a = 100) and the spread in cm is 100 times bigger, so the variance, in cm², is 100² = 10 000 times bigger.

a height of N(1.5, 0.04) m  →  mean 1.5 · 100 = 150 cm,   variance 0.04 · 100² = 400 cm²
σ = √400 = 20 cm, and in metres σ = √0.04 = 0.2 m ✓

One trap. Say x has variance 4. Add an independent y that also has variance 4: x + y has variance 4 + 4 = 8. But x + x = 2x has variance 2² · 4 = 16, because both copies always err the same way and never cancel. Independence matters.

Put adding and scaling together and you see why averaging works. The average of two independent readings, each N(20, 4), is half their sum: the sum has variance 8, and halving multiplies it by (½)², so the average has variance 2.

One reading (red) is N(20, 4). The average of two independent readings is N(20, 2): half the variance, so its bell is narrower and taller.

A bigger example: four steps in a row

The robot starts at N(30, 4) m and takes 4 steps, each N(5, 1) m, all independent. Add them one at a time, or all at once:

mean: 30 + 5 + 5 + 5 + 5 = 50 m
variance: 4 + 1 + 1 + 1 + 1 = 8 m²    σ = √8 ≈ 2.83 m
95% range: 50 ± 2 · 2.83, so about 44.3 m to 55.7 m

Every step adds 1 to the variance. After 1000 steps it would be 4 + 1000 = 1004, so σ ≈ 31.7 m. Counting steps alone slowly loses track of the robot, which is why lesson 2 adds a GPS.

Why this is all the algebra we need

These few rules are almost the whole toolkit. In lesson 2 the filter's predict step adds the robot's movement to its position: that is "means add, variances add". The correct step fuses two Gaussians: that is slide 5. A drone's tilt (attitude) filter runs these same two moves hundreds of times per second.

Next: a thin bell means readings packed close to the centre, a wide one means they scatter. Slide 4 uses that to decide which of two sensors to trust.

Examples

Example 1 · The 68-95 rule

A thermometer's readings follow N(20, 4), so σ = √4 = 2°.

Show solutionHide solution
68%: from 20 − 2 to 20 + 2, so between 18° and 22°
95%: from 20 − 4 to 20 + 4, so between 16° and 24°

A reading of 27° would be 3.5σ away. Possible, but very unlikely.

Example 2 · A robot takes a step

We believe a robot is at N(30, 4) m. Its encoder says it moves N(5, 1) m in one step: we think it moved 5 m, but we are only sure to within about ±1 m. The errors are independent.

Show solutionHide solution
mean: 30 + 5 = 35 m
variance: 4 + 1 = 5 m²    σ = √5 ≈ 2.24 m

The new position is N(35, 5). The robot moved, and we became a little less sure where it is. This is exactly the predict step of the 1D (one-dimensional: one number, like position along a line) Kalman filter.

Example 3 · Averaging two readings

Two independent readings, each N(20, 4). What is the distribution of their average (x₁ + x₂) / 2?

Show solutionHide solution
sum: N(20 + 20, 4 + 4) = N(40, 8)
scale by a = ½: mean 40 · ½ = 20,   variance 8 · (½)² = 2

The average is N(20, 2): half the variance of one reading. That is why averaging works, and slide 5 will show the same halving when two equal sensors are fused.

Example 4

x ~ N(3, 2) and y ~ N(−1, 5) are independent. Find the distribution of x + y and of x − y.

Show solutionHide solution

x + y: mean 3 + (−1) = 2, variance 2 + 5 = 7. So N(2, 7).

x − y = x + (−1)·y. The mean is 3 − (−1) = 4. The variance of (−1)·y is (−1)² · 5 = 5, so the variance is 2 + 5 = 7 again. So N(4, 7). Subtracting noise still adds variance.

Example 5

A height sensor reads N(2, 0.01) in metres. Write the same reading in centimetres. What is σ in cm?

Show solutionHide solution

Multiply by a = 100. Mean: 2 · 100 = 200 cm. Variance: 0.01 · 100² = 100 cm².

So N(200, 100) and σ = √100 = 10 cm. Check: in metres σ = √0.01 = 0.1 m, which is 10 cm ✓

Exercise The encoder gets worse. Set var2 = 5.0 and check that the sum's variance becomes 4 + 5 = 9. The samples' variance should come out close to 9, not exactly 9 (the lab gives about 8.83). Edit the code and press Run.
slide_3.py Loading Python…
1
Output · Ctrl+Enter runs
Press Run to execute this slide's code. Ctrl+Enter works too.

Enjoying lesson 1? Get the full course: all 4 lessons through the matrix filter and the EKF, your progress saved lesson by lesson, every solution and 5 extra problems per lesson. $20 once, yours for life.

Questions and comments

Stuck on something, or found a mistake? Ask here and I'll answer. You sign in with GitHub.

The full course

From noisy sensors to the EKF, by running it

Lessons 1 and 2 are free. Finish all four and you will be able to:

You get: all 4 lessons with Python labs · every worked example and solution · 5 extra problems per lesson · your progress tracked · Steppe, your AI tutor inside every lesson, day or night · every future update.

$20 once · lifetime access · 14-day money-back guarantee
Continue to checkout →

Already a member? Log in to the course