Kalman Filter, Live
Lesson 3 · Transforming covariance

Transforming covariance: the rule A P Aᵀ

Slide 2 drew the filter's uncertainty as an ellipse. But the filter never sits still: every tick (one time step) it moves its state forward with a matrix. The uncertainty has to move too. Think of a recipe for 4 people that you scale up for 8: if your scoop of flour was a bit off, doubling the recipe doubles that error too. Matrices do the same to a whole cloud of errors. They stretch it, turn it and tilt it. One rule says exactly how: if y = Ax + b, the covariance of y is A P Aᵀ. It is the matrix version of a rule for one number that you will see first.

Lesson 3 · slide 3APAᵀ
y = A x + b   →   μᵧ = A μ + b    Pᵧ = A P Aᵀ
Key ideas
  1. For one number: Var(a x + b) = a² σ². Scaling squares, shifting does nothing.
  2. For a vector: the mean goes through the map, μᵧ = Aμ + b, and the covariance becomes A P Aᵀ.
  3. The shift b moves the ellipse but never changes its shape.
  4. The filter uses this rule twice: F P Fᵀ in the predict step (F is the motion matrix) and H P Hᵀ in the correct step (H is the matrix that picks out what the sensor reads).

The rule for one number

Say a thermometer reads in Celsius with variance σ² = 4. Convert to Fahrenheit: y = 1.8 x + 32. The +32 slides every reading up by the same amount, so the spread does not change. The ×1.8 stretches every gap between readings by 1.8, so the variance grows by 1.8² = 3.24.

Var(a x + b) = a² σ²
Fahrenheit: 1.8² · 4 = 3.24 · 4 = 12.96,   so σ goes from 2 °C to 3.6 °F

Scaling squares. Shifting does nothing. Keep this in mind: the matrix rule has exactly the same shape.

The rule for a vector: the mean

Now x is a vector with mean μ and covariance P, and we build a new vector with a matrix: y = A x + b. The mean goes straight through:

μᵧ = A μ + b
A = 2001 (stretch x by 2),   μ = (2, 1),   b = (3, 1):   μᵧ = (4, 1) + (3, 1) = (7, 2)

The rule for a vector: the covariance

The covariance picks up A on both sides:

Pᵧ = A P Aᵀ

Compare with a² σ². The number a appears twice, once on each side of σ². The matrix A also appears twice, once on each side of P. The transpose Aᵀ (A with rows and columns swapped) on the right does two jobs. It makes the result symmetric, like every covariance matrix. And it makes the sizes fit when A is not square: for a 1 by 2 row H, H P Hᵀ is a single number, the variance of one reading. The shift b is gone, just like the +32.

Refresher: matrix times matrix. Entry (row i, column j) of the product is row i of the left matrix dotted with column j of the right one. A dot product means: multiply matching entries and add. One full 2 by 2 product:

1234 5678 = 19224350
(1, 1): 1·5 + 2·7 = 19   (1, 2): 1·6 + 2·8 = 22   (2, 1): 3·5 + 4·7 = 43   (2, 2): 3·6 + 4·8 = 50

For three matrices, multiply the first two, then multiply that result by the third.

Quick check: start from a round cloud, P = I, and stretch only x by a factor of 2. (I is the identity matrix: 1s on the diagonal, 0s elsewhere. As a covariance it means variance 1 for each number and no links.)

2001 1001 2001 = 4001

The first variance quadrupled (2² = 4), the second stayed at 1. That is the one-number rule for each part, as it should be.

A round cloud, P = I. The graph paper stretches sideways by 2, and every dot goes with it. The circle becomes an ellipse twice as wide: P becomes A P Aᵀ, with 4 and 1 on the diagonal.

Why A shows up twice

Write the deviation of the vector as d = x − μ: how far x sits from its mean, one entry per variable. A covariance matrix is the average of the deviation times itself: P = average of d dᵀ. Here d is a column and dᵀ is a row. A column times a row gives a full table of products. In 2D:

d dᵀ = dₓdₓdₓdᵧdᵧdₓdᵧdᵧ
For d = (2, −1):   d dᵀ = 4−2−21

Each entry is one product of deviations, so the average is the "average product of deviations" from slide 1, done for every pair at once.

The new deviation is y − μᵧ = A(x − μ) = A d, because b cancels. Put it in twice: (A d)(A d)ᵀ. A transpose flips the order of a product: (A d)ᵀ = dᵀAᵀ. Quick check with A = 1101 and d = (2, 1): A d = (3, 1), written as a row 3, 1. And dᵀAᵀ = row 2, 1 times 1011 = row 2·1 + 1·1, 2·0 + 1·1 = row 3, 1. Same. So (A d)(A d)ᵀ = A d dᵀ Aᵀ. A is the same for every sample, so it comes outside the average, just as the average of 3·d is 3 times the average of d. So the average becomes A P Aᵀ. One A from each copy of the deviation.

Motion makes errors linked

Now the move the filter makes every tick. A robot's state is (position, velocity). In 1 second, the position grows by the velocity, and the velocity stays the same. The matrix that does this is F = 1101: row 1 says "new position = position + velocity", row 2 says "velocity stays". Slide 4 builds F properly. Start from P with 4 and 1 on the diagonal (position error σ = 2 m, speed error σ = 1 m/s, not linked). First F times P, row by column:

F times P: each row of F meets each column of P, and their dot product fills one entry. The result is F P = 4101.

Then that result times Fᵀ, F with rows and columns swapped:

(F P) times Fᵀ. The answer F P Fᵀ has 5 and 1 on the diagonal and a new covariance of 1 off it.
F P Fᵀ = 5111

Two things happened. The position variance grew from 4 to 5, because an unsure speed adds to the position error. And a covariance of 1 appeared: a robot that went too fast is now also too far. The picture shows it as a tilt.

Position across, velocity up. F slides each dot sideways by its own velocity: dots high up (too fast) move right, dots low down move left. The upright ellipse leans over, and P becomes 5, 1, 1, 1.

Keep reading with the full course

Finish lesson 3 and you will be able to: track position and velocity in 2D with the matrix Kalman filter.

Still locked here:

  • The rest of this explanation
  • 3 more examples with step-by-step solutions
  • Five extra members-only problems with worked solutions

$20 once, lifetime access. Lessons 1 and 2 stay free.

Examples

Example 1 · Stretch and shift

P = I, mean (0, 0), A = 2001, b = (3, 1). This is the lab's starting setup.

Show solutionHide solution
New mean: A·(0, 0) + (3, 1) = (3, 1)
New covariance: A I Aᵀ = A Aᵀ = 4001

The circle becomes an ellipse twice as wide, centred at (3, 1). The shift moved it, the stretch shaped it.

Example 2 · One tick of motion

A robot's state is (position, velocity) with P = 4001. One tick of 1 s moves it by F = 1101 (position += velocity).

Show solutionHide solution
F P = 4101
F P Fᵀ = 4101 1011 = 5111

Two things happened. The position variance grew from 4 to 5, because an unsure speed adds to the position error. And a covariance of 1 appeared: a robot that went too fast is now also too far. Position and velocity errors became correlated by motion alone.

Exercise Use the motion matrix: set A = [[1, 1], [0, 1]] and start from P = [[4, 0], [0, 1]] (change both lines of A and both lines of P). You should get A P Aᵀ = [[5, 1], [1, 1]], a tilted ellipse. Edit the code and press Run.
slide_3.py Loading Python…
1
Output · Ctrl+Enter runs
Press Run to execute this slide's code. Ctrl+Enter works too.

Questions and comments

Stuck on something, or found a mistake? Ask here and I'll answer. You sign in with GitHub.

The full course

From noisy sensors to the EKF, by running it

Lessons 1 and 2 are free. Finish all four and you will be able to:

You get: all 4 lessons with Python labs · every worked example and solution · 5 extra problems per lesson · your progress tracked · Steppe, your AI tutor inside every lesson, day or night · every future update.

$20 once · lifetime access · 14-day money-back guarantee
Continue to checkout →

Already a member? Log in to the course