Kalman Filter, Live
Lesson 3 · Correlation

Correlation and covariance: when two numbers move together

Lesson 2's filter tracked one number: the robot's position. Real robots track several numbers at once, like position and speed. These numbers are not independent. Think of a café: on hot days it sells more ice cream, so if you know the weather you can guess the sales. Two numbers that move together like this are correlated. One number, the covariance, captures this link. Its sign says the direction: up together, or one up while the other goes down. Its size also depends on the units, and slide 2 fixes that. Covariance is the new ingredient of the matrix Kalman filter, the version that tracks several numbers at once (slide 5).

Lesson 3 · slide 1Correlation
cov(x, y) = average of (x − μₓ)(y − μᵧ)    > 0: move together    < 0: opposite    ≈ 0: no link
Key ideas
  1. Two quantities are correlated when knowing one tells you something about the other.
  2. A deviation is how far a value sits from its mean: x − μ.
  3. Covariance is the average product of the two deviations. Its sign says which way the cloud of points tilts.
  4. Variance is the covariance of a variable with itself: the average of (x − μ)².

Ice cream, hot chocolate and coffee

A café writes down the temperature and its sales every day. Three patterns show up.

Ice cream goes up on hot days and down on cold days. The two numbers rise and fall together. We say they are positively correlated.

Hot chocolate does the opposite: it sells on cold days. When the temperature goes up, sales go down. They are negatively correlated.

Coffee shows no up-or-down trend with temperature. Knowing whether it is warmer or colder tells you nothing about whether coffee sales go up or down. They are uncorrelated.

The picture: a tilted cloud

Put each day on a scatter plot: temperature across, sales up. Ice-cream days form a cloud that tilts up to the right. Hot-chocolate days tilt down to the right. Coffee days form a cloud with no tilt at all.

Ice cream: each dot is one day, temperature across and sales up, both measured from their averages. The cloud tilts up to the right. The rings mark where most days fall. Slide 2 explains them; ignore them, and the numbers under the picture, for now.
Hot chocolate: warm days sit to the right and low. The cloud tilts down to the right.
Coffee: the cloud has no tilt. Moving right (warmer) does not move the dots up or down on average.

We want one number that captures this tilt. Its sign should say which way the cloud leans, and its size should say how strongly.

Step 1: the deviation

Take one day. Its deviation in temperature is how far that day sits from the average temperature: x − μₓ. Do the same for sales: y − μᵧ. A deviation is positive above the average and negative below it.

Temperatures 10, 14, 18, 22 °C:   μₓ = 64 / 4 = 16
Deviations: 10 − 16 = −6,   14 − 16 = −2,   18 − 16 = 2,   22 − 16 = 6

Step 2: multiply the two deviations

On a hot day with high sales, both deviations are positive, so the product is positive. On a cold day with low sales, both are negative, and the product is positive again. If high goes with low, one deviation is positive and the other negative, so the product is negative.

Hottest day: 22 °C (deviation 6) and 55 ice creams (average 35, deviation 20)
Product: 6 · 20 = 120,   positive: hot and busy go together

Step 3: average the products

Average the products over all N days. That average is the covariance:

cov(x, y) = σₓᵧ = (1/N) · Σ (x[k] − μₓ)(y[k] − μᵧ)

Here x[k] means the temperature on day k, and y[k] the sales on day k. We also write the covariance σₓᵧ (σ with the two variable names below it). Σ means "add up over all days k". If most products are positive, the covariance is positive. If most are negative, it is negative. If the two variables are unrelated, positives and negatives cancel and the covariance is close to 0. The worked examples below do all four ice-cream days and get cov = 50.

Keep reading with the full course

Finish lesson 3 and you will be able to: track position and velocity in 2D with the matrix Kalman filter.

Still locked here:

  • The rest of this explanation
  • 3 more examples with step-by-step solutions
  • Five extra members-only problems with worked solutions

$20 once, lifetime access. Lessons 1 and 2 stay free.

Examples

Example 1 · Ice cream on four days

Temperatures 10, 14, 18, 22 °C. Ice-cream sales 20, 35, 30, 55.

Show solutionHide solution
Means: μₓ = 64 / 4 = 16,   μᵧ = 140 / 4 = 35
Deviations x: −6, −2, 2, 6    y: −15, 0, −5, 20
Products: 90, 0, −10, 120  →  sum 200
cov = 200 / 4 = 50

Positive: ice cream and temperature move together. One day (18°, 30 sales) went against the trend, and its product is negative, but the other days win.

Example 2 · Hot chocolate on the same days

Same temperatures, hot-chocolate sales 40, 32, 28, 20.

Show solutionHide solution
μᵧ = 120 / 4 = 30,   deviations y: 10, 2, −2, −10
Products: (−6)(10) = −60, (−2)(2) = −4, (2)(−2) = −4, (6)(−10) = −60
cov = −128 / 4 = −32

Negative: when it gets warmer, hot chocolate sells less.

Exercise Turn ice cream into hot chocolate. Set slope = -2.0 so sales fall as it gets warmer, and watch the cloud of points tilt down to the right. (The ellipse drawn around it is a picture of the spread. The next slide explains it.) Edit the code and press Run.
slide_1.py Loading Python…
1
Output · Ctrl+Enter runs
Press Run to execute this slide's code. Ctrl+Enter works too.

Questions and comments

Stuck on something, or found a mistake? Ask here and I'll answer. You sign in with GitHub.

The full course

From noisy sensors to the EKF, by running it

Lessons 1 and 2 are free. Finish all four and you will be able to:

You get: all 4 lessons with Python labs · every worked example and solution · 5 extra problems per lesson · your progress tracked · Steppe, your AI tutor inside every lesson, day or night · every future update.

$20 once · lifetime access · 14-day money-back guarantee
Continue to checkout →

Already a member? Log in to the course