A robot has no eyes in the sky and no gut feeling. All it has are sensors — and every sensor lies a little. This is the story of how robots combine several unreliable answers into one reliable one.
Measure the same table three times with a tape measure and you'll get three slightly different numbers. Sensors are the same. A distance sensor on a robot might report 98 cm, then 101 cm, then 99 cm for a wall that never moved. That random wobble is called noise.
Engineers put a number on the wobble: the standard deviation, written σ (sigma) — roughly "the typical size of the error." A sensor with σ = 3 cm usually lands within ±3 cm of the truth, and almost always within ±2σ = ±6 cm. In symbols, one reading looks like this:
z — what the sensor reports (the only thing you get to see) · x — the true value you actually want · v — this particular reading's random error · σ — the standard deviation, the typical size of v.You see z; you want x.
Sensor fusion is the art of combining several noisy sensors so the combination is better than any one of them alone. That sounds impossible — how does adding a worse sensor ever help? Try it below.
There's a formula for the best dial setting, called inverse-variance weighting: trust each sensor in proportion to 1/σ². (Squaring σ gives the variance — the standard currency of uncertainty, because variances of independent errors simply add.)
wlaser — the weight (trust fraction, 0–1) given to the laser; the sonar gets the leftover 1 − w · zlaser, zsonar — the two sensors' readings · σlaser, σsonar — each sensor's typical error (3 cm and 8 cm here) · σ² — a σ squared, called the variance.Notice the crossover: the sonar's variance sits on top of the laser's weight. The sloppier the sonar, the harder the dial swings toward the laser — 88% here, close to the ~85% you likely found by hand.
Even better, the blend's own uncertainty obeys a beautiful rule — variances combine like resistors in parallel:
σblend — the typical error of the weighted blend itself · σlaser, σsonar — the individual sensors' typical errors · 1/σ² — a quantity engineers call precision: precisions add, so the blend is always more precise than either input.2.81 cm beats the laser's 3 cm — which is why the blend won in the widget. Adding a sensor, however sloppy, can only shrink the blend's σ. It can never make things worse (as long as you weight it honestly).
Close your eyes and walk across a room, counting steps. You always know roughly where you are — each step adds about 60 cm in the direction you're facing. That's dead reckoning: estimating position by adding up your own movements.
Robots do this with wheel encoders (counting wheel turns) and gyroscopes. It updates hundreds of times per second and is beautifully smooth. But every step has a tiny error — a bit of wheel slip, a slightly off heading — and those errors never get erased. They pile up. The longer you drive, the more lost you get.
In symbols, dead reckoning is just running a sum. The estimate after step k is the estimate after step k−1, plus the step your odometry (wheel encoders + gyro) says you took:
x̂k — the position estimate after step k ("x-hat"; the hat means estimate of x, not x itself) · k — the step number: 1, 2, 3, … · Δxk — the true distance moved during step k · vk — that step's measurement error (slip, quantization, gyro wander).Every vk gets baked into the running sum forever — nothing in this equation can remove old errors.
Summing N independent random errors is a random walk, and random walks have a signature growth rate: the typical accumulated error grows like σstep·√N. Worse, any consistent bias — say a wheel that's 1% smaller than the code thinks — adds error proportional to N itself, growing linearly and without limit. Engineers call this unbounded error: given enough time, dead reckoning is guaranteed to be arbitrarily wrong.
Now the opposite kind of sensor. A global positioning fix — GPS outdoors, or a camera spotting a known landmark indoors — measures where you are from the outside. Each fix is noisy: it might be a few meters off in a random direction. And fixes arrive slowly, maybe once a second.
But here's its superpower: the errors don't accumulate. A GPS fix taken after an hour of driving is exactly as good as one taken at the start. GPS never gets more lost.
That's because GPS is an absolute measurement of position, not a relative measurement of motion. Compare its equation to dead reckoning's — there's no running sum, so there's nowhere for old errors to hide:
zk — the GPS fix received at time k · xk — the true position at that moment · vk — that fix's error, fresh every time and independent of all previous fixes · σgps — the fix error's typical size, the same at k = 1 and k = 1,000,000.No running sum, so old errors have nowhere to hide: the error is bounded — it wobbles around σgps forever — versus dead reckoning's unbounded σstep·√N. The trade-off: fixes arrive at a low update rate (a few Hz at best) and can lag reality by a fraction of a second (latency).
GPS needs a sky. Indoors — a warehouse, a gym, a competition field — satellites are useless. But look back at Part 3: nothing there actually required satellites. What it required was an absolute measurement, a sensor that reports "here is where you are on the map." A camera can be exactly that sensor.
The recipe has three ingredients. A camera. A set of fiducial markers — high-contrast printed patterns like AprilTags or QR-style squares, engineered so software can detect them and tell them apart at a glance. And a map: a list of exactly where each marker was surveyed and bolted down. When the camera spots marker #7, the vision pipeline measures the marker's range and bearing — really its full pose relative to the camera — and since the map says where marker #7 lives in the world, you can solve backwards for where the robot must be standing:
zvision — the robot's position according to this tag sighting; it plays the exact role a GPS fix z played in Part 3 · m — the marker's position, looked up from the map (known in advance) · d — the camera-measured offset from robot to marker (range and bearing, turned into a displacement) · written for one axis; in 2-D it's the same subtraction done for x and y separately.All the imperfections — pixel noise, lens distortion, a slightly mis-surveyed map — get lumped into a fresh error v on each sighting, exactly like a GPS fix's noise.
Squint and this is the Part 3 equation again: one absolute fix, fresh noise every frame, no running sum. Bounded error. Zero drift. A modest update rate (however fast the pipeline runs) and some latency (the image takes time to process). To the Kalman filter, a tag sighting and a satellite fix are the same species of measurement — everything in the next two parts applies unchanged if you swap "GPS ping" for "tag sighting."
There is one genuinely useful difference: vision noise isn't constant. A large, close, squarely-viewed tag gives a crisp fix (small σ); a distant tag seen edge-on gives a mushy one (σ grows quickly with range and viewing angle). And the filter has a ready-made answer — let R change per measurement. Hand it a fresh R = σ² with every sighting, and the gain K = P/(P + R) automatically leans hard on close tags and politely shrugs at far ones. Same machinery as before; the trust dial just reads the fine print on each individual measurement.
This pattern is everywhere: competition robots localizing off AprilTags mounted around the field, warehouse robots reading floor markers, VR headsets tracking features on your walls — and on Mars, where there are no satellites at all, landers and helicopters navigate by matching camera views against orbital terrain maps: natural landmarks acting as markers nobody had to bolt down.
In 1960, engineer Rudolf Kálmán wrote down a recipe for exactly this problem — NASA used it to guide Apollo to the Moon. Strip away the math and the recipe is two steps, repeated forever:
1. Predict. Between GPS fixes, move your position estimate using dead reckoning. But be humble: every prediction makes you a little less sure, so let your "confidence bubble" grow.
2. Correct. When a GPS fix arrives, blend it with your prediction — exactly like the trust dial in Widget 01. And here's the genius part: the filter sets the dial automatically. Big confidence bubble (you've been guessing a while)? Trust the GPS more. Tiny bubble (you just corrected)? Mostly ignore this noisy fix. After each correction, the bubble shrinks.
That automatic trust setting has a fancy name — the Kalman gain — but it's just the dial: "how much do I believe this new measurement versus what I already thought?"
Here is the entire filter for one dimension. The estimate is x̂, and the confidence bubble is tracked as a variance P (bubble radius ≈ √P). Two noise numbers are chosen ahead of time by the engineer: Q, the process noise — how much variance each prediction adds — and R, the measurement noise — the sensor's variance, σgps².
x̂ — the current position estimate · P — the estimate's variance: the confidence bubble, squared (bubble radius ≈ √P) · u — the step reported by odometry since the last cycle · Q — process noise: variance added by each prediction, chosen from how sloppy the odometry is · z — the incoming GPS fix · R — measurement noise: the fix's variance, σgps² · (z − x̂) — the innovation: "how surprised was I by this measurement?" · K — the Kalman gain, the trust dial, a number between 0 and 1 computed automatically from the two variances · ← — "gets replaced by": each line overwrites the old value with the new one.
Read the gain formula as a story. If P ≫ R — bubble huge after a long stretch of guessing — then K → 1 and the estimate leaps to the fix. If P ≪ R — you just corrected, and the sensor is comparatively noisy — then K → 0 and the fix is politely ignored. And the last line, P ← (1 − K)·P, guarantees every measurement shrinks the bubble at least a little: the "sloppy second opinions still help" rule from Part 1, now running automatically. It's the same math, too — K = P/(P + R) is inverse-variance weighting, applied between your prediction and your measurement.
Now run the full system. The robot drives its route while the Kalman filter works in real time: dead reckoning carries the estimate smoothly between fixes, and each GPS ping tugs it back toward the truth — tugging hard when confidence is low, gently when confidence is high.
Under the hood, this demo runs the Part 5 loop once per axis — a state of (x, y) with its own P for each. Real robots pack more into the state vector: heading θ, velocity, sometimes even the gyro's slowly-wandering bias, so the filter can learn and cancel it. And because turning makes the motion equations nonlinear (sines and cosines of θ), production systems use variants like the Extended Kalman Filter (EKF) — the identical predict–gain–correct loop, with the equations re-linearized around the current estimate at every step. The logic never changes: grow P by Q, shrink it through R, let K referee.
One more idea before you drive. The two noise sliders below aren't measurements — they're tuning parameters, the numbers a robotics engineer dials in as an honest confession of how bad each source is. Wheel slip sets Q (process noise) and GPS noise sets R (measurement noise). Because they feed the gain K = P/(P + R), each acts as inverse trust: cranking a source's noise number up turns its weight in the fusion down. And notice that formula only compares P against R — so what matters is the relative size of the two noise values, not either one on its own. Scale both up together and the trust split barely moves; the filter only shifts its weighting when one source gets worse compared to the other.
1. Every sensor lies a little — random wobble is called noise, and it's unavoidable.
2. Blending beats the best sensor. Weight each source by how much you trust it, and the combination wins.
3. Dead reckoning drifts. Adding up your own motions is smooth and fast, but errors pile up forever.
4. Global fixes are noisy but honest. GPS wobbles, but it never gets progressively lost — and a camera spotting mapped fiducial markers is the same kind of sensor, no satellites required. Any "here's where you are on the map" measurement plays this role.
5. Predict, then correct. The Kalman filter carries the estimate with dead reckoning, corrects with GPS, and sets the trust dial automatically — trusting measurements more when its own confidence bubble has grown.
You'll find this exact loop inside phones, drones, self-driving cars, Mars rovers, and the robot navigation code in competition robotics — anywhere a machine has to answer the oldest navigation question there is: where am I?