Playground · Month 5 · reinforcement learning
Teach a robot vacuum to find its dock.
Nobody tells this robot where the dock is. It only gets points: plus for docking, minus for the stairs, a small minus for every move. Press start and watch it go from random crashing to a clean route home.
Arrows: what the robot would do in each spot. Green: it expects to reach the dock from here. Red: it expects trouble.
moves per episode
the update rule
Q(s,a) ← Q + α[r + γ·max Q(s′) − Q]
α = 0.5 · γ = 0.95
+10 reach the dock · −10 fall down the stairs
−0.1 every move · −0.5 bump into furniture
Reward
The only feedback the robot gets. Change the rewards and you change what it learns.
Exploration
Turn ε to 0 early and it can lock onto a bad habit. Too high and it never stops wandering.
Q-values
Its running guess of how good each move is from each spot. The arrows are just the best guess per cell.