Reinforcement LearningCode with Python

Here is the jalebi agent in plain Python — no library needed. random.seed(38) fixes the random numbers, so the program prints the same thing every time you run it. The stalls really have qualities 5, 6 and 8, and every plate is off by up to 2 points. Run it, then change ε and run it again.

The Return in a Loop

The return from cell 3 is a short sum. gamma ** k means γ to the power k; the first reward has k = 0, so it is not discounted. Run it, then try other values of γ.

Q-Learning on the Lane

The whole of Line World in about 20 lines. Q[cell][0] is the value of stepping left and Q[cell][1] the value of stepping right; the line with alpha in it is the Q update. Run it, then compare 10 episodes with 200, and γ = 0.9 with γ = 0.5.

After 200 episodes with γ = 0.9, the right-hand values are the discounted sweet shop: 10, 9, 8.1, 7.29 — and almost 6.56 in cell 1, which Sina rarely reaches from the middle, so its numbers settle last.