Reinforcement LearningIntroduction
It is mela week, and Sina has a mission: find the crispiest jalebi in town. Three stalls stand side by side — Gupta's, Meena's and Raju's. No board says which one is best. There is only one way to find out: buy a plate, taste it, and remember.
Have you ever taught a dog a trick? You don't hand it a manual. You wait until it does something close to the trick and give it a treat. Over many tries, the dog works out what earns treats.
That is reinforcement learning (RL). The computer is not told what to do: it acts in a world, gets rewards or penalties, and slowly learns which actions pay off. In the earlier chapters, a model learned from a table of right answers — that is supervised learning. In reinforcement learning, the computer has to
Nobody hands Sina a list of "correct stalls". She learns from the plates she buys — the reward for each of her own choices. In this chapter you play her game first, and then hand it to an agent that learns by itself.
Problem Structure
Every RL problem is built from the same parts. Think of a video game that you play without reading the instructions.
The Intuition
You are the agent. You look at the screen — the state of the game — and press a button: an action. If that action finds treasure, your score jumps: a reward. Your strategy for which button to press in each situation is your policy.
The Mechanics
- Agent: the learner that makes the decisions.
- Environment: everything the agent interacts with.
- State: the situation the environment shows the agent right now.
- Action: what the agent chooses to do.
- Reward: the number the environment sends back, positive for good and negative for bad.
- Policy: the agent's rule from each state to an action.
Sort each part of the game into its RL name.
What Makes an Agent Smart?
A smart agent does not simply grab the biggest reward on offer right now. It looks for the best outcome over time — like walking past a samosa stall because the sweet shop further on is worth more. To think ahead, it learns a value function. What does a value function do?
Note
The agent's goal is a policy that collects the most total reward over a whole episode — one complete run of the game, from start to finish.
The Learning Loop
Put the events of one time step in order.
Observe, act, collect the reward, land in a new state, learn — and round again. Every agent in this chapter runs this loop, from Sina at the jalebi stalls to Sina finding her way through a maze.