Train, Test and ValidationThe Training Set
The training set is the biggest share of the data — usually 60 to 80 % — because it is where the model learns its numbers: the slope and intercept of a line, or the and of an S-curve. But which examples land in it matters just as much as how many.
Shuffle Before You Split
Gurpreet wants a phone app that tells ripe mangoes from raw ones. He photographs 20 mangoes from his orchard: the ripe ones in the morning, the raw ones in the afternoon. So his photos are in order — all the ripe ones first. Complete both missions.
If data has any order — by date, by class, by city — and you split it without shuffling, each set sees a different world. Gurpreet's model would train on ripe mangoes and be tested on raw ones it had never met. Shuffle first; every library function for splitting does it for you.
How Much Goes Where?
Common splits are 70 / 15 / 15 and 80 / 10 / 10. With 1000 photos and a 70 / 15 / 15 split, the training set gets
Tip
With millions of examples, even 1 % is plenty for a test set, so splits like 98 / 1 / 1 are normal. With very little data, every example is precious; then cross-validation, a later chapter in this course, lets the data take turns being the validation set.