Curvingo · Made in India

Glossary

Select one of the keywords on the left…

Testing Your AIIntroduction

Reading time: ~5 min

Harpreet studies agriculture in Ludhiana. For her project she wants to know how much fertiliser gives the best wheat harvest. She has the records of the farmer from Train, Test and Validation — 17 farms — and has visited three more herself: 20 farms in all. Each one is a pair of numbers: bags of fertiliser per hectare, and tonnes of wheat per hectare.

Twenty farms is not much. Harpreet needs most of them to train her model, and some of them to check it.

Remember Kabir, who learnt one sample paper by heart? Why is it no good testing a model on the very data it learnt from?

It takes too long to work out the score.
The model can look perfect just by remembering the answers.
It needs too many data points.

So we hold some data back. But with only 20 farms, every farm held back is a farm the model cannot learn from — and the score then depends on which farms were held back. This chapter shows how to use every farm for training and for checking: k-fold cross-validation.

Sina