Popular Courses
Linear Regression
Introduction
Decision Trees
Introduction
Neural Networks
Introduction

Sign in to Curvingo

or

New Account     Reset Password     

Testing Your AI
Introduction
Our Previous Approach
K-Fold Cross-Validation
Leave-One-Out Cross Validation (LOOCV)
See For Yourself
Which K Should You Pick?
Conclusion
Built with Mathigon

Share

Send us Feedback

Please let us know if you have any feedback and suggestions, or if you find any errors and bugs in our content.

Sorry, your message couldn’t be submitted. Please try again!

Thanks for your feedback!

Reset Progress

This will delete your progress and chat data for all chapters in this course, and cannot be undone!

Glossary

Select one of the keywords on the left…

Testing Your AIIntroduction

Reading time: ~5 min

Imagine a student preparing for a massive final exam. If their teacher hands them a practice test, and the exact same questions magically appear on the actual final, what does their perfect score actually prove? It simply proves that the student memorized the answers! It doesn’t tell you whether they truly understood the subject.

The exact same problem happens in machine learning. If we evaluate an AI using the precise data it used to learn (its "training data"), the AI will look like an absolute simply because it memorized the answers. We won't know if it can actually "generalize" and perform well on brand-new, unseen data.

To make sure we have this locked in:

Why shouldn't we test our AI on its own training data?

It takes too long to compute the results.
The model will look artificially perfect by memorizing answers.
It requires too many data points to work properly.

Since constantly collecting brand new real-world data to test our AI is extremely expensive and time-consuming, we need a clever way to securely test our model's true abilities using only the data we already have. This chapter introduces the absolute gold standard for safely testing your model without needing new data: K-Fold Cross-Validation.

First, we are going to quickly review the standard "Validation Set", and then dive deep into how K-Fold Cross-Validation completely supercharges that technique to guarantee the most reliable test scores possible. We'll also break down the importance of choosing the right value for the magical k variable!

Sina