Train-Test Split

Before you can measure anything, you have to decide what to measure it on. Get that wrong and every metric in the rest of this unit will lie to you.

1What the split is

Train-test split

A technique for evaluating the performance of a machine learning algorithm. The procedure takes a dataset and divides it into two subsets: the training dataset and the testing dataset. It can be used for any supervised learning algorithm, and it is appropriate when there is a sufficiently large dataset available.

Each subset has exactly one job:

The training set is used to make the model learn.
The input elements of the testing set are given to the trained model. It makes predictions, and those predicted values are compared with the expected values.

The objective is to estimate how the model will perform on new data — data that was not used to train it. That is how the model will really be used: fitted on data whose answers you know, then asked about cases where you do not know the answer yet.

One table, three ways of looking at it

The first three columns are the features. The last one is the label — the answer we want the machine to give.

Colourfeature	Sizefeature	Shapefeature	Fruitlabel
Red	Small	Round	Strawberry
Red	Medium	Round	Apple
Yellow	Medium	Long	Banana
Green	Medium	Round	Apple
Orange	Medium	Round	Orange
Yellow	Long	Long	Banana

Labelled data. Every row carries its answer. A machine can learn the link between the features and the label from this — which is what supervised learning needs.

2Never mark your own homework

It is not recommended to evaluate a model on the data used to build it. The model will simply remember the whole training set, and will therefore always predict the correct label for any point in it.

Test it on what it studied, or on something new?

The same model, marked two ways. One of these numbers is a lie.

Marked on the training set

—

Press the first button.

Marked on the testing set

—

Press the second button.

Same model, two ways of marking it. Try both and compare.

Overfitting

Overfitting is when a model memorises its training data instead of learning the general pattern. It then scores brilliantly on data it has already seen and performs badly on anything new.

The test with the answers attached

A teacher who sets a test made only of the questions she solved in class, with the worked answers still on the board, will get a class full of full marks. She has learnt nothing about who understands the topic. The marks are real; the conclusion drawn from them is worthless.

3From the board papers

Why is it not recommended to evaluate a machine learning model using the same data on which it was trained?
(a) It helps the model learn faster (b) It reduces bias in the dataset (c) It improves performance on unseen data (d) It may lead to overfitting and give an overly optimistic accuracyCBSE sample paper 2025–26

1 mark

▶Show the answer

(d) It may lead to overfitting and give an overly optimistic accuracy. The number you get back is high and meaningless — “overly optimistic” is exactly the right phrase for it.

Sarthak made a face mask detector. He collected a dataset and used all of it to train the model. Then he used the same data to evaluate it, which gave the correct answer every time — but the model could not perform on an unknown dataset. Name the concept.CBSE sample papers 2023–24 and 2024–25

1 mark

▶Show the answer

Overfitting. The two halves of the question are the definition: perfect on the data it was trained on, useless on anything it has not seen.

Explain train-test split with an example.Class X handbook, Reflection Time

2 marks

▶Show the answer

Train-test split is a technique for evaluating a machine learning algorithm by dividing the dataset into two subsets — a training set and a testing set. The training set is used to make the model learn; the testing set is then fed to the trained model and its predictions are compared with the expected values, so that the model’s performance on new data can be estimated.
For example, with 1000 labelled images of masked and unmasked faces you might train on 800 and keep 200 aside. The model never sees those 200 while learning, so how well it does on them is a fair estimate of how it will behave on a face it has never met.

4Check yourself

Quick Check

What is the train-test split used for?

Quick Check

A model that memorises its training data and fails on new data is said to be:

Quick Check

Which subset does the model learn from?

← Previous

📏 What is Evaluation?

Accuracy and Error 🎯