Accuracy from the Matrix

This is the easiest score to calculate and the easiest to be fooled by. Both halves of that sentence are examined.

1The formula

Classification accuracy

The number of correct predictions as a ratio of all predictions made.
Accuracy = Correct predictions / Total predictions
= (TP + TN) / (TP + TN + FP + FN)
Multiply by 100 for a percentage.

The top of the fraction is the two True cells. The bottom is all four cells, which is just the size of the test set.

2Where accuracy lies to you

Now the part that matters. Suppose you test a model on 1000 students. In reality 900 pass and only 100 fail — an unbalanced dataset. And suppose you have built a deliberately faulty model that, whatever input it is given, always answers “Yes, they pass”.

It never looks at the input at all. What accuracy does it score?

The confusion matrix, and what comes out of it

Pick a metric to light up the cells it is built from. Precision is the top row; recall is the left column.

	Actual / Reality
	Yes	No
Predicted Yes	TP 900	FP 100
Predicted No	FN 0	TN 0

Accuracy = (900 + 0) / 1000 = 90.00%

Precision = 900 / (900 + 100) = 0.9000

Recall = 900 / (900 + 0) = 1.0000

F1 = 2 × 0.9000 × 1.0000 / (0.9000 + 1.0000) = 0.9474

The model that always says Yes. Positive class: Student passes. It never looks at the input, yet it scores 90% accuracy — because 900 of the 1000 students really did pass. This is why accuracy alone cannot be trusted on an unbalanced dataset.Source: Class X handbook, Activity 3

90%

The faulty model scores 90% accuracy— because 900 of the 1000 really did pass, so saying “pass” every time is right 900 times. It is a model that has learned nothing, cannot be wrong in an interesting way, and would be useless to any school. And its accuracy beats plenty of genuinely careful models.

Forecasting sunshine in a desert

Predict “no rain” every single day in a place that gets rain five days a year, and you will be right 98% of the time. You are not a meteorologist. You are a person who says the same thing every morning.

3When accuracy can be used

The condition

Accuracy is only suitable when there are roughly equal numbers of observations in each class — a balanced dataset, which is rarely the case — and when all predictions and prediction errors are equally important, which is often not the case either.

When the data is unbalanced, use precision, recall or the F1 score instead. Those are the next two lessons.

4From the board papers

A school tested an AI model that predicts whether students pass or fail. Out of 100 students, it correctly predicted 40 would pass and they did; it correctly identified 30 who would fail; it predicted 20 would pass but they failed; and it predicted 10 would fail but they passed.
(a) Draw the confusion matrix. (b) Calculate the accuracy, showing your working. (c) Write the total number of wrong predictions.CBSE sample paper 2025–26

4 marks

▶Show the answer

(a) Taking “passes” as the positive class: TP = 40, FP = 20, FN = 10, TN = 30.

	Actual: Pass	Actual: Fail
Predicted: Pass	40	20
Predicted: Fail	10	30

(b) Accuracy = (TP + TN) / total = (40 + 30) / 100 = 70/100 = 0.7 = 70%.
(c) Wrong predictions = FP + FN = 20 + 10 = 30. (And 70 + 30 = 100, which checks out.)

Which evaluation parameter takes into consideration all the correct predictions?CBSE sample papers 2023–24 and 2024–25

1 mark

▶Show the answer

Accuracy. It is the only one of the four whose numerator is both True cells, TP + TN. Precision and recall each use TP only.

What is classification accuracy? Can it be used at all times for evaluating AI models?Class X handbook, Reflection Time

2 marks

▶Show the answer

Classification accuracy is the number of correct predictions as a ratio of all predictions made, that is (TP + TN) / (TP + TN + FP + FN).
No, it cannot be used at all times. It is only suitable when the dataset is balanced and when all prediction errors matter equally. On an unbalanced dataset a model that always predicts the majority class can score very high accuracy while being useless, so precision, recall or F1 score should be used instead.

In a medical test for a rare disease, 1000 people are tested. 50 actually have the disease and 950 do not. The test correctly identifies 40 of the 50 as positive, but also wrongly identifies 30 healthy people as positive. What is the accuracy of the test?Class X handbook, Test Yourself — see the note below

1 mark

▶Show the answer

TP = 40, FN = 50 − 40 = 10, FP = 30, TN = 950 − 30 = 920.
Accuracy = (40 + 920) / 1000 = 960/1000 = 0.96 = 96%.

The textbook’s four options are 97%, 90%, 85% and 70% — none of which is the answer to its own question. Work it from the formula and write 96%. Notice too how high that number is while the test missed 10 of the 50 sick people: 950 of the 1000 were healthy, so this is an unbalanced dataset and accuracy is flattering it.

5Check yourself

Quick Check

What is the formula for classification accuracy?

Quick Check

When is accuracy NOT a suitable metric?

Quick Check

A model tested on 1000 samples has TP = 200, TN = 600, FP = 100, FN = 100. Its accuracy is:

← Previous

🔲 The Confusion Matrix

Precision and Recall ⚖️