Accuracy

ELI5

Imagine you’re sorting apples and oranges into two boxes: one for apples and one for oranges. If you place most of the apples in the apple box and most of the oranges in the orange box, you’re doing a good job. Accuracy is like keeping track of how many fruits you put in the right box compared to the total number of fruits. So, if you sorted 90 out of 100 fruits correctly, your accuracy is 90%.

Introduction

Accuracy is one of the most fundamental metrics for evaluating classification models in machine learning. It measures the proportion of correct predictions made by the model out of all predictions. Accuracy is intuitive and easy to understand, making it a popular choice for initial model evaluation.

Definition

In the context of a binary classification problem, where the outcomes are labeled as positive (P) or negative (N), predictions can be categorized into four types:

•	True Positives (TP): Correctly predicted positive instances.
•	True Negatives (TN): Correctly predicted negative instances.
•	False Positives (FP): Negative instances incorrectly predicted as positive.
•	False Negatives (FN): Positive instances incorrectly predicted as negative.

Mathematical Formula

The accuracy () is calculated using the following formula:



This equation can also be expressed as:



Interpretation

An accuracy score ranges from 0 to 1, where:

•	1 indicates perfect accuracy—all predictions are correct.
•	0 indicates no correct predictions.

For example, an accuracy of 0.8 (or 80%) means that 80% of the predictions made by the model are correct.

Example Calculation

Suppose we have a dataset of 100 samples:

•	True Positives (TP): 50
•	True Negatives (TN): 30
•	False Positives (FP): 10
•	False Negatives (FN): 10

Calculating accuracy:



So, the model has an accuracy of 80%.

When to Use Accuracy

Accuracy is most effective when:

•	The classes in the dataset are balanced (i.e., roughly equal numbers of positive and negative instances).
•	The cost of false positives and false negatives is similar.

Limitations

While accuracy is a useful metric, it has limitations:

•	Class Imbalance: In datasets where one class significantly outnumbers the other, accuracy can be misleading. For instance, if 95% of the instances are negative, a model that predicts every instance as negative will have 95% accuracy but is useless.
•	Does Not Differentiate Error Types: Accuracy treats all errors equally, not distinguishing between false positives and false negatives, which may have different implications in real-world applications.

Alternative Metrics

In cases where accuracy is not sufficient, consider using:

•	Precision: Measures the correctness of positive predictions.
•	Recall (Sensitivity): Measures the ability to find all positive instances.
•	F1 Score: Harmonic mean of precision and recall.
•	Area Under the ROC Curve (AUC-ROC): Measures the ability of the model to discriminate between classes.

Conclusion

Accuracy is a fundamental metric that provides a quick assessment of a model’s performance. However, it should be used cautiously, especially in imbalanced datasets. Complementing accuracy with other metrics ensures a more comprehensive evaluation of the model.

Next Steps:

•	Explore other metrics like Precision and Recall for a deeper understanding.
•	Learn about handling imbalanced datasets in our Imbalanced Data Guide.