Log Loss, also known as logarithmic loss or cross-entropy loss, is a popular performance metric used to evaluate the accuracy of classification models, especially when the model outputs probabilities for each class rather than hard labels. It quantifies how closely the predicted probabilities match the actual class labels in a dataset. The lower the Log Loss, the better the model’s predictions align with the real outcomes.
In mathematical terms, Log Loss penalizes predictions that are both confident and wrong. If a model predicts a very high probability for the incorrect class (for example, 0.99 when the true label is 0), Log Loss increases significantly. This makes it a stricter metric than simple accuracy, which only checks if the highest-probability prediction is correct, regardless of the actual probability assigned.
Log Loss is calculated as the negative average of the log of predicted probabilities assigned to the true classes. For binary classification, the formula for a single example is:
– (y * log(p) + (1 – y) * log(1 – p))
Here, y is the actual label (0 or 1), and p is the predicted probability that the label is 1. For multi-class classification, the formula generalizes to sum the log probabilities over all classes. The log function used here is generally base e (the natural logarithm), and the loss is averaged over all samples in the dataset.
One of the reasons Log Loss is widely used in machine learning is that it provides a smooth, differentiable metric. This makes it suitable as a loss function to train classification models using optimization techniques like gradient descent. Since Log Loss reacts strongly to incorrect predictions made with high confidence, it encourages models not only to predict the correct class but also to be well-calibrated in their probability estimates.
Log Loss is commonly used in evaluating models such as logistic regression, neural networks, and gradient boosted trees. It is especially valuable in situations where probability calibration matters, such as in medical diagnosis or risk assessment, where knowing how confident a model is in its predictions can be as important as the predictions themselves.
When working with imbalanced datasets, Log Loss can reveal issues that accuracy might mask. For instance, if one class is much more frequent than others, a model that always predicts the majority class may have high accuracy but a poor Log Loss, since it fails to assign appropriate probabilities to the minority class.
Overall, Log Loss is a key metric for those building and evaluating machine learning models with probabilistic outputs. Understanding it helps practitioners develop more reliable, interpretable, and trustworthy AI systems.
