The area under the PR curve, often abbreviated as PR AUC, is a metric used in machine learning to evaluate the performance of a binary classifier. PR stands for Precision–Recall, so the PR curve is a plot of the precision (how many selected items are relevant) versus recall (how many relevant items are selected) at different threshold settings. The area under this curve gives a single value summarizing the trade-off between precision and recall for the model across all thresholds.
Understanding the area under the PR curve is crucial, especially when dealing with imbalanced datasets. In many real-world scenarios—like fraud detection, rare disease diagnosis, or spam detection—one class (e.g., fraudulent cases or positive diagnoses) is much rarer than the other. In these cases, metrics like accuracy or even ROC AUC can be misleading, because they may not properly capture how well a model identifies the rare but vital positive cases. The PR AUC, in contrast, focuses specifically on the performance with respect to the positive class.
To construct the PR curve, you vary the decision threshold for what counts as a positive prediction. At each threshold, you calculate precision and recall. Plotting these points gives you the PR curve. The area under this curve (AUC) is then computed, usually ranging from 0 to 1. A PR AUC of 1.0 means perfect precision and recall, while a value close to zero indicates poor performance. A random classifier’s PR AUC is roughly equal to the proportion of positive instances in the dataset, so the baseline can be quite low if positives are rare.
Why choose PR AUC over other metrics? Unlike the ROC curve, which plots true positive rate against false positive rate, the PR curve is more sensitive to the presence of false positives and focuses on the positive class. This makes PR AUC especially informative when the positive class is the minority and false positives are costly. For example, in medical testing, you want to ensure that the test catches as many real cases as possible (high recall), but also that the positive predictions are trustworthy (high precision).
Interpreting PR AUC is straightforward: higher values mean better performance, but always compare it relative to the baseline (the fraction of positives in your data). If your model’s PR AUC is only slightly above the baseline, it may not be much better than random chance. If it’s significantly higher, your model is effectively distinguishing the positive examples from the negatives.
When using the area under the PR curve, remember that it is most meaningful in the context of imbalanced datasets and when the costs of false positives and false negatives are not equal. It provides a more nuanced and informative view of classifier performance than accuracy or ROC AUC in these situations.
