What is a ROC curve?

A ROC curve (Receiver Operating Characteristic curve) is a graphical representation used to evaluate the performance of a binary classification model. It shows the trade-off between sensitivity (True Positive Rate) and specificity (1 – False Positive Rate) as the decision threshold of the classifier changes.

Key Components of the ROC Curve:

1. True Positive Rate (TPR) / Sensitivity / Recall:
   • TPR is the proportion of actual positive cases that are correctly identified by the model.
   • Formula:
     • TPR= TP/(TP+FN )​
   •  Where:
     • TP = True Positives
     • FN = False Negatives
2. False Positive Rate (FPR):
   • FPR is the proportion of actual negative cases that are incorrectly classified as positive by the model.
   • Formula:
     • FPR=FP/(FP+TN)​
   • Where:
     • FP = False Positives
     • TN = True Negatives
3. Threshold:
   • The threshold is the value at which you decide whether a model’s output probability is classified as positive or negative. For example, if the probability is above 0.5, classify as positive, otherwise as negative.
   • The ROC curve shows how the TPR and FPR change as the threshold varies from 0 to 1.

ROC Curve Construction:

1. Plot the ROC Curve:
   • On the x-axis, plot the False Positive Rate (FPR).
   • On the y-axis, plot the True Positive Rate (TPR).
   • As the threshold is adjusted, the model will make different classifications, and the TPR and FPR will change accordingly.
   • The ROC curve is a plot of these values.
2. Area Under the Curve (AUC):
   • The AUC is a single scalar value that summarizes the overall performance of the classifier.
   • AUC ranges from 0 to 1:
     • An AUC of 0.5 means the model is no better than random guessing.
     • An AUC of 1.0 means perfect classification.
     • A higher AUC indicates better performance.

Interpreting the ROC Curve:

• Good Classifier: A ROC curve that is closer to the top-left corner indicates a better classifier, meaning high TPR and low FPR.
• Bad Classifier: If the ROC curve is close to the diagonal line (45-degree line from bottom-left to top-right), the classifier is performing similarly to random guessing.

Example:

For a binary classifier (e.g., spam detection), the ROC curve helps you decide:

• How much false alarm (false positive) you are willing to tolerate in exchange for detecting more actual spam (true positives).
• You can adjust the threshold to either be more strict (lowering false positives) or more lenient (increasing true positives but allowing more false positives).

ROC vs. Precision-Recall Curve:

• ROC curves are especially useful when you have balanced datasets. However, for imbalanced datasets (where one class is much more frequent than the other), a Precision-Recall curve may provide more insight into classifier performance.

Example Use Case:

If you are building a model to predict whether a machine is likely to fail (positive class) or not (negative class), an ROC curve helps you evaluate:

• How well the model distinguishes between the failure and non-failure states.
• How changing the threshold affects the balance between correctly predicting failures (TPR) and incorrectly labeling non-failures as failures (FPR).

Code Example:

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_curve, auc

# Step 1: Generate synthetic binary classification dataset
X, y = make_classification(n_samples=1000, n_features=20, n_classes=2, random_state=42)

# Step 2: Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Step 3: Train a logistic regression classifier
classifier = LogisticRegression()
classifier.fit(X_train, y_train)

# Step 4: Get predicted probabilities (we need probabilities for the positive class)
y_prob = classifier.predict_proba(X_test)[:, 1] # Probabilities for the positive class (class 1)

# Step 5: Compute ROC curve and AUC
fpr, tpr, thresholds = roc_curve(y_test, y_prob)
roc_auc = auc(fpr, tpr)

# Step 6: Plot the ROC curve
plt.figure(figsize=(8, 6))
plt.plot(fpr, tpr, color=’blue’, label=f’ROC curve (AUC = {roc_auc:.2f})’)
plt.plot([0, 1], [0, 1], color=’gray’, linestyle=’–‘) # Diagonal line (random classifier)
plt.title(‘ROC Curve’)
plt.xlabel(‘False Positive Rate (FPR)’)
plt.ylabel(‘True Positive Rate (TPR)’)
plt.legend(loc=’lower right’)
plt.grid(True)
plt.show()