cv-focal-loss
Alpha-weighted focal loss that down-weights easy examples to focus training on hard, misclassified pixels in imbalanced segmentation tasks.
SKILL.mdskills/cv/focal-loss/SKILL.md
GitHubFocal Loss
Overview
Focal loss addresses extreme class imbalance in segmentation (e.g., defects covering <1% of pixels). Standard BCE treats all pixels equally, so the loss is dominated by easy negatives. Focal loss adds a modulating factor (1-p_t)^gamma that suppresses the contribution of well-classified examples. With gamma=2, a pixel classified at 0.9 confidence contributes 100x less loss than one at 0.5. Combined with alpha-weighting for positive/negative balance, focal loss typically improves recall on rare classes by 5-15%.
Quick Start
import torch
import torch.nn as nn
import torch.nn.functional as F
class FocalLoss(nn.Module):
def __init__(self, alpha=0.8, gamma=2.0):
super().__init__()
self.alpha = alpha
self.gamma = gamma
def forward(self, inputs, targets):
inputs = torch.sigmoid(inputs).view(-1)
targets = targets.view(-1)
bce = F.binary_cross_entropy(inputs, targets, reduction='none')
p_t = inputs * targets + (1 - inputs) * (1 - targets)
focal_weight = self.alpha * (1 - p_t) ** self.gamma
return (focal_weight * bce).mean()
# Usage
criterion = FocalLoss(alpha=0.8, gamma=2.0)
loss = criterion(logits, masks)
Workflow
- Apply sigmoid to raw logits
- Compute per-pixel binary cross-entropy
- Compute modulating factor
(1-p_t)^gammato down-weight easy examples - Scale by alpha for positive/negative class balance
- Average over all pixels
Key Decisions
- Gamma: 2.0 is standard; higher (3-5) for extreme imbalance, lower (0.5-1) for mild
- Alpha: 0.8 gives more weight to positives (minority class); tune on validation
- vs BCE: Use focal when positive pixels are <5% of total; BCE is fine for balanced masks
- Combining: Can add to Dice loss for a focal-dice hybrid