Repository navigation
FocalLoss mean reduction is multiplied by the number of channels when ignore_index is set #9214
Description
Describe the bug
The text below was written with AI assistance (Claude and GPT-6 Astra).
With ignore_index set, FocalLoss(reduction="mean") divides the summed loss by mask.sum() (
MONAI/monai/losses/focal_loss.py
Lines 240 to 245 in 6920610
ignore_index to a label that never occurs changes the loss from x to C * x, and the gradients scale with it. The "sum" reduction and AsymmetricFocalLoss (mask.expand_as(loss).sum()) are not affected.
I found this by reading and testing the code.
To Reproduce
import torch
from monai.losses import FocalLoss
torch.manual_seed(0)
logits = torch.randn(2, 4, 16, 16, 8)
labels = torch.randint(0, 4, (2, 1, 16, 16, 8))
plain = FocalLoss(to_onehot_y=True, use_softmax=True)(logits, labels)
masked = FocalLoss(to_onehot_y=True, use_softmax=True, ignore_index=255)(logits, labels) # no voxel is 255
print(plain.item(), masked.item(), (masked / plain).item())Output on dev (6920610):
0.30678269267082214 1.2271307706832886 4.0
On MONAI's MNI152 2 mm T1 test volume with its 6-label structural segmentation (small UNet logits), the ratio is 6.0. With the top 40% of axial slices set to 255, FocalLoss(ignore_index=255) gives 1.795147, while the same loss on only the labelled slices gives 0.299191.
Expected behavior
Ignoring voxels should give the same mean as removing them, and an unused ignore_index should not change the loss.
Environment
MONAI 1.6.1rc0+50.g6920610c (dev), PyTorch 2.14.1+cpu, NumPy 2.5.4, Python 3.12.13, Linux.
Additional context
I have a one-line fix that counts the mask once per loss channel, as AsymmetricFocalLoss already does, plus a regression test, and can open a PR.
I made this report and directed the investigation, with AI assistance from Claude and GPT-6 Astra.
Activity