Binary semantic segmentation using SegFormer
Hello, I’m trying to perform binary segmentation, using SegFormer on a custom dataset, where the mask (PNG grayscale image) consists of value1for object of interest and0for everything else. I’ve usedthistutorial and therefore I appliedreduce_labels=Truewhich leaves me with label0for object and255for background. However, when I pass labels to model I’m getting this error:ValueError: The number of labels should be greater than oneHow should I fix it, please? Should I consider region outside of my object not as a background, but rather as another label?
2022-11-08T20:53:46Z
Hi,The current implementation of SegFormer actually only supports multi-class classification per pixel, using the regular CrossEntropyLoss.For binary classification per pixel, one should actually use theBCEWithLogitsLoss(which applies a sigmoid activation on the logits + computes cross-entropy loss).I’ll add support for this! Thanks for bringing this up.
2022-11-08T22:24:49Z
Hello,Thank you for your answer. I’m still wondering if this couldn’t be overcome by pretending there are two classes (object and not object), therefore two labels and no background. Or something like a virtual background on255which simply won’t be present in training data.Thanks.
2022-11-09T08:37:40Z
Yes that’s totally possible, in that case you can just use the regular cross-entropy loss. Segmentation maps should in that case contain values 0 and 1 (which the model needs to learn for each pixel), and you can use 255 for pixels you want to ignore (i.e. the model shouldn’t learn). So you can set config.num_labels to 2 in that case.
2022-11-16T12:29:37Z
Hello,Thank you very much. Can you please reply when you add support for BCEWithLogitsLoss?Thanks
2022-11-16T15:21:37Z
Hi,@Ivor22, I’ve opened a PR here:[SegFormer] Add support for segmentation masks with one label by NielsRogge · Pull Request #20279 · huggingface/transformers · GitHub.If it works fine for you, I’ll add it to all other models.
2023-07-26T09:54:44Z
As this seems to work now, should I usenum_labels = 1in this case?I think the documentation still only refers and explains thenum_labels >1case?And apologize, from the github pull request, is this now automatically using dice loss?