[prototype] The new Transforms API should support different fill values for Segmentation masks than Images

[datumbox]

🚀 The feature

While migrating the reference scripts for Segmentation to the latest API, @pmeier noticed that one of the transforms uses a different fill value for Images vs Segmentation masks:

vision/references/segmentation/transforms.py

Lines 59 to 60 in d367a01

	image = pad_if_smaller(image, self.size)
	target = pad_if_smaller(target, self.size, fill=255)

The 255 value is used to encode values that the loss should ignore:

vision/references/segmentation/train.py

Line 39 in d367a01

losses[name] = nn.functional.cross_entropy(x, target, ignore_index=255)

This means that we should have different fill values for Images and Segmentation masks. Other libraries such as fvcore already do this by adding separate fill pad_value and seg_pad_value parameters.

It's worth noting that a parameter behaving similarly is interpolation only that in the case of masks it's fixed to nearest and thus it's not necessary to have 2 of them.

Motivation, pitch

Here is what needs to be done:

• All eligible kernels (affine, rotate, pad, perspective, elastic, etc) that could receive a fill parameter should receive it for for SegmentationMask
• Make it possible for all related Transforms (Pad, RandomZoomOut, RandomRotation, RandomAffine, ElasticTransform, FixedSizeCrop, etc) to receive different parameters for Image and Masks.
• Update the tests.

Alternatives

No response

Additional context

No response

cc @vfdev-5 @bjuncek @pmeier

[vfdev-5]

Done in #6599 and #6586