Support for customized channel TIF input image

[YubinXie]

I am currently working on medical images, where one field of view could have more than 3 channels (up to 50+ channels). Current loader does not support customized channel input. If you give it a stacked TIF image, it use convert 'RGB' function to make it something else. If I change the loader and make it output the n-d array of the input image, it gives future error because it assumes PIL image format as input. Any solutions? Thanks.

[fmassa]

I think in order to properly support this it would be better to refactor our to_tensor / to_pil_image to be more generic.

As a workaround, do you need to perform any data augmentations in your code (rotation, flip, etc)?
If not, you can just use the LambdaTransform to convert your array into a torch tensor.

Also, do you want to normalize the image to be in a specific range?

It would be great if we could have some pointers of some reference images and tools that could be used to load them, so that we can take that into account when improving those transforms.

[YubinXie]

It worked if I have no transformation. But I do need transformation and normalization.

One thing I felt weird is that, current transformation is done via PIL, that is the reason PIL format is required. I think it would make more sense if you can do transformation on the np.array or tensor format.

[fmassa]

Yes, requiring the image to be a PIL image has some limitations.
That's why I'm thinking about modifying the existing transforms and create a wrapper Image class, that could hold other formats as well.

But for that to work, we would need to know a bit better how the other formats are structured, and what is the representation that is used. Could you give us some pointers?

[YubinXie]

Majority images are grey or RGB. The only images I know that contain channel number above 4 is TIF stacked image. So I would say add a support for TIF stacked image for read-in part. Once read-in an image, convert it to numpy array, or pytorch tensor. And apply all the transformation on these.

[vadimkantorov]

@fmassa It would be nice if PyTorch supported running without Pillow(-SIMD which can have installation issues, since it requires to uninstall pillow package which some other pip packages can reinstall back).

With nn.functional.upsample and .flip in core, is image file loading the only thing that PIL is indispensable?

[fmassa]

@vadimkantorov almost.
Note that for the moment, upsample only works on floating point data, but I'd expect that we would add support for other data types in the near future.
Also, there are a few other operations that require a bit of work to get running (like rotations), even if that could be implemented using grid_sample (for floating point types only).

One thing I was thinking was to eventually extend the support of grid_sample to also perform bicubic interpolation, I think it could be useful for a few applications.

[mayaghaei]

I just discovered a way to do this, I am not sure it can be solution to your problem, but I'll share it in case it can be useful to others.
In my case I had multi-channel Tiff images, and I wanted to classify them using CNNs in Pytorch. I honestly gave up on data augmentation using Transforms in Pytorch, and I performed data augmentation offline (let's say in my input folders I have original data as well as augmented ones).
The game changer is however defining your own loader + taking advantage of Tifffile library in python. This is how I did it for my training set (val and test should be the same):

import tifffile
def my_tiff_loader(filename):
    return tifffile.imread(filename)

train_transform = transforms.Compose([transforms.ToTensor()])

train_data = datasets.ImageFolder('PATH TO TRAINSET', loader=my_tiff_loader, transform=train_transform)