Broken reading for some GIFs

[abionics]

🐛 Describe the bug

I have encountered an issue where using torchvision.io.read_image (as well as decode_image and decode_gif) on certain GIF files produces an invalid output. Here is a code:

import cv2
from torchvision.io import read_image

gif = read_image('sample.gif').numpy().transpose(0, 2, 3, 1)
image = gif[15]  # 15-th frame

image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
cv2.imshow('read_torch', image)
cv2.waitKey()

Sample of buggy GIF:

Its source: https://i.pinimg.com/originals/a0/02/a3/a002a3e51a2adc85d6c0a4684892e743.gif

Output:

Comparing with PIL (Pillow)

I understand that this may be an underlying issue of GIFLIB and will not be fixed. However, since the behavior is unexpected and could impact users, it might be beneficial to include a note in the docs.

Versions

I found this behavior on Macbook and Linux server. They have different setup, but both of them use the latest versions:

torch==2.6.0
torchvision==0.21.0

[AntoineSimoulin]

Hey @abionics, Thanks for reporting the issue, and apologies for the delayed response.

After investigating, I suspect this behavior is due to how GIF animations are optimized. Many GIFs store only the parts of the image that change between frames. These incremental frames are then layered over the previous ones during playback. As a result, to accurately display any given frame, you need to reconstruct it by compositing it with all the preceding frames. This Stack Overflow thread provides a good overview: Incomplete images from decoded GIF.

In our case, I suspect the issue is that we're not properly reconstructing the full frame before displaying it. Indeed, the first frame renders correctly as it doesn’t rely on any prior frames. Does my analysis makes sense to you?

import cv2
from torchvision.io import read_image

cv2.imshow('read_torch', image)
gif = read_image('sample.gif').numpy().transpose(0, 2, 3, 1)
image = gif[0]  # 1-st frame

image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
cv2.imshow(image)

[abionics]

Hi @AntoineSimoulin, thank you for detailed response and investigation!

It makes sense and it looks like a real reason. Additionally, I uploaded this GIF to online splitter and tried different modes, here is a link. The "Redraw every frame with details from previous frames" mode gives a correct result, while "Ignore optimizations" mode shows only the changed parts (as read_image does)