Add more complete descriptions for PixelFormats

[chcunningham]

For instance, for I420 we could mention

• the number of planes is 3
• they are Y, U, and V
• their horizontal pixel sampling rate is 1, 2, and 2, respectively.
• their vertical pixel sampling rate is ""

This is good for interop and documentation. It also allows us to give more complete steps to validate PlaneInit dictionaries.

[chcunningham]

Triage note: marking 'editorial' as we have broad consensus on the meaning of well known pixel formats (certainly true for the single I420 currently in the spec), and this issue merely tracks the work to document it's meaning.

[padenot]

Yes, but this needs to be specified normatively and unambiguously before any implement ships. I think we all agree what it is though:

(w x h) pixels of luma, and then immediately (w x h / 4) pixels for the U chroma plane and then immediately(w x h / 4) pixels for V Chroma plane. w and h are the coded width and height, not necessarily the display width and height.

In this format, a component for a pixel is 8-bits in all 3 planes.

Because of the API, I don't think it's necessary to specify whether the Y, U and V planes linear and contiguous in memory when the buffer is in regular memory: it's always access via readInto (modulo names changes).

If an implementation wants to ship any other format it needs to be specified unambiguously in the same fashion.

[padenot]

Also Y U and V are in this precise order.

[chcunningham]

I believe we agree on the definition given above. @sandersdan to double check me.

[sandersdan]

Also Y U and V are in this precise order.

Just to clarify, that is how the APIs handle the planes but does not describe their layout in memory (which as you mentioned is arbitrary).

I'll add only that there are also general restrictions I would like to apply to all formats that also apply to I420:

• coded sizes must be a multiple of the largest subsampling factor: I420 coded sizes must be even.
• crop coordinates must be a multiple of the largest subsampling factor: I420 crops must have even origin and size values.

We could go further and specify byte ordering (eg. top-left at lowest memory address, increasing along rows up the the stride size, then wrapping to the left of the next row). Again this is probably the same for all formats, but it can cause confusion due to GL using y-flipped coordinates relative to this definition.

(Edit: just realized that stride wasn't mentioned in any post before this one. Stride behavior should be the same for all formats, but it does affect the calculations mentioned above; w * h is only meaningful when stride == planeCodedWidth, in general it's h * stride bytes per I420 plane.)

[padenot]

Thanks for the comment. I agree with everything and will write a PR with all of this.

[chcunningham]

@padenot good to close now that #288 is merged?