From 8479a9bce99c5d9776776bb2e8287542a5aac4b6 Mon Sep 17 00:00:00 2001 From: Chris Cunningham Date: Tue, 16 Mar 2021 10:25:21 -0700 Subject: [PATCH 1/9] Add Image Decoding, associated interfaces and algorithms Fixes #50. --- index.src.html | 780 ++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 742 insertions(+), 38 deletions(-) diff --git a/index.src.html b/index.src.html index 02a8b26a..5e6ce31b 100644 --- a/index.src.html +++ b/index.src.html @@ -9,9 +9,8 @@ Editor: Chris Cunningham, w3cid 114832, Google Inc. https://google.com/ Editor: Paul Adenot, w3cid 62410, Mozilla https://www.mozilla.org/ -Abstract: This specification defines interfaces for encoding and decoding audio -Abstract: and video. It also includes an interface for retrieving raw video -Abstract: frames from MediaStreamStracks. +Abstract: This specification defines interfaces for encoding and decoding audio, +Abstract: video, and images. Markup Shorthands:css no, markdown yes, dfn yes !Participate: Git Repository. @@ -48,6 +47,7 @@ spec: mimesniff; urlPrefix: https://mimesniff.spec.whatwg.org/# type: dfn; text: MIME type; url: mime-type + type: dfn; text: valid MIME type string; url:valid-mime-type spec: infra; urlPrefix: https://infra.spec.whatwg.org/# type: dfn; text: queue; url: queues @@ -64,6 +64,12 @@ type: attribute; text: powerEfficient; url: dom-mediacapabilitiesinfo-powerefficient + + Definitions {#definitions} ========================== @@ -101,8 +107,22 @@ :: A set of parameters describing a sequence of AVC coded video as defined by [[!iso14496-10]]. -Codec Processing Model {#codec-processing-model} -================================================ +: Progressive Image +:: An image that supports decoding to multiple levels of detail, with lower + levels becoming available while the encoded data is not yet fully buffered. + +: Progressive Image Frame Generation +:: A generational identifier for a given [=Progressive Image=] decoded output. + The mechanism for computing a frame's generation is implementer / format + defined. + +: Primary Image Track +:: An image track that is marked by the given image file as being the default + track. The mechanism for indiciating a primary track is format defined. + + +Codec Processing Model {#codec-processing-model-section} +=================================================================== Background {#processing-model-background} ----------------------------------------- @@ -278,7 +298,8 @@ 2. Increment {{VideoDecoder/decodeQueueSize}}. 3. [=Queue a control message=] to decode the |chunk|. - Running a control message to decode the chunk means performing these steps: + [=Running a control message=] to decode the chunk means performing these + steps: 1. Attempt to use {{VideoDecoder/[[codec implementation]]}} to decode the chunk. 2. If decoding results in an error, queue a task on the [=control thread=] @@ -305,7 +326,7 @@ 3. [=Queue a control message=] to flush the codec with |promise|. 4. Return |promise|. - Running a control message to flush the codec means performing these steps + [=Running a control message=] to flush the codec means performing these steps with |promise|. 1. Signal {{AudioDecoder/[[codec implementation]]}} to emit all [=internal pending outputs=]. @@ -506,7 +527,7 @@ 2. Increment {{VideoDecoder/decodeQueueSize}}. 3. [=Queue a control message=] to decode the |chunk|. - Running a control message to decode the chunk means performing these steps: + [=Running a control message=] to decode the chunk means performing these steps: 1. Attempt to use {{VideoDecoder/[[codec implementation]]}} to decode the chunk. 2. If decoding results in an error, queue a task on the [=control thread=] @@ -533,7 +554,7 @@ 3. [=Queue a control message=] to flush the codec with |promise|. 4. Return |promise|. - Running a control message to flush the codec means performing these steps + [=Running a control message=] to flush the codec means performing these steps with |promise|. 1. Signal {{VideoDecoder/[[codec implementation]]}} to emit all [=internal pending outputs=]. @@ -598,29 +619,34 @@
Run these steps: 1. For each |output| in |outputs|: - 1. Let |planes| be a sequence of {{Plane}}s containing the decoded video - frame data from |output|. - 2. Let |pixelFormat| be the {{PixelFormat}} of |planes|. - 3. Let |frameInit| be a {{VideoFrameInit}} with the following keys: - 1. Let {{VideoFrameInit/timestamp}} and {{VideoFrameInit/duration}} - be the {{EncodedVideoChunk/timestamp}} and - {{EncodedVideoChunk/duration}} from the {{EncodedVideoChunk}} - associated with |output|. - 2. Let {{VideoFrameInit/codedWidth}} and - {{VideoFrameInit/codedHeight}} - be the width and height of the decoded video frame |output| in - pixels, prior to any cropping or aspect ratio adjustments. - 3. Let {{VideoFrameInit/cropLeft}}, {{VideoFrameInit/cropTop}}, - {{VideoFrameInit/cropWidth}}, and {{VideoFrameInit/cropHeight}} - be the crop region of the decoded video frame |output| in - pixels, prior to any aspect ratio adjustments. - 4. Let {{VideoFrameInit/displayWidth}} and - {{VideoFrameInit/displayHeight}} be the display size of the - decoded video frame in pixels. - 4. Let |frame| be a {{VideoFrame}}, constructed with |pixelFormat|, - |planes|, and |frameInit|. + 1. Let |frame| be the result of running the [=Create a VideoFrame=] + algorithm with |output|. 5. Invoke {{VideoDecoder/[[output callback]]}} with |frame|.
+
Create a VideoFrame (with |output|)
+
+ 1. Let |planes| be a sequence of {{Plane}}s containing the decoded video + frame data from |output|. + 2. Let |pixelFormat| be the {{PixelFormat}} of |planes|. + 3. Let |frameInit| be a {{VideoFrameInit}} with the following keys: + 1. Let {{VideoFrameInit/timestamp}} and {{VideoFrameInit/duration}} + be the {{EncodedVideoChunk/timestamp}} and + {{EncodedVideoChunk/duration}} from the {{EncodedVideoChunk}} + associated with |output|. + 2. Let {{VideoFrameInit/codedWidth}} and + {{VideoFrameInit/codedHeight}} + be the width and height of the decoded video frame |output| in + pixels, prior to any cropping or aspect ratio adjustments. + 3. Let {{VideoFrameInit/cropLeft}}, {{VideoFrameInit/cropTop}}, + {{VideoFrameInit/cropWidth}}, and {{VideoFrameInit/cropHeight}} + be the crop region of the decoded video frame |output| in + pixels, prior to any aspect ratio adjustments. + 4. Let {{VideoFrameInit/displayWidth}} and + {{VideoFrameInit/displayHeight}} be the display size of the + decoded video frame in pixels. + 4. Return a new {{VideoFrame}}, constructed with |pixelFormat|, + |planes|, and |frameInit|. +
Reset VideoDecoder
Run these steps: @@ -731,7 +757,7 @@ 3. Set {{AudioEncoder/state}} to `"configured"`. 4. [=Queue a control message=] to configure the encoder using |config|. - Running a control message to configure the encoder means performing these + [=Running a control message=] to configure the encoder means performing these steps: 1. Let |supported| be the result of running the Check Configuration Support algorith with |config|. @@ -760,7 +786,7 @@ 5. Increment {{AudioEncoder/encodeQueueSize}}. 6. [=Queue a control message=] to encode |frameClone|. - Running a control message to encode the frame means performing these steps. + [=Running a control message=] to encode the frame means performing these steps. 1. Attempt to use {{AudioEncoder/[[codec implementation]]}} to encode |frameClone|. 2. If encoding results in an error, queue a task on the [=control thread=] @@ -787,7 +813,7 @@ 3. [=Queue a control message=] to flush the codec with |promise|. 4. Return |promise|. - Running a control message to flush the codec means performing these steps + [=Running a control message=] to flush the codec means performing these steps with |promise|. 1. Signal {{AudioEncoder/[[codec implementation]]}} to emit all [=internal pending outputs=]. @@ -981,7 +1007,7 @@ 3. Set {{VideoEncoder/state}} to `"configured"`. 4. [=Queue a control message=] to configure the encoder using |config|. - Running a control message to configure the encoder means performing these + [=Running a control message=] to configure the encoder means performing these steps: 1. Let |supported| be the result of running the Check Configuration Support algorith with |config|. @@ -1011,7 +1037,7 @@ 5. Increment {{VideoEncoder/encodeQueueSize}}. 6. [=Queue a control message=] to encode |frameClone|. - Running a control message to encode the frame means performing these steps. + [=Running a control message=] to encode the frame means performing these steps. 1. Attempt to use {{VideoEncoder/[[codec implementation]]}} to encode |frameClone| according to |options|. 2. If encoding results in an error, queue a task on the [=control thread=] @@ -1038,7 +1064,7 @@ 3. [=Queue a control message=] to flush the codec with |promise|. 4. Return |promise|. - Running a control message to flush the codec means performing these steps + [=Running a control message=] to flush the codec means performing these steps with |promise|. 1. Signal {{VideoEncoder/[[codec implementation]]}} to emit all [=internal pending outputs=]. @@ -2394,14 +2420,692 @@ NOTE: User Agents are encouraged to avoid expensive copies of large objects (for instance, {{VideoFrame}} pixel data). Frame types are - immutable, so the above step may be implemented using memory sharing - techniques such as reference counting. + immutable, so the above step may be implemented using memory + sharing techniques such as reference counting. 3. Return |cloneFrame|.
+Image Decoding {#image-decoding} +==================================== + +Background {#image-decoding-background} +------------------------------------- + +This section is non-normative. + +Image codec definitions are typically accompanied by a definition for a +corresponding file format. Hence, image decoders often perform both duties of +unpacking (demuxing) as well as decoding the encoded image data. The WebCodecs +{{ImageDecoder}} follows this pattern, which motivates an interface design that +is notably different from that of {{VideoDecoder}} and {{AudioDecoder}}. + +In spite of these differences, {{ImageDecoder}} uses the same +[=codec processing model=] as the other codec interfaces. Additionally, +{{ImageDecoder}} uses the {{VideoFrame}} interface to describe decoded outputs. + +ImageDecoder Interface {#imagedecoder-interface} +------------------------------------------------ + +
+
+[Exposed=(Window,DedicatedWorker)]
+interface ImageDecoder {
+  constructor(ImageDecoderInit init);
+
+  readonly attribute boolean complete;
+  readonly attribute ImageTrackList tracks;
+
+  Promise<undefined> decodeMetadata();
+  Promise<ImageDecodeResult> decode(optional unsigned long frameIndex = 0,
+                                    optional boolean completeFramesOnly = true);
+
+  static Promise<boolean> isTypeSupported(DOMString type);
+};
+
+
+ +### Internal Slots ### {#imagedecoder-internal-slots} + +: \[[ImageTrackList]] +:: An {{ImageTrackList}} describing the tracks found in + {{ImageDecoder/[[encoded data]]}} + +: \[[complete]] +:: A boolean indicating whether {{ImageDecoder/[[encoded data]]}} is completely + buffered. + +: [[codec implementation]] +:: An underlying image decoder implementation provided by the User Agent. + +: [[encoded data]] +:: A [=byte sequence=] containing the encoded image data to be decoded. + +: [[prefer animation]] +:: Boolean reflecting the value of {{ImageDecoderInit/preferAnimation}} given + at construction. + +: [[pending metadata promises]] +:: A list of unresolved promises returned by calls to decodeMetadata(). + +: [[pending decode promises]] +:: A list of unresolved promises returned by calls to decode(). + +: [[internal selected track index]] +:: Identifies the image track within {{ImageDecoder/[[encoded data]]}} that is + used by decoding algorithms on the [=codec thread=]. + +: [[tracks established]] +:: A boolean indicating whether the track list has been established in + {{ImageDecoder/[[ImageTrackList]]}}. + +: [[fatal error]] +:: A boolean indicating that the ImageDecoder is in a permanent error state. + +: [[progressive frame generations]] +:: A mapping of frame indices to [=Progressive Image Frame Generations=]. The + values represent the Progressive Image Frame Generation for the + {{VideoFrame}} which was most recently output by a call to + {{ImageDecoder/decode()}} with the given frame index. + + +### Constructor ### {#imagedecoder-constructor} + +: + ImageDecoder(init) + +:: NOTE: Calling {{ImageDecoder/decode()}} on the constructed {{ImageDecoder}} + will trigger a {{NotSupportedError}} if the user agent does not support + |type|. Authors should first check support by calling + {{ImageDecoder/isTypeSupported()}} with |type|. User agents are not + required to support any particular type. + + When invoked, run these steps: + 1. If |init| is not [=valid ImageDecoderInit=], throw a {{TypeError}}. + 2. Let |d| be a new {{ImageDecoder}} object. In the steps below, all + mentions of {{ImageDecoder}} members apply to |d| unless stated + otherwise. + 3. Assign {{ImageDecoder/[[ImageTrackList]]}} a new {{ImageTrackList}} + initialized as follows: + 1. Assign a new [=list=] to {{ImageTrackList/[[track list]]}}. + 2. Assign `-1` to {{ImageTrackList/[[selected index]]}}. + 4. Assign `null` to {{ImageDecoder/[[codec implementation]]}}. + 5. If `init.preferAnimation` [=map/exists=], assign `init.preferAnimation` + to the {{ImageDecoder/[[prefer animation]]}} internal slot. Otherwise, + assign 'null' to {{ImageDecoder/[[prefer animation]]}} internal slot. + 6. Assign a new [=list=] to {{ImageDecoder/[[pending metadata promises]]}}. + 7. Assign a new [=list=] to {{ImageDecoder/[[pending decode promises]]}}. + 8. Assign `false` to {{ImageDecoder/[[tracks established]]}}. + 9. Assign `false` to {{ImageDecoder/[[fatal error]]}}. + 10. Assign a new [=map=] to {{ImageDecoder/[[progressive frame + generations]]}}. + 11. If |init|'s {{ImageDecoderInit/data}} member is of type + {{ReadableStream}}: + 1. Assign a new [=list=] to {{ImageDecoder/[[encoded data]]}}. + 2. Assign `false` to {{ImageDecoder/complete}} + 3. [=Queue a control message=] to [=configure the image decoder=] with + |init|. + 4. Let |reader| be the result of [=getting a reader=] for + {{ImageDecoderInit/data}}. + 5. In parallel, perform the [=Fetch Stream Data Loop=] on |d| with + |reader|. + 12. Otherwise: + 1. Assert that `init.data` is of type {{BufferSource}}. + 2. Assign a copy of `init.data` to {{ImageDecoder/[[encoded data]]}}. + 3. Assign `true` to {{ImageDecoder/complete}} + 4. Queue a control message to [=configure the image decoder=] with + |init|. + 5. Queue a control message to [=decode track metadata=]. + 13. return |d|. + + [=Running a control message=] to configure the image decoder + means running these steps: + 1. Let |supported| be the result of running the [=Check Configuration + Support=] algorithm with `init.type`. + 2. If |supported| is `false`, queue a task on the control thread event + loop to run the [=ImageDecoder/Handle Error=] algorithm and abort + these steps. + 3. If |supported| is `true`, assign the + {{ImageDecoder/[[codec implementation]]}} internal slot with an + implementation supporting `init.type` + 4. Configure {{ImageDecoder/[[codec implementation]]}} in accordance with + the values given for {{ImageDecoderInit/premultiplyAlpha}}, + {{ImageDecoderInit/colorSpaceConversion}}, + {{ImageDecoderInit/desiredWidth}}, and + {{ImageDecoderInit/desiredHeight}}. + + [=Running a control message=] to decode track metadata means + running these steps: + 1. Run the [=Decode Track Metadata=] algorithm. + +### Attributes ### {#imagedecoder-attributes} + +: tracks +:: Provides metadata for the available tracks and a mechanism for selecting a + track to decode. + + The {{ImageDecoder/tracks}} getter steps are to return + {{ImageDecoder/[[ImageTrackList]]}}. + +: complete +:: Indicates whether {{ImageDecoder/[[encoded data]]}} is completely buffered. + + The {{ImageDecoder/complete}} getter steps are to return + {{ImageDecoder/[[complete]]}}. + + +### Methods ### {#imagedecoder-methods} +: decodeMetadata() +:: Returns a promise which resolves once enough + {{ImageDecoder/[[encoded data]]}} has been buffered and decoded to + establish {{ImageDecoder/tracks}}. + + NOTE: {{ImageTrack}} {{ImageTrack/frameCount}} may receive subsequent + updates until {{ImageDecoder/complete}} is `true`. + + When invoked, run these steps: + 1. If {{ImageDecoder/[[fatal error]]}} is `true`, return a {{Promise}} + rejected with an {{EncodingError}} {{DOMException}}. + 2. If {{ImageDecoder/[[tracks established]]}} is `true`, return a resolved + {{Promise}}. + 3. Let |promise| be a new {{Promise}}. + 4. Append |promise| to {{ImageDecoder/[[pending metadata promises]]}} + 5. Return |promise|. + +: decode(frameIndex, completeFramesOnly) +:: Enqueues a control message to decode the frame at frameIndex, resolving the + promise to output the decoded frame. When |completeFramesOnly| is `false`, + the returned promise may contain a [=Progressive Image=] output with + reduced detail, as indicated by {{ImageDecodeResult/complete}}. + Subsequent calls to {{ImageDecoder/decode()}} with the same |frameIndex| + will only resolve when an update to the progressively decoded image is + available or when it is completely decoded. + + When invoked, run these steps: + 1. If {{ImageDecoder/[[fatal error]]}} is `true`, return a {{Promise}} + rejected with an {{InvalidStateError}} {{DOMException}}. + 2. If {{ImageDecoder/[[ImageTrackList]]}}'s + {{ImageTrackList/[[selected index]]}} is '-1', return a {{Promise}} + rejected with an {{InvalidStateError}} {{DOMException}}. + 3. Let |promise| be a new {{Promise}}. + 4. [=Queue a control message=] to decode the the image with |frameIndex|, + |completeFramesOnly|, and |promise|. + 5. Append |promise| to {{ImageDecoder/[[pending decode promises]]}}. + 6. Return |promise|. + + [=Running a control message=] to decode the image means running these + steps: + 1. Wait for {{ImageDecoder/[[tracks established]]}} to become `true`. + 2. If |completeFramesOnly| is `true`, run the [=Decode Complete Frame=] + algorithm with |frameIndex| and |promise|. + 3. Otherwise, run the [=Decode Progressive Frame=] algorithm with + |frameIndex| and |promise|. + +: isTypeSupported(type) +:: Returns a promise indicating whether the provided config is supported by the + user agent. + + When invoked, run these steps: + 1. If |type| is not a [=valid image MIME type=], return a {{Promise}} + rejected with {{TypeError}}. + 2. Let |p| be a new {{Promise}}. + 3. In parallel, resolve |p| with the result of running the + [=Check Type Support=] algorithm with |type|. + 4. Return |p|. + +### Algorithms ### {#imagedecoder-algorithms} + +: Fetch Stream Data Loop (with |reader|) +:: Run these steps: + 1. Let |readRequest| be the following [=read request=]. + + : [=read request/chunk steps=], given |chunk| + :: 1. If {{ImageDecoder/[[fatal error]]}} is `true`, abort these steps. + 2. If |chunk| is not a Uint8Array object, run the [=Handle Error=] + algorithm with a {{NotReadableError}} {{DOMException}} and + abort these steps. + 3. Let |bytes| be the byte sequence represented by the Uint8Array + object. + 4. Append |bytes| to the {{ImageDecoder/[[encoded data]]}} + internal slot. + 5. If {{ImageDecoder/[[tracks established]]}} is `false`, run the + [=Establish Tracks=] algorithm. + 6. Otherwise, run the [=Update Tracks=] algorithm. + 7. Run the [=Fetch Stream Data Loop=] algorithm with |reader|. + + : [=read request/close steps=] + :: 1. Assign `true` to {{ImageDecoder/complete}} + + : [=read request/error steps=] + :: 1. Queue a task on the [=control thread=] event loop to run the + [=Handle Error=] algorithm. + + 2. Read a chunk from |reader| given |readRequest|. + +: Establish Tracks +:: Run these steps: + 1. Assert {{ImageDecoder/[[tracks established]]}} is `false`. + 2. If {{ImageDecoder/[[encoded data]]}} does not contain enough data to + determine the number of tracks, abort these steps. + 3. If the number of tracks is found to be '0', run the + [=ImageDecoder/Handle Error=] algorithm and abort these steps. + 4. Let |selectedTrackIndex| be `-1`. + 5. Let |newTrackList| be a new [=list=]. + 6. For each |image track| found in {{ImageDecoder/[[encoded data]]}}: + 1. Let |newTrack| be a new {{ImageTrack}}, initialized as follows: + 1. Assign [=this=] to {{ImageTrack/[[ImageDecoder]]}}. + 2. Assign {{ImageDecoder/tracks}} to + {{ImageTrack/[[ImageTrackList]]}}. + 3. If |image track| is found to be animated, assign `true` to + |newTrack|'s {{ImageTrack/[[animated]]}} internal slot. + Otherwise, assign `false`. + 4. If |image track| is found to describe a frame count, assign + that count to |newTrack|'s {{ImageTrack/[[frame count]]}} + internal slot. Otherwise, assign `0`. + + NOTE: If [=this=] was constructed with + {{ImageDecoderInit/data}} as a {{ReadableStream}}, the + {{ImageTrack/frameCount}} may change as additional bytes are + appended to {{ImageDecoder/[[encoded data]]}}. See the + [=Update Tracks=] algorithm. + + 5. If |image track| is found to describe a repetition count, + assign that count to {{ImageTrack/[[repetition count]]}} + internal slot. Otherwise, assign `0`. + + NOTE: A value of `Infinity` indicates infinite repetitions. + + 2. Append |newTrack| to |newTrackList|. + 3. Let |trackIndex| be the index of |newTrack| in |newTrackList|. + 4. If |selectedTrackIndex| is `-1`: + 1. If {{ImageDecoder/[[prefer animation]]}} is equal to + |newTrack|'s {{ImageTrack/[[animated]]}}, assign |trackIndex| + to |selectedTrackIndex|. + 2. Otherwise, if the {{ImageDecoder/[[encoded data]]}} indicates + that |newTrack| is the [=Primary Image Track=], assign + |trackIndex| to |selectedTrackIndex|. + 3. Otherwise, if the {{ImageDecoder/[[encoded data]]}} does not + indicate that any track is primary, assign |trackIndex| to + |selectedTrackIndex|. + 5. If |trackIndex| equals |selectedTrackIndex|, assign `true` to + |newTrack|'s {{ImageTrack/[[selected]]}} internal slot. + 7. Assert that |selectedTrackIndex| is not `-1`. + 8. Assign |selectedTrackIndex| to {{ImageDecoder/[[internal selected track + index]]}}. + 9. Assign `true` to {{ImageDecoder/[[tracks established]]}}. + 10. Queue a task on the [=control thread=] event loop to perform the + following steps: + 1. Assign |newTrackList| to the {{ImageDecoder/tracks}} + {{ImageTrackList/[[track list]]}} internal slot . + 2. Assign |selectedTrackIndex| to {{ImageDecoder/tracks}} + {{ImageTrackList/[[selected index]]}}. + 3. Fire a simple event named {{ImageTrackList/change}} at the + {{ImageDecoder/tracks}} object. + 4. For each |promise| in + {{ImageDecoder/[[pending metadata promises]]}}, resolve |promise| + and remove it from the list. + +: Update Tracks +:: A track update struct is a [=struct=] that consists of a + track index ({{unsigned long}}) + and a frame count + ({{unsigned long}}). + + Run these steps: + 1. Assert {{ImageDecoder/[[tracks established]]}} is `true`. + 2. Let |trackChanges| be a new [=list=]. + 3. Let |trackList| be a copy of {{ImageDecoder/tracks}}' + {{ImageTrackList/[[track list]]}}. + 4. For each |track| in |trackList|: + 1. Let |trackIndex| be the position of |track| in |trackList|. + 2. Let |latestFrameCount| be the frame count as indicated by + {{ImageDecoder/[[encoded data]]}} for the track corresponding to + |track|. + 3. Assert that |latestFrameCount| is greater than or equal to + `track.frameCount`. + 4. If |latestFrameCount| is greater than `track.frameCount`: + 1. Let |change| be a [=track update struct=] whose + [=track update struct/track index=] is |trackIndex| and + [=track update struct/frame count=] is |latestFrameCount|. + 2. Append |change| to |tracksChanges|. + 5. If |tracksChanges| is [=list/empty=], abort these steps. + 6. For each update in |trackChanges|: + 1. Let |updateTrack| be the {{ImageTrack}} at position + `update.trackIndex` within {{ImageDecoder/tracks}}' + {{ImageTrackList/[[track list]]}}. + 2. Assign `update.frameCount` to |updateTrack|'s + {{ImageTrack/[[frame count]]}}. + 7. Fire a simple event named {{ImageTrackList/change}} at the + {{ImageDecoder/tracks}} object. + +: Decode Complete Frame (with |frameIndex| and + |promise|) +:: 1. Assert that {{ImageDecoder/[[tracks established]]}} is `true`. + 2. Assert that {{ImageDecoder/[[internal selected track index]]}} is not + `-1`. + 3. Let |encodedFrame| be the encoded frame identified by |frameIndex| and + {{ImageDecoder/[[internal selected track index]]}}. + 4. Wait for any of the following conditions to be true (whichever happens + first): + 1. {{ImageDecoder/[[encoded data]]}} contains enough bytes to + completely decode |encodedFrame|. + 2. {{ImageDecoder/complete}} is `true`. + 3. {{ImageDecoder/[[fatal error]]}} is `true`. + 5. If {{ImageDecoder/[[encoded data]]}} does not contain enough bytes to + completely decode |encodedFrame|: + 1. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. + 2. Reject |promise| with {{InvalidStateError}} {{DOMException}}. + 3. Abort these steps. + 6. Attempt to use {{ImageDecoder/[[codec implementation]]}} to decode + |encodedFrame|. + 7. If decoding produces an error: + 1. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. + 2. Reject |promise| with {{EncodingError}} {{DOMException}}. + 3. Abort these steps. + 8. If {{ImageDecoder/[[progressive frame generations]]}} contains an entry + keyed by |frameIndex|, remove the entry from the map. + 9. Let |output| be the decoded image data emitted by + {{ImageDecoder/[[codec implementation]]}} corresponding to + |encodedFrame|. + 10. Let |decodeResult| be a new {{ImageDecodeResult}} initialized as + follows: + 1. Assign 'true' to {{ImageDecodeResult/complete}}. + 2. Assign {{ImageDecodeResult/image}} with the result of running the + [=Create a VideoFrame=] algorithm with |output|. + 13. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. + 14. Resolve |promise| with |decodeResult|. + +: Decode Progressive Frame (with |frameIndex| and + |promise|) +:: 1. Assert that {{ImageDecoder/[[tracks established]]}} is `true`. + 2. Assert that {{ImageDecoder/[[internal selected track index]]}} is not + `-1`. + 3. Let |encodedFrame| be the encoded frame identified by |frameIndex| and + {{ImageDecoder/[[internal selected track index]]}}. + 4. Let |lastFrameGeneration| be `null`. + 5. If {{ImageDecoder/[[progressive frame generations]]}} contains a map + entry with the key |frameIndex|, assign the value of the map entry to + |lastFrameGeneration|. + 6. Wait for any of the following conditions to be true (whichever happens + first): + 1. {{ImageDecoder/[[encoded data]]}} contains enough bytes to decode + |encodedFrame| to produce an output who's [=Progressive Image + Frame Generation=] does not equal |lastFrameGeneration|. + 2. {{ImageDecoder/complete}} is `true`. + 3. {{ImageDecoder/[[fatal error]]}} is `true`. + 7. If {{ImageDecoder/[[encoded data]]}} does not contain enough bytes to + progressively decode |encodedFrame| to produce an output who's + [=Progressive Image Frame Generation=] does not equal + |lastFrameGeneration|: + 1. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. + 2. Reject |promise| with {{InvalidStateError}} {{DOMException}}. + 3. Abort these steps. + 8. Attempt to use {{ImageDecoder/[[codec implementation]]}} to decode + |encodedFrame|. + 9. If decoding produces an error: + 1. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. + 2. Reject |promise| with {{EncodingError}} {{DOMException}}. + 3. Abort these steps. + 10. Let |output| be the decoded image data emitted by + {{ImageDecoder/[[codec implementation]]}} corresponding to + |encodedFrame|. + 11. Let |decodeResult| be a new {{ImageDecodeResult}}. + 12. If |output| is the final full-detail progressive output corresponding + to |encodedFrame|: + 1. Assign `true` to |decodeResult|'s {{ImageDecodeResult/complete}}. + 2. If {{ImageDecoder/[[progressive frame generations]]}} contains an + entry keyed by |frameIndex|, remove the entry from the map. + 13. Otherwise: + 1. Assign `false` to |decodeResult|'s {{ImageDecodeResult/complete}}. + 2. Let |frameGeneration| be the [=Progressive Image Frame Generation=] + for |output|. + 3. Add a new entry to {{ImageDecoder/[[progressive frame + generations]]}} with key |frameIndex| and value |frameGeneration|. + 14. Assign |decodeResult|.{{ImageDecodeResult/image}} with the result of + running the [=Create a VideoFrame=] algorithm with |output|. + 15. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. + 16. Resolve |promise| with |decodeResult|. + +: Check Type Support (with |type|) +:: 1. If the user agent can provide a codec to support decoding |type|, return + `true`. + 2. Otherwise, return `false`. + +: Handle Error (with |exception|) +:: 1. Assign `true` to {{ImageDecoder/[[fatal error]]}}. + 2. For each |metadataPromise| in + {{ImageDecoder/[[pending metadata promises]]}}: + 1. Reject |metadataPromise| with |exception|. + 2. Remove |metadataPromise| from + {{ImageDecoder/[[pending metadata promises]]}}. + 3. For each |decodePromise| in + {{ImageDecoder/[[pending decode promises]]}}: + 1. Reject |decodePromise| with |exception|. + 2. Remove |decodePromise| from + {{ImageDecoder/[[pending decode promises]]}}. + +ImageDecoderInit Interface {#imagedecoderinit-interface} +-------------------------------------------------------- +
+
+typedef (BufferSource or ReadableStream) ImageBufferSource;
+dictionary ImageDecoderInit {
+  required DOMString type;
+  required ImageBufferSource data;
+  PremultiplyAlpha premultiplyAlpha = "default";
+  ColorSpaceConversion colorSpaceConversion = "default";
+  unsigned long desiredWidth;
+  unsigned long desiredHeight;
+  boolean preferAnimation;
+};
+
+
+ +To determine if an {{ImageDecoderInit}} is a valid ImageDecoderInit, +run these steps: +1. If |type| is not a [=valid image MIME type=], return `false`. +2. If |data| is of type {{ReadableStream}} and the ReadableStream is + [=ReadableStream/disturbed=] or [=ReadableStream/locked=], return `false`. +3. If |data| is of type {{BufferSource}}: + 1. If the result of running IsDetachedBuffer (described in + [[!ECMASCRIPT]]) on |data| is `false`, return `false`. + 2. If |data| is [=empty=], return `false`. +4. If {{ImageDecoderInit/desiredWidth}} [=map/exists=] and + {{ImageDecoderInit/desiredHeight}} does not exist, return `false`. +5. If {{ImageDecoderInit/desiredHeight}} [=map/exists=] and + {{ImageDecoderInit/desiredWidth}} does not exist, return `false`. +6. Return `true`. + +A valid image MIME type is a string that is a [=valid MIME type +string=] and for which the `type`, per Section 3.1.1.1 of [[RFC7231]], is +`image`. + +: type +:: String containing the MIME type of the image file to be decoded. + +: data +:: {{BufferSource}} or {{ReadableStream}} of bytes representing an encoded + image file as described by {{ImageDecoderInit/type}}. + +: premultiplyAlpha +:: Controls whether decoded outputs' color channels are to be premultiplied by + their alpha channel, as defined by {{ImageBitmapOptions/premultiplyAlpha}} + in {{ImageBitmapOptions}}. + +: colorSpaceConversion +:: Controls whether decoded outputs' color space is converted or ignored, as + defined by {{ImageBitmapOptions/colorSpaceConversion}} in + {{ImageBitmapOptions}}. + +: desiredWidth +:: Indicates a desired width for decoded outputs. Implementation is best + effort; decoding to a desired width may not be supported by all formats/ + decoders. + +: desiredHeight +:: Indicates a desired height for decoded outputs. Implementation is best + effort; decoding to a desired height may not be supported by all + formats/decoders. + +ImageDecodeResult Interface {#imagedecoderesult-interface} +---------------------------------------------------------- +
+
+dictionary ImageDecodeResult {
+  required VideoFrame image;
+  required boolean complete;
+};
+
+
+ +: image +:: The decoded image. + +: complete +:: Indicates whether {{ImageDecodeResult/image}} contains the final full-detail + output. + + NOTE: {{ImageDecodeResult/complete}} is always `true` when + {{ImageDecoder/decode()}} is invoked with |completeFramesOnly| set to + `true`. + +ImageTrackList Interface {#imagetracklist-interface} +---------------------------------------------------- +
+
+interface ImageTrackList : EventTarget {
+  getter ImageTrack (unsigned long index);
+
+  readonly attribute unsigned long length;
+  readonly attribute long selectedIndex;
+
+  attribute EventHandler onchange;
+};
+
+
+ +### Internal Slots ### {#imagetracklist-internal-slots} + +: [[track list]] +:: The list of {{ImageTrack}}s describe by this {{ImageTrackList}}. + +: \[[selected index]] +:: The index of the selected track in {{ImageTrackList/[[track list]]}}. A + value of `-1` indeicates that no track is selected. + +### Attributes ### {#imagetracklist-attributes} + +: length +:: The {{ImageTrackList/length}} getter steps are to return the length of + {{ImageTrackList/[[track list]]}}. + +: selectedIndex +:: The {{ImageTrackList/selectedIndex}} getter steps are to return + {{ImageTrackList/[[selected index]]}}; + +: onchange +:: An [=event handler IDL attribute=] whose [=event handler event type=] is + {{ImageTrackList/change}}. + +### Event Summary ### {#imagetracklist-eventsummary} + +: change +:: Fired at the {{ImageTrackList}} when the list of {{ImageTrack}}s is + established or when attributes of the listed {{ImageTrack}}s are updated. + +ImageTrack Interface {#imagetrack-interface} +-------------------------------------------- +
+
+interface ImageTrack {
+  readonly attribute boolean animated;
+  readonly attribute long frameCount;
+  readonly attribute unrestricted float repetitionCount;
+  attribute boolean selected;
+};
+
+
+ +### Internal Slots ### {#imagetrack-internal-slots} +: \[[ImageDecoder]] +:: The {{ImageDecoder}} instance that constructed this {{ImageTrack}}. + +: \[[ImageTrackList]] +:: The {{ImageTrackList}} instance that lists this {{ImageTrack}}. + +: \[[animated]] +:: Indicates whether this track contains an animated image with multiple + frames. + +: [[frame count]] +:: The number of frames in this track. + +: [[repetition count]] +:: The number of times the animation is intended to repeat. + +: \[[selected]] +:: Indicates whether this track is selected for decoding. + +### Attributes ### {#imagetrack-attributes} + +: animated +:: The {{ImageTrack/animated}} getter steps are to return the value of + {{ImageTrack/[[animated]]}}. + + NOTE: This attribute provides an early indication that + {{ImageTrack/frameCount}} will ultimately exceed 0 for images where the + {{ImageTrack/frameCount}} starts at '0' and later increments as new + chunks of the {{ReadableStream}} {{ImageDecoderInit/data}} arrive. + +: frameCount +:: The {{ImageTrack/frameCount}} getter steps are to return the value of + {{ImageTrack/[[frame count]]}}. + +: repetitionCount +:: The {{ImageTrack/repetitionCount}} getter steps are to return the value of + {{ImageTrack/[[repetition count]]}}. + +: selected +:: The {{ImageTrack/selected}} getter steps are to return the value of + {{ImageTrack/[[selected]]}}. + + The {{ImageTrack/selected}} setter steps are: + 1. Let |newValue| be [=the given value=]. + 2. If |newValue| equals {{ImageTrack/[[selected]]}}, abort these steps. + 3. Assign |newValue| to {{ImageTrack/[[selected]]}}. + 4. Let |parentTrackList| be {{ImageTrack/[[ImageTrackList]]}} + 5. Let |oldSelectedIndex| be the value of |parentTrackList| + {{ImageTrackList/[[selected index]]}}. + 6. If |oldSelectedIndex| is not `-1`: + 1. Let |oldSelectedTrack| be the {{ImageTrack}} in |parentTrackList| + {{ImageTrackList/[[track list]]}} at the position of + |oldSelectedIndex|. + 2. Assign `false` to |oldSelectedTrack| {{ImageTrack/[[selected]]}} + + 7. If |newValue| is `true`, let |selectedIndex| be the index of [=this=] + {{ImageTrack}} within |parentTrackList|'s + {{ImageTrackList/[[track list]]}}. Otherwise, let |selectedIndex| be + `-1`. + 8. Assign |selectedIndex| to |parentTrackList| + {{ImageTrackList/[[selected index]]}}. + 9. Let |parentDecoder| be {{ImageTrack/[[ImageDecoder]]}}. + 10. [=Reset the control message queue=] associated with |parentDecoder|. + 11. For each |promise| in |parentDecoder|'s + {{ImageDecoder/[[pending decode promises]]}} internal slot: + 1. Reject |promise| with an {{AbortError}} {{DOMException}}. + 2. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. + 12. [=Queue a control message=] to {{ImageTrack/[[ImageDecoder]]}}'s + [=control message queue=] to update the internal selected track + index with |selectedIndex|. + + [=Running a control message=] to update the internal selected track index + means running these steps: + 1. Set {{ImageDecoder/[[internal selected track index]]}} to + |selectedIndex|. + 2. Remove all entries from + {{ImageDecoder/[[progressive frame generations]]}}. Security Considerations{#security-considerations} From 2921c86b5642a5099294f6571b60088e8baf22e9 Mon Sep 17 00:00:00 2001 From: Chris Cunningham Date: Fri, 19 Mar 2021 10:27:15 -0700 Subject: [PATCH 2/9] Various fixes to address dalecurtis@ feedback --- index.src.html | 335 ++++++++++++++++++++++++++++++++----------------- 1 file changed, 218 insertions(+), 117 deletions(-) diff --git a/index.src.html b/index.src.html index 5e6ce31b..082078ed 100644 --- a/index.src.html +++ b/index.src.html @@ -113,8 +113,8 @@ : Progressive Image Frame Generation :: A generational identifier for a given [=Progressive Image=] decoded output. - The mechanism for computing a frame's generation is implementer / format - defined. + Each successive generation adds additional detail to the decoded output. + The mechanism for computing a frame's generation is implementer defined. : Primary Image Track :: An image track that is marked by the given image file as being the default @@ -2437,7 +2437,7 @@ This section is non-normative. Image codec definitions are typically accompanied by a definition for a -corresponding file format. Hence, image decoders often perform both duties of +corresponding file format. Hence image decoders often perform both duties of unpacking (demuxing) as well as decoding the encoded image data. The WebCodecs {{ImageDecoder}} follows this pattern, which motivates an interface design that is notably different from that of {{VideoDecoder}} and {{AudioDecoder}}. @@ -2459,8 +2459,9 @@ readonly attribute ImageTrackList tracks; Promise decodeMetadata(); - Promise decode(optional unsigned long frameIndex = 0, - optional boolean completeFramesOnly = true); + Promise decode(optional ImageDecodeOptions options); + undefined reset(); + undefined close(); static Promise isTypeSupported(DOMString type); }; @@ -2501,8 +2502,9 @@ :: A boolean indicating whether the track list has been established in {{ImageDecoder/[[ImageTrackList]]}}. -: [[fatal error]] -:: A boolean indicating that the ImageDecoder is in a permanent error state. +: \[[closed]] +:: A boolean indicating that the ImageDecoder is in a permanent closed state + and can no longer be used. : [[progressive frame generations]] :: A mapping of frame indices to [=Progressive Image Frame Generations=]. The @@ -2538,7 +2540,7 @@ 6. Assign a new [=list=] to {{ImageDecoder/[[pending metadata promises]]}}. 7. Assign a new [=list=] to {{ImageDecoder/[[pending decode promises]]}}. 8. Assign `false` to {{ImageDecoder/[[tracks established]]}}. - 9. Assign `false` to {{ImageDecoder/[[fatal error]]}}. + 9. Assign `false` to {{ImageDecoder/[[closed]]}}. 10. Assign a new [=map=] to {{ImageDecoder/[[progressive frame generations]]}}. 11. If |init|'s {{ImageDecoderInit/data}} member is of type @@ -2562,10 +2564,11 @@ [=Running a control message=] to configure the image decoder means running these steps: - 1. Let |supported| be the result of running the [=Check Configuration + 1. Let |supported| be the result of running the [=ImageDecoder/Check Type Support=] algorithm with `init.type`. - 2. If |supported| is `false`, queue a task on the control thread event - loop to run the [=ImageDecoder/Handle Error=] algorithm and abort + 2. If |supported| is `false`, queue a task on the [=control thread=] event + loop to run the [=ImageDecoder/Close ImageDecoder=] algorithm + with a {{NotSupportedError}} {{DOMException}} and abort these steps. 3. If |supported| is `true`, assign the {{ImageDecoder/[[codec implementation]]}} internal slot with an @@ -2578,7 +2581,7 @@ [=Running a control message=] to decode track metadata means running these steps: - 1. Run the [=Decode Track Metadata=] algorithm. + 1. Run the [=ImageDecoder/Establish Tracks=] algorithm. ### Attributes ### {#imagedecoder-attributes} @@ -2606,7 +2609,7 @@ updates until {{ImageDecoder/complete}} is `true`. When invoked, run these steps: - 1. If {{ImageDecoder/[[fatal error]]}} is `true`, return a {{Promise}} + 1. If {{ImageDecoder/[[closed]]}} is `true`, return a {{Promise}} rejected with an {{EncodingError}} {{DOMException}}. 2. If {{ImageDecoder/[[tracks established]]}} is `true`, return a resolved {{Promise}}. @@ -2614,34 +2617,44 @@ 4. Append |promise| to {{ImageDecoder/[[pending metadata promises]]}} 5. Return |promise|. -: decode(frameIndex, completeFramesOnly) -:: Enqueues a control message to decode the frame at frameIndex, resolving the - promise to output the decoded frame. When |completeFramesOnly| is `false`, - the returned promise may contain a [=Progressive Image=] output with - reduced detail, as indicated by {{ImageDecodeResult/complete}}. - Subsequent calls to {{ImageDecoder/decode()}} with the same |frameIndex| - will only resolve when an update to the progressively decoded image is - available or when it is completely decoded. +: decode(options) +:: Enqueues a control message to decode the frame according to |options|. When invoked, run these steps: - 1. If {{ImageDecoder/[[fatal error]]}} is `true`, return a {{Promise}} + 1. If {{ImageDecoder/[[closed]]}} is `true`, return a {{Promise}} rejected with an {{InvalidStateError}} {{DOMException}}. 2. If {{ImageDecoder/[[ImageTrackList]]}}'s {{ImageTrackList/[[selected index]]}} is '-1', return a {{Promise}} rejected with an {{InvalidStateError}} {{DOMException}}. - 3. Let |promise| be a new {{Promise}}. - 4. [=Queue a control message=] to decode the the image with |frameIndex|, - |completeFramesOnly|, and |promise|. - 5. Append |promise| to {{ImageDecoder/[[pending decode promises]]}}. - 6. Return |promise|. + 3. If |options| is `undefined`, assign a new {{ImageDecodeOptions}} to + |options|. + 4. Let |promise| be a new {{Promise}}. + 5. [=Queue a control message=] to decode the the image with |options|, and + |promise|. + 6. Append |promise| to {{ImageDecoder/[[pending decode promises]]}}. + 7. Return |promise|. [=Running a control message=] to decode the image means running these steps: 1. Wait for {{ImageDecoder/[[tracks established]]}} to become `true`. - 2. If |completeFramesOnly| is `true`, run the [=Decode Complete Frame=] - algorithm with |frameIndex| and |promise|. - 3. Otherwise, run the [=Decode Progressive Frame=] algorithm with - |frameIndex| and |promise|. + 2. If |options|.{{ImageDecodeOptions/completeFramesOnly}} is `false` and + the image is a [=Progressive Image=] for which the user agent supports + progressive decoding, run the [=Decode Progressive Frame=] algorithm with |options|.{{ImageDecodeOptions/frameIndex}} and |promise|. + 3. Otherwise, run the [=Decode Complete Frame=] algorithm with + |options|.{{ImageDecodeOptions/frameIndex}} and |promise|. + +: reset() +:: Immediately aborts all pending work. + + When invoked, run the [=ImageDecoder/Reset ImageDecoder=] algorithm with + and {{AbortError}} {{DOMException}}. + +: close() +:: Immediately aborts all pending work and releases system resources. Close is + final. + + When invoked, run the [=ImageDecoder/Close ImageDecoder=] algorithm with + and {{AbortError}} {{DOMException}}. : isTypeSupported(type) :: Returns a promise indicating whether the provided config is supported by the @@ -2662,10 +2675,11 @@ 1. Let |readRequest| be the following [=read request=]. : [=read request/chunk steps=], given |chunk| - :: 1. If {{ImageDecoder/[[fatal error]]}} is `true`, abort these steps. - 2. If |chunk| is not a Uint8Array object, run the [=Handle Error=] - algorithm with a {{NotReadableError}} {{DOMException}} and - abort these steps. + :: 1. If {{ImageDecoder/[[closed]]}} is `true`, abort these steps. + 2. If |chunk| is not a Uint8Array object, queue a task on the + [=control thread=] event loop to run the + [=ImageDecoder/Close ImageDecoder=] algorithm with a + {{DataError}} {{DOMException}} and abort these steps. 3. Let |bytes| be the byte sequence represented by the Uint8Array object. 4. Append |bytes| to the {{ImageDecoder/[[encoded data]]}} @@ -2680,7 +2694,8 @@ : [=read request/error steps=] :: 1. Queue a task on the [=control thread=] event loop to run the - [=Handle Error=] algorithm. + [=ImageDecoder/Close ImageDecoder=] algorithm with a + {{NotReadableError}} {{DOMException}} 2. Read a chunk from |reader| given |readRequest|. @@ -2689,11 +2704,11 @@ 1. Assert {{ImageDecoder/[[tracks established]]}} is `false`. 2. If {{ImageDecoder/[[encoded data]]}} does not contain enough data to determine the number of tracks, abort these steps. - 3. If the number of tracks is found to be '0', run the - [=ImageDecoder/Handle Error=] algorithm and abort these steps. - 4. Let |selectedTrackIndex| be `-1`. - 5. Let |newTrackList| be a new [=list=]. - 6. For each |image track| found in {{ImageDecoder/[[encoded data]]}}: + 3. If the number of tracks is found to be '0', queue a task on the + [=control thread=] event loop to run the + [=ImageDecoder/Close ImageDecoder=] algorithm and abort these steps. + 4. Let |newTrackList| be a new [=list=]. + 5. For each |image track| found in {{ImageDecoder/[[encoded data]]}}: 1. Let |newTrack| be a new {{ImageTrack}}, initialized as follows: 1. Assign [=this=] to {{ImageTrack/[[ImageDecoder]]}}. 2. Assign {{ImageDecoder/tracks}} to @@ -2717,36 +2732,47 @@ NOTE: A value of `Infinity` indicates infinite repetitions. + 6. Assign `false` to |newTrack|'s {{ImageTrack/[[selected]]}} + internal slot. 2. Append |newTrack| to |newTrackList|. - 3. Let |trackIndex| be the index of |newTrack| in |newTrackList|. - 4. If |selectedTrackIndex| is `-1`: - 1. If {{ImageDecoder/[[prefer animation]]}} is equal to - |newTrack|'s {{ImageTrack/[[animated]]}}, assign |trackIndex| - to |selectedTrackIndex|. - 2. Otherwise, if the {{ImageDecoder/[[encoded data]]}} indicates - that |newTrack| is the [=Primary Image Track=], assign - |trackIndex| to |selectedTrackIndex|. - 3. Otherwise, if the {{ImageDecoder/[[encoded data]]}} does not - indicate that any track is primary, assign |trackIndex| to - |selectedTrackIndex|. - 5. If |trackIndex| equals |selectedTrackIndex|, assign `true` to - |newTrack|'s {{ImageTrack/[[selected]]}} internal slot. - 7. Assert that |selectedTrackIndex| is not `-1`. + 6. Let |selectedTrackIndex| be the result of running the + [=ImageDecoder/Get Default Selected Track Index=] algorithm with + |newTrackList|. + 7. Let |selectedTrack| be the track at position |selectedTrackIndex| within + |newTrackList|. + 8. Assign `true` to |selectedTrack|'s {{ImageTrack/[[selected]]}} internal + slot. 8. Assign |selectedTrackIndex| to {{ImageDecoder/[[internal selected track index]]}}. 9. Assign `true` to {{ImageDecoder/[[tracks established]]}}. 10. Queue a task on the [=control thread=] event loop to perform the following steps: 1. Assign |newTrackList| to the {{ImageDecoder/tracks}} - {{ImageTrackList/[[track list]]}} internal slot . + {{ImageTrackList/[[track list]]}} internal slot. 2. Assign |selectedTrackIndex| to {{ImageDecoder/tracks}} {{ImageTrackList/[[selected index]]}}. - 3. Fire a simple event named {{ImageTrackList/change}} at the - {{ImageDecoder/tracks}} object. - 4. For each |promise| in + 3. For each |promise| in {{ImageDecoder/[[pending metadata promises]]}}, resolve |promise| and remove it from the list. +: Get Default Selected Track Index (with + |trackList|) +:: Run these steps: + 1. If {{ImageDecoder/[[encoded data]]}} identifies a [=Primary Image + Track=]: + 1. Let |primaryTrack| be the {{ImageTrack}} from |trackList| that + describes the [=Primary Image Track=]. + 2. Let |primaryTrackIndex| be position of |primaryTrack| within + |trackList|. + 3. If {{ImageDecoder/[[prefer animation]]}} is `null`, return + |primaryTrackIndex|. + 4. If |primaryTrack|.{{ImageTrack/animated}} equals + {{ImageDecoder/[[prefer animation]]}}, return |primaryTrackIndex|. + 2. If any {{ImageTrack}}s in |trackList| have {{ImageTrack/animated}} equal + to {{ImageDecoder/[[prefer animation]]}}, return the position of the + earliest such track in |trackList|. + 3. Return `0`. + : Update Tracks :: A track update struct is a [=struct=] that consists of a track index ({{unsigned long}}) @@ -2771,14 +2797,16 @@ [=track update struct/frame count=] is |latestFrameCount|. 2. Append |change| to |tracksChanges|. 5. If |tracksChanges| is [=list/empty=], abort these steps. - 6. For each update in |trackChanges|: - 1. Let |updateTrack| be the {{ImageTrack}} at position - `update.trackIndex` within {{ImageDecoder/tracks}}' - {{ImageTrackList/[[track list]]}}. - 2. Assign `update.frameCount` to |updateTrack|'s - {{ImageTrack/[[frame count]]}}. - 7. Fire a simple event named {{ImageTrackList/change}} at the - {{ImageDecoder/tracks}} object. + 6. Queue a task on the [=control thread=] event loop to perform the + following steps: + 1. For each update in |trackChanges|: + 1. Let |updateTrack| be the {{ImageTrack}} at position + `update.trackIndex` within {{ImageDecoder/tracks}}' + {{ImageTrackList/[[track list]]}}. + 2. Assign `update.frameCount` to |updateTrack|'s + {{ImageTrack/[[frame count]]}}. + 3. Fire a simple event named {{ImageTrack/change}} at the + {{ImageDecoder/tracks}} object. : Decode Complete Frame (with |frameIndex| and |promise|) @@ -2792,18 +2820,16 @@ 1. {{ImageDecoder/[[encoded data]]}} contains enough bytes to completely decode |encodedFrame|. 2. {{ImageDecoder/complete}} is `true`. - 3. {{ImageDecoder/[[fatal error]]}} is `true`. + 3. {{ImageDecoder/[[closed]]}} is `true`. 5. If {{ImageDecoder/[[encoded data]]}} does not contain enough bytes to - completely decode |encodedFrame|: - 1. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. - 2. Reject |promise| with {{InvalidStateError}} {{DOMException}}. - 3. Abort these steps. + completely decode |encodedFrame|, run the + [=ImageDecoder/Reject Infeasible Decode=] algorithm with |promise| and + abort these steps. 6. Attempt to use {{ImageDecoder/[[codec implementation]]}} to decode |encodedFrame|. - 7. If decoding produces an error: - 1. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. - 2. Reject |promise| with {{EncodingError}} {{DOMException}}. - 3. Abort these steps. + 7. If decoding produces an error, run the + [=ImageDecoder/Reject Failed Decode=] algorithm with |promise| and + abort these steps. 8. If {{ImageDecoder/[[progressive frame generations]]}} contains an entry keyed by |frameIndex|, remove the entry from the map. 9. Let |output| be the decoded image data emitted by @@ -2814,8 +2840,8 @@ 1. Assign 'true' to {{ImageDecodeResult/complete}}. 2. Assign {{ImageDecodeResult/image}} with the result of running the [=Create a VideoFrame=] algorithm with |output|. - 13. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. - 14. Resolve |promise| with |decodeResult|. + 11. Run the [=ImageDecoder/Resolve Decode=] algorithm with |promise| and + |decodeResult|. : Decode Progressive Frame (with |frameIndex| and |promise|) @@ -2832,22 +2858,19 @@ first): 1. {{ImageDecoder/[[encoded data]]}} contains enough bytes to decode |encodedFrame| to produce an output who's [=Progressive Image - Frame Generation=] does not equal |lastFrameGeneration|. + Frame Generation=] exceeds |lastFrameGeneration|. 2. {{ImageDecoder/complete}} is `true`. - 3. {{ImageDecoder/[[fatal error]]}} is `true`. + 3. {{ImageDecoder/[[closed]]}} is `true`. 7. If {{ImageDecoder/[[encoded data]]}} does not contain enough bytes to - progressively decode |encodedFrame| to produce an output who's - [=Progressive Image Frame Generation=] does not equal - |lastFrameGeneration|: - 1. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. - 2. Reject |promise| with {{InvalidStateError}} {{DOMException}}. - 3. Abort these steps. + decode |encodedFrame| to produce an output who's + [=Progressive Image Frame Generation=] exceeds |lastFrameGeneration|, + run the [=ImageDecoder/Reject Infeasible Decode=] algorithm with + |promise| and abort these steps. 8. Attempt to use {{ImageDecoder/[[codec implementation]]}} to decode |encodedFrame|. - 9. If decoding produces an error: - 1. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. - 2. Reject |promise| with {{EncodingError}} {{DOMException}}. - 3. Abort these steps. + 9. If decoding produces an error, run the + [=ImageDecoder/Reject Failed Decode=] algorithm with |promise| and + abort these steps. 10. Let |output| be the decoded image data emitted by {{ImageDecoder/[[codec implementation]]}} corresponding to |encodedFrame|. @@ -2868,13 +2891,43 @@ 15. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. 16. Resolve |promise| with |decodeResult|. +: Resolve Decode (with |promise| and |result|) +:: 1. Queue a task on the [=control thread=] event loop to run these steps: + 1. If {{ImageDecoder/[[closed]]}}, abort these steps. + 2. Assert that |promise| is an element of + {{ImageDecoder/[[pending decode promises]]}}. + 3. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. + 4. Resolve |promise| with |result|. + +: Reject Infeasible Decode (with |promise|) +:: 1. Assert that {{ImageDecoder/complete}} is `true` or + {{ImageDecoder/[[closed]]}} is `true`. + 2. If {{ImageDecoder/complete}} is `true`, let |exception| be a + {{RangeError}}. Otherwise, let |exception| be an + {{InvalidStateError}} {{DOMException}}. + 3. Queue a task on the [=control thread=] event loop to run these steps: + 1. If {{ImageDecoder/[[closed]]}}, abort these steps. + 2. Assert that |promise| is an element of + {{ImageDecoder/[[pending decode promises]]}}. + 3. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. + 4. Reject |promise| with |exception|. + +: Reject Failed Decode (with |promise|) +:: 1. Queue a task on the [=control thread=] event loop to run these steps: + 1. If {{ImageDecoder/[[closed]]}}, abort these steps. + 2. Assert that |promise| is an element of + {{ImageDecoder/[[pending decode promises]]}}. + 3. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}. + 4. Reject |promise| with {{EncodingError}} {{DOMException}}. + : Check Type Support (with |type|) :: 1. If the user agent can provide a codec to support decoding |type|, return `true`. 2. Otherwise, return `false`. -: Handle Error (with |exception|) -:: 1. Assign `true` to {{ImageDecoder/[[fatal error]]}}. +: Reset ImageDecoder (with |exception|) +:: 1. Signal {{ImageDecoder/[[codec implementation]]}} to abort any active + decoding operation. 2. For each |metadataPromise| in {{ImageDecoder/[[pending metadata promises]]}}: 1. Reject |metadataPromise| with |exception|. @@ -2886,6 +2939,13 @@ 2. Remove |decodePromise| from {{ImageDecoder/[[pending decode promises]]}}. +: Close ImageDecoder (with |exception|) +:: 1. Run the [=ImageDecoder/Reset ImageDecoder=] algorithm with |exception|. + 1. Assign `true` to {{ImageDecoder/[[closed]]}}. + 2. Clear {{ImageDecoder/[[codec implementation]]}} and release associated + [=system resources=]. + + ImageDecoderInit Interface {#imagedecoderinit-interface} -------------------------------------------------------- 
@@ -2949,6 +3009,45 @@
     effort; decoding to a desired height may not be supported by all
     formats/decoders.
 
+ImageDecodeOptions Interface {#imagedecodeoptions-interface}
+------------------------------------------------------------
+
+
+dictionary ImageDecodeOptions {
+  unsigned long frameIndex = 0;
+  boolean completeFramesOnly = true;
+};
+
+

+
+: frameIndex
+:: The index of the frame to decode.
+
+: completeFramesOnly
+:: For [=Progressive Images=], a value of `false` indicates that the decoder
+    may output an {{ImageDecodeResult/image}} with reduced detail. Each
+    subsequent call to {{ImageDecoder/decode()}} for the same
+    {{ImageDecodeOptions/frameIndex}} will resolve to produce an image with a
+    higher [=Progressive Image Frame Generation=] (more image detail) than the
+    previous call, until finally the full-detail image is produced.
+
+    If {{ImageDecodeOptions/completeFramesOnly}} is assigned `true`, or if the
+    image is not a [=Progressive Image=], or if the user agent does not support
+    progressive decoding for the given image type, calls to
+    {{ImageDecoder/decode()}} will only resolve once the full detail image is
+    decoded.
+
+    

+      NOTE: For [=Progressive Images=], setting
+          {{ImageDecodeOptions/completeFramesOnly}} to `false` may be used to
+          offer users a preview an image that is still being buffered from the
+          network (via the {{ImageDecoderInit/data}} {{ReadableStream}}).
+
+          Upon decoding the full detail image, the {{ImageDecodeResult}}'s
+          {{ImageDecodeResult/complete}} will be set to true.
+    

+
+
 ImageDecodeResult Interface {#imagedecoderesult-interface}
 ----------------------------------------------------------
 
@@ -2968,20 +3067,19 @@
     output.
 
     NOTE: {{ImageDecodeResult/complete}} is always `true` when
-        {{ImageDecoder/decode()}} is invoked with |completeFramesOnly| set to
-        `true`.
+        {{ImageDecoder/decode()}} is invoked with
+        {{ImageDecodeOptions/completeFramesOnly}} set to `true`.
 
 ImageTrackList Interface {#imagetracklist-interface}
 ----------------------------------------------------
 
 
-interface ImageTrackList : EventTarget {
+interface ImageTrackList {
   getter ImageTrack (unsigned long index);
 
   readonly attribute unsigned long length;
   readonly attribute long selectedIndex;
-
-  attribute EventHandler onchange;
+  readonly attribute ImageTrack? selected;
 };
 
 

@@ -3005,24 +3103,21 @@
 :: The {{ImageTrackList/selectedIndex}} getter steps are to return
     {{ImageTrackList/[[selected index]]}};
 
-: onchange
-:: An [=event handler IDL attribute=] whose [=event handler event type=] is
-    {{ImageTrackList/change}}.
-
-### Event Summary ### {#imagetracklist-eventsummary}
-
-: change
-:: Fired at the {{ImageTrackList}} when the list of {{ImageTrack}}s is
-    established or when attributes of the listed {{ImageTrack}}s are updated.
+: selected
+:: The {{ImageTrackList/selected}} getter steps are:
+    1. If {{ImageTrackList/[[selected index]]}} is `-1`, return `null`.
+    2. Otherwise, return the ImageTrack from {{ImageTrackList/[[track list]]}}
+        at the position indicated by {{ImageTrackList/[[selected index]]}}.
 
 ImageTrack Interface {#imagetrack-interface}
 --------------------------------------------
 
 
-interface ImageTrack {
+interface ImageTrack : EventTarget {
   readonly attribute boolean animated;
-  readonly attribute long frameCount;
+  readonly attribute unsigned long frameCount;
   readonly attribute unrestricted float repetitionCount;
+  attribute EventHandler onchange;
   attribute boolean selected;
 };
 
@@ -3067,6 +3162,10 @@
 :: The {{ImageTrack/repetitionCount}} getter steps are to return the value of
     {{ImageTrack/[[repetition count]]}}.
 
+: onchange
+:: An [=event handler IDL attribute=] whose [=event handler event type=] is
+    {{ImageTrack/change}}.
+
 : selected
 :: The {{ImageTrack/selected}} getter steps are to return the value of
     {{ImageTrack/[[selected]]}}.
@@ -3090,24 +3189,26 @@
         `-1`.
     8. Assign |selectedIndex| to |parentTrackList|
         {{ImageTrackList/[[selected index]]}}.
-    9. Let |parentDecoder| be  {{ImageTrack/[[ImageDecoder]]}}.
-    10. [=Reset the control message queue=] associated with |parentDecoder|.
-    11. For each |promise| in |parentDecoder|'s
-        {{ImageDecoder/[[pending decode promises]]}} internal slot:
-        1. Reject |promise| with an {{AbortError}} {{DOMException}}.
-        2. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
-    12. [=Queue a control message=] to {{ImageTrack/[[ImageDecoder]]}}'s
+    9. Run the [=ImageDecoder/Reset ImageDecoder=] algorithm on
+        {{ImageTrack/[[ImageDecoder]]}}.
+    10. [=Queue a control message=] to {{ImageTrack/[[ImageDecoder]]}}'s
         [=control message queue=] to update the internal selected track
         index with |selectedIndex|.
 
     [=Running a control message=] to update the internal selected track index
     means running these steps:
-    1. Set {{ImageDecoder/[[internal selected track index]]}} to
-        |selectedIndex|.
+    1. Assign |selectedIndex| to
+        {{ImageDecoder/[[internal selected track index]]}}.
     2. Remove all entries from
         {{ImageDecoder/[[progressive frame generations]]}}.
 
 
+### Event Summary ### {#imagetracklist-eventsummary}
+
+: change
+:: Fired at the {{ImageTrack}} when the {{ImageTrack/frameCount}} is altered.
+
+
 Security Considerations{#security-considerations}
 =================================================
 

From 12fd2333792ceae9d96ca0d191e3a75bf113cdeb Mon Sep 17 00:00:00 2001
From: Chris Cunningham 
Date: Fri, 19 Mar 2021 14:38:55 -0700
Subject: [PATCH 3/9] Defer decoding track metadata until decodeMetadata() or
 decode()

Also includes other minor fixes.
---
 index.src.html | 51 +++++++++++++++++++++++++++++++-------------------
 1 file changed, 32 insertions(+), 19 deletions(-)

diff --git a/index.src.html b/index.src.html
index 082078ed..441cc451 100644
--- a/index.src.html
+++ b/index.src.html
@@ -2485,7 +2485,7 @@
 :: A [=byte sequence=] containing the encoded image data to be decoded.
 
 : [[prefer animation]]
-:: Boolean reflecting the value of {{ImageDecoderInit/preferAnimation}} given
+:: A boolean reflecting the value of {{ImageDecoderInit/preferAnimation}} given
     at construction.
 
 : [[pending metadata promises]]
@@ -2498,6 +2498,10 @@
 :: Identifies the image track within {{ImageDecoder/[[encoded data]]}} that is
     used by decoding algorithms on the [=codec thread=].
 
+: [[decode metadata requested]]
+:: A boolean indicating whether a [=control message=] to [=ImageDecoder/decode
+    track metadata=] has been enqueued to the [=control message queue=].
+
 : [[tracks established]]
 :: A boolean indicating whether the track list has been established in
     {{ImageDecoder/[[ImageTrackList]]}}.
@@ -2539,11 +2543,13 @@
         assign 'null' to {{ImageDecoder/[[prefer animation]]}} internal slot.
     6. Assign a new [=list=] to {{ImageDecoder/[[pending metadata promises]]}}.
     7. Assign a new [=list=] to {{ImageDecoder/[[pending decode promises]]}}.
-    8. Assign `false` to {{ImageDecoder/[[tracks established]]}}.
-    9. Assign `false` to {{ImageDecoder/[[closed]]}}.
-    10. Assign a new [=map=] to {{ImageDecoder/[[progressive frame
+    8. Assign `-1` to {{ImageDecoder/[[internal selected track index]]}}.
+    9. Assign `false` to {{ImageDecoder/[[decode metadata requested]]}}.
+    10. Assign `false` to {{ImageDecoder/[[tracks established]]}}.
+    11. Assign `false` to {{ImageDecoder/[[closed]]}}.
+    12. Assign a new [=map=] to {{ImageDecoder/[[progressive frame
         generations]]}}.
-    11. If |init|'s {{ImageDecoderInit/data}} member is of type
+    13. If |init|'s {{ImageDecoderInit/data}} member is of type
         {{ReadableStream}}:
         1. Assign a new [=list=] to {{ImageDecoder/[[encoded data]]}}.
         2. Assign `false` to {{ImageDecoder/complete}}
@@ -2553,14 +2559,13 @@
             {{ImageDecoderInit/data}}.
         5. In parallel, perform the [=Fetch Stream Data Loop=] on |d| with
             |reader|.
-    12. Otherwise:
+    14. Otherwise:
         1. Assert that `init.data` is of type {{BufferSource}}.
         2. Assign a copy of `init.data` to  {{ImageDecoder/[[encoded data]]}}.
         3. Assign `true` to {{ImageDecoder/complete}}
-        4. Queue a control message to [=configure the image decoder=] with
-            |init|.
-        5. Queue a control message to [=decode track metadata=].
-    13. return |d|.
+    15. Queue a control message to [=configure the image decoder=] with
+        |init|.
+    16. return |d|.
 
     [=Running a control message=] to configure the image decoder
     means running these steps:
@@ -2579,10 +2584,6 @@
         {{ImageDecoderInit/desiredWidth}}, and
         {{ImageDecoderInit/desiredHeight}}.
 
-    [=Running a control message=] to decode track metadata means
-    running these steps:
-    1. Run the [=ImageDecoder/Establish Tracks=] algorithm.
-
 ### Attributes ### {#imagedecoder-attributes}
 
 : tracks
@@ -2610,13 +2611,21 @@
 
     When invoked, run these steps:
     1. If {{ImageDecoder/[[closed]]}} is `true`, return a {{Promise}}
-        rejected with an {{EncodingError}} {{DOMException}}.
+        rejected with an {{InvalidStateError}} {{DOMException}}.
     2. If {{ImageDecoder/[[tracks established]]}} is `true`, return a resolved
         {{Promise}}.
     3. Let |promise| be a new {{Promise}}.
     4. Append |promise| to {{ImageDecoder/[[pending metadata promises]]}}
+    5. If {{ImageDecoder/[[decode metadata requested]]}} is `false`:
+        1. [=Queue a control message=] to [=ImageDecoder/decode track
+            metadata=].
+        2. Assign `true` to {{ImageDecoder/[[decode metadata requested]]}}.
     5. Return |promise|.
 
+    [=Running a control message=] to decode track
+    metadata means running these steps:
+    1. Run the [=ImageDecoder/Establish Tracks=] algorithm.
+
 : decode(options)
 :: Enqueues a control message to decode the frame according to |options|.
 
@@ -2628,11 +2637,15 @@
         rejected with an {{InvalidStateError}} {{DOMException}}.
     3. If |options| is `undefined`, assign a new {{ImageDecodeOptions}} to
         |options|.
-    4. Let |promise| be a new {{Promise}}.
-    5. [=Queue a control message=] to decode the the image with |options|, and
+    4. If {{ImageDecoder/[[decode metadata requested]]}} is `false`:
+        1. [=Queue a control message=] to [=ImageDecoder/decode track
+            metadata=].
+        2. Assign `true` to {{ImageDecoder/[[decode metadata requested]]}}.
+    5. Let |promise| be a new {{Promise}}.
+    6. [=Queue a control message=] to decode the the image with |options|, and
         |promise|.
-    6. Append |promise| to {{ImageDecoder/[[pending decode promises]]}}.
-    7. Return |promise|.
+    7. Append |promise| to {{ImageDecoder/[[pending decode promises]]}}.
+    8. Return |promise|.
 
     [=Running a control message=] to decode the image means running these
     steps:

From 5f9255678b92fe61f80577e68579fc6a999f0a77 Mon Sep 17 00:00:00 2001
From: Chris Cunningham 
Date: Fri, 19 Mar 2021 14:54:48 -0700
Subject: [PATCH 4/9] Add missing dfn for ImageDecoderInit.preferAnimation

---
 index.src.html | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/index.src.html b/index.src.html
index 441cc451..2283e8dc 100644
--- a/index.src.html
+++ b/index.src.html
@@ -3022,6 +3022,12 @@
     effort; decoding to a desired height may not be supported by all
     formats/decoders.
 
+: preferAnimation
+:: For images with multiple tracks, this indicates whether the
+    initial track selection should prefer an animated track.
+
+    NOTE: See the [=ImageDecoder/Get Default Selected Track Index=] algorithm.
+
 ImageDecodeOptions Interface {#imagedecodeoptions-interface}
 ------------------------------------------------------------
 

From a7890de0802500b3832f3c620a17b0943a4e7d6d Mon Sep 17 00:00:00 2001
From: Chris Cunningham 
Date: Tue, 20 Apr 2021 21:26:10 -0700
Subject: [PATCH 5/9] Address lingering ImageDecoder TODOs, dalecurtis@
 feedback

---
 index.src.html | 99 ++++++++++++++++++++++++++++----------------------
 1 file changed, 55 insertions(+), 44 deletions(-)

diff --git a/index.src.html b/index.src.html
index 450fd464..f0bec735 100644
--- a/index.src.html
+++ b/index.src.html
@@ -2699,28 +2699,32 @@
         first):
         1. {{ImageDecoder/[[encoded data]]}} contains enough bytes to
             completely decode |encodedFrame|.
-        2. {{ImageDecoder/complete}} is `true`.
-        3. {{ImageDecoder/[[closed]]}} is `true`.
-    5. If {{ImageDecoder/[[encoded data]]}} does not contain enough bytes to
+        2. {{ImageDecoder/[[encoded data]]}} is found to be malformed.
+        3. {{ImageDecoder/complete}} is `true`.
+        4. {{ImageDecoder/[[closed]]}} is `true`.
+    5. If {{ImageDecoder/[[encoded data]]}} is found to be malformed, run the
+        [=ImageDecoder/Fatally Reject Bad Data=] algorithm and abort these
+        steps.
+    6. If {{ImageDecoder/[[encoded data]]}} does not contain enough bytes to
         completely decode |encodedFrame|, run the
         [=ImageDecoder/Reject Infeasible Decode=] algorithm with |promise| and
         abort these steps.
-    6. Attempt to use {{ImageDecoder/[[codec implementation]]}} to decode
+    7. Attempt to use {{ImageDecoder/[[codec implementation]]}} to decode
         |encodedFrame|.
-    7. If decoding produces an error, run the
-        [=ImageDecoder/Reject Failed Decode=] algorithm with |promise| and
-        abort these steps.
-    8. If {{ImageDecoder/[[progressive frame generations]]}} contains an entry
+    8. If decoding produces an error, run the
+        [=ImageDecoder/Fatally Reject Bad Data=] algorithm and abort these
+        steps.
+    9. If {{ImageDecoder/[[progressive frame generations]]}} contains an entry
         keyed by |frameIndex|, remove the entry from the map.
-    9. Let |output| be the decoded image data emitted by
+    10. Let |output| be the decoded image data emitted by
         {{ImageDecoder/[[codec implementation]]}} corresponding to
         |encodedFrame|.
-    10. Let |decodeResult| be a new {{ImageDecodeResult}} initialized as
+    11. Let |decodeResult| be a new {{ImageDecodeResult}} initialized as
         follows:
         1. Assign 'true' to {{ImageDecodeResult/complete}}.
         2. Assign {{ImageDecodeResult/image}} with the result of running the
             [=Create a VideoFrame=] algorithm with |output|.
-    11. Run the [=ImageDecoder/Resolve Decode=] algorithm with |promise| and
+    12. Run the [=ImageDecoder/Resolve Decode=] algorithm with |promise| and
         |decodeResult|.
 
 : Decode Progressive Frame (with |frameIndex| and
@@ -2739,37 +2743,41 @@
         1. {{ImageDecoder/[[encoded data]]}} contains enough bytes to decode
             |encodedFrame| to produce an output who's [=Progressive Image
             Frame Generation=] exceeds |lastFrameGeneration|.
-        2. {{ImageDecoder/complete}} is `true`.
-        3. {{ImageDecoder/[[closed]]}} is `true`.
-    7. If {{ImageDecoder/[[encoded data]]}} does not contain enough bytes to
-        decode |encodedFrame| to produce an output who's
+        2. {{ImageDecoder/[[encoded data]]}} is found to be malformed.
+        3. {{ImageDecoder/complete}} is `true`.
+        4. {{ImageDecoder/[[closed]]}} is `true`.
+    7. If {{ImageDecoder/[[encoded data]]}} is found to be malformed, run the
+        [=ImageDecoder/Fatally Reject Bad Data=] algorithm and abort these
+        steps.
+    8. Otherwise, if {{ImageDecoder/[[encoded data]]}} does not contain enough
+        bytes to decode |encodedFrame| to produce an output who's
         [=Progressive Image Frame Generation=] exceeds |lastFrameGeneration|,
         run the [=ImageDecoder/Reject Infeasible Decode=] algorithm with
         |promise| and abort these steps.
-    8. Attempt to use {{ImageDecoder/[[codec implementation]]}} to decode
+    9. Attempt to use {{ImageDecoder/[[codec implementation]]}} to decode
         |encodedFrame|.
-    9. If decoding produces an error, run the
-        [=ImageDecoder/Reject Failed Decode=] algorithm with |promise| and
-        abort these steps.
-    10. Let |output| be the decoded image data emitted by
+    10. If decoding produces an error, run the
+        [=ImageDecoder/Fatally Reject Bad Data=] algorithm and abort these
+        steps.
+    11. Let |output| be the decoded image data emitted by
         {{ImageDecoder/[[codec implementation]]}} corresponding to
         |encodedFrame|.
-    11. Let |decodeResult| be a new {{ImageDecodeResult}}.
-    12. If |output| is the final full-detail progressive output corresponding
+    12. Let |decodeResult| be a new {{ImageDecodeResult}}.
+    13. If |output| is the final full-detail progressive output corresponding
         to |encodedFrame|:
         1. Assign `true` to |decodeResult|'s {{ImageDecodeResult/complete}}.
         2. If {{ImageDecoder/[[progressive frame generations]]}} contains an
             entry keyed by |frameIndex|, remove the entry from the map.
-    13. Otherwise:
+    14. Otherwise:
         1. Assign `false` to |decodeResult|'s {{ImageDecodeResult/complete}}.
         2. Let |frameGeneration| be the [=Progressive Image Frame Generation=]
             for |output|.
         3. Add a new entry to {{ImageDecoder/[[progressive frame
             generations]]}} with key |frameIndex| and value |frameGeneration|.
-    14. Assign |decodeResult|.{{ImageDecodeResult/image}} with the result of
+    15. Assign |decodeResult|.{{ImageDecodeResult/image}} with the result of
         running the [=Create a VideoFrame=] algorithm with |output|.
-    15. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
-    16. Resolve |promise| with |decodeResult|.
+    16. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
+    17. Resolve |promise| with |decodeResult|.
 
 : Resolve Decode (with |promise| and |result|)
 :: 1. Queue a task on the [=control thread=] event loop to run these steps:
@@ -2792,13 +2800,11 @@
         3. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
         4. Reject |promise| with |exception|.
 
-: Reject Failed Decode (with |promise|)
+: Fatally Reject Bad Data
 :: 1. Queue a task on the [=control thread=] event loop to run these steps:
         1. If {{ImageDecoder/[[closed]]}}, abort these steps.
-        2. Assert that |promise| is an element of
-            {{ImageDecoder/[[pending decode promises]]}}.
-        3. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
-        4. Reject |promise| with {{EncodingError}} {{DOMException}}.
+        2. Run the [=ImageDecoder/Close ImageDecoder=] algorithm with an
+            {{EncodingError}} {{DOMException}}.
 
 : Check Type Support (with |type|)
 :: 1. If the user agent can provide a codec to support decoding |type|, return
@@ -2824,6 +2830,9 @@
     1. Assign `true` to {{ImageDecoder/[[closed]]}}.
     2. Clear {{ImageDecoder/[[codec implementation]]}} and release associated
         [=system resources=].
+    3. Remove all entries from {{ImageDecoder/[[ImageTrackList]]}}.
+    4. Assign `-1` to {{ImageDecoder/[[ImageTrackList]]}}'s
+        {{ImageTrackList/[[selected index]]}}.
 
 
 ImageDecoderInit Interface {#imagedecoderinit-interface}
@@ -2965,7 +2974,7 @@
 
   readonly attribute unsigned long length;
   readonly attribute long selectedIndex;
-  readonly attribute ImageTrack? selected;
+  readonly attribute ImageTrack? selectedTrack;
 };
 
 

@@ -2989,8 +2998,8 @@
 :: The {{ImageTrackList/selectedIndex}} getter steps are to return
     {{ImageTrackList/[[selected index]]}};
 
-: selected
-:: The {{ImageTrackList/selected}} getter steps are:
+: selectedTrack
+:: The {{ImageTrackList/selectedTrack}} getter steps are:
     1. If {{ImageTrackList/[[selected index]]}} is `-1`, return `null`.
     2. Otherwise, return the ImageTrack from {{ImageTrackList/[[track list]]}}
         at the position indicated by {{ImageTrackList/[[selected index]]}}.
@@ -3057,27 +3066,29 @@
     {{ImageTrack/[[selected]]}}.
 
     The {{ImageTrack/selected}} setter steps are:
-    1. Let |newValue| be [=the given value=].
-    2. If |newValue| equals {{ImageTrack/[[selected]]}}, abort these steps.
-    3. Assign |newValue| to {{ImageTrack/[[selected]]}}.
-    4. Let |parentTrackList| be {{ImageTrack/[[ImageTrackList]]}}
-    5. Let |oldSelectedIndex| be the value of |parentTrackList|
+    1. If {{ImageTrack/[[ImageDecoder]]}}'s {{ImageDecoder/[[closed]]}} slot is
+        `true`, abort these steps.
+    2. Let |newValue| be [=the given value=].
+    3. If |newValue| equals {{ImageTrack/[[selected]]}}, abort these steps.
+    4. Assign |newValue| to {{ImageTrack/[[selected]]}}.
+    5. Let |parentTrackList| be {{ImageTrack/[[ImageTrackList]]}}
+    6. Let |oldSelectedIndex| be the value of |parentTrackList|
         {{ImageTrackList/[[selected index]]}}.
-    6. If |oldSelectedIndex| is not `-1`:
+    7. If |oldSelectedIndex| is not `-1`:
         1. Let |oldSelectedTrack| be the {{ImageTrack}} in |parentTrackList|
             {{ImageTrackList/[[track list]]}} at the position of
             |oldSelectedIndex|.
         2. Assign `false` to |oldSelectedTrack| {{ImageTrack/[[selected]]}}
 
-    7. If |newValue| is `true`, let |selectedIndex| be the index of [=this=]
+    8. If |newValue| is `true`, let |selectedIndex| be the index of [=this=]
         {{ImageTrack}} within |parentTrackList|'s
         {{ImageTrackList/[[track list]]}}. Otherwise, let |selectedIndex| be
         `-1`.
-    8. Assign |selectedIndex| to |parentTrackList|
+    9. Assign |selectedIndex| to |parentTrackList|
         {{ImageTrackList/[[selected index]]}}.
-    9. Run the [=ImageDecoder/Reset ImageDecoder=] algorithm on
+    10. Run the [=ImageDecoder/Reset ImageDecoder=] algorithm on
         {{ImageTrack/[[ImageDecoder]]}}.
-    10. [=Queue a control message=] to {{ImageTrack/[[ImageDecoder]]}}'s
+    11. [=Queue a control message=] to {{ImageTrack/[[ImageDecoder]]}}'s
         [=control message queue=] to update the internal selected track
         index with |selectedIndex|.
 

From 4e3f17c925008f61408325aeab1ee5d51e4f18fc Mon Sep 17 00:00:00 2001
From: Chris Cunningham 
Date: Fri, 23 Apr 2021 11:22:20 -0700
Subject: [PATCH 6/9] Replace decodeMetadata() w/ tracks.ready promise.

Queue task to establish tracks upon construction. As this is no longer
user driven, we replace the method with an attribute to let users know
when the track list is "ready".

Establishing tracks was previously user driven by a call to
decodeMetadat(). After further consideration, this seems needlessly
complex. Decoding track metadata is not resource intensive and we expect
that most usage of ImageDecoder is such that they wish to decode actual
frames right away. Users who wish to defer decoding track ImageData may
still do so by deferring construction of the ImageDecoder.

This commit also includes other small fixes (formatting, output
timestamp and duration, and closing ImageDecoder if we fail to establish
tracks once data is "complete").
---
 index.src.html | 131 +++++++++++++++++++++----------------------------
 1 file changed, 56 insertions(+), 75 deletions(-)

diff --git a/index.src.html b/index.src.html
index f0bec735..3f0b2767 100644
--- a/index.src.html
+++ b/index.src.html
@@ -614,29 +614,32 @@
   

     Run these steps:
     1. For each |output| in |outputs|:
-        1. Let |frame| be the result of running the [=Create a VideoFrame=]
-            algorithm with |output|.
-        5. Invoke {{VideoDecoder/[[output callback]]}} with |frame|.
+        1. Let |timestamp| and |duration| be the
+            {{EncodedVideoChunk/timestamp}} and {{EncodedVideoChunk/duration}}
+            from the {{EncodedVideoChunk}} associated with |output|.
+        2. Let |frame| be the result of running the [=Create a VideoFrame=]
+            algorithm with |output|, |timestamp|, and |duration|.
+        3. Invoke {{VideoDecoder/[[output callback]]}} with |frame|.
   

-  
Create a VideoFrame (with |output|)

+  

+    Create a VideoFrame (with |output|, |timestamp|, and |duration|)
+  

   

     1. Let |planes| be a sequence of {{Plane}}s containing the decoded video
             frame data from |output|.
     2. Let |pixelFormat| be the {{PixelFormat}} of |planes|.
     3. Let |frameInit| be a {{VideoFrameInit}} with the following keys:
-        1. Let {{VideoFrameInit/timestamp}} and {{VideoFrameInit/duration}}
-            be the {{EncodedVideoChunk/timestamp}} and
-            {{EncodedVideoChunk/duration}} from the {{EncodedVideoChunk}}
-            associated with |output|.
-        2. Let {{VideoFrameInit/codedWidth}} and
+        1. Assign |timestamp| to {{VideoFrameInit/timestamp}}.
+        2. Assign |duration| to {{VideoFrameInit/duration}}.
+        3. Let {{VideoFrameInit/codedWidth}} and
             {{VideoFrameInit/codedHeight}}
             be the width and height of the decoded video frame |output| in
             pixels, prior to any cropping or aspect ratio adjustments.
-        3. Let {{VideoFrameInit/cropLeft}}, {{VideoFrameInit/cropTop}},
+        4. Let {{VideoFrameInit/cropLeft}}, {{VideoFrameInit/cropTop}},
             {{VideoFrameInit/cropWidth}}, and {{VideoFrameInit/cropHeight}}
             be the crop region of the decoded video frame |output| in
             pixels, prior to any aspect ratio adjustments.
-        4. Let {{VideoFrameInit/displayWidth}} and
+        5. Let {{VideoFrameInit/displayWidth}} and
             {{VideoFrameInit/displayHeight}} be the display size of the
             decoded video frame in pixels.
     4. Return a new {{VideoFrame}}, constructed with |pixelFormat|,
@@ -2325,7 +2328,6 @@
   readonly attribute boolean complete;
   readonly attribute ImageTrackList tracks;
 
-  Promise decodeMetadata();
   Promise decode(optional ImageDecodeOptions options);
   undefined reset();
   undefined close();
@@ -2355,9 +2357,6 @@
 :: A boolean reflecting the value of {{ImageDecoderInit/preferAnimation}} given
     at construction.
 
-: [[pending metadata promises]]
-:: A list of unresolved promises returned by calls to decodeMetadata().
-
 : [[pending decode promises]]
 :: A list of unresolved promises returned by calls to decode().
 
@@ -2365,10 +2364,6 @@
 :: Identifies the image track within {{ImageDecoder/[[encoded data]]}} that is
     used by decoding algorithms on the [=codec thread=].
 
-: [[decode metadata requested]]
-:: A boolean indicating whether a [=control message=] to [=ImageDecoder/decode
-    track metadata=] has been enqueued to the [=control message queue=].
-
 : [[tracks established]]
 :: A boolean indicating whether the track list has been established in
     {{ImageDecoder/[[ImageTrackList]]}}.
@@ -2408,15 +2403,13 @@
     5. If `init.preferAnimation` [=map/exists=], assign `init.preferAnimation`
         to the {{ImageDecoder/[[prefer animation]]}} internal slot. Otherwise,
         assign 'null' to {{ImageDecoder/[[prefer animation]]}} internal slot.
-    6. Assign a new [=list=] to {{ImageDecoder/[[pending metadata promises]]}}.
     7. Assign a new [=list=] to {{ImageDecoder/[[pending decode promises]]}}.
     8. Assign `-1` to {{ImageDecoder/[[internal selected track index]]}}.
-    9. Assign `false` to {{ImageDecoder/[[decode metadata requested]]}}.
-    10. Assign `false` to {{ImageDecoder/[[tracks established]]}}.
-    11. Assign `false` to {{ImageDecoder/[[closed]]}}.
-    12. Assign a new [=map=] to {{ImageDecoder/[[progressive frame
+    9. Assign `false` to {{ImageDecoder/[[tracks established]]}}.
+    10. Assign `false` to {{ImageDecoder/[[closed]]}}.
+    11. Assign a new [=map=] to {{ImageDecoder/[[progressive frame
         generations]]}}.
-    13. If |init|'s {{ImageDecoderInit/data}} member is of type
+    12. If |init|'s {{ImageDecoderInit/data}} member is of type
         {{ReadableStream}}:
         1. Assign a new [=list=] to {{ImageDecoder/[[encoded data]]}}.
         2. Assign `false` to {{ImageDecoder/complete}}
@@ -2426,13 +2419,14 @@
             {{ImageDecoderInit/data}}.
         5. In parallel, perform the [=Fetch Stream Data Loop=] on |d| with
             |reader|.
-    14. Otherwise:
+    13. Otherwise:
         1. Assert that `init.data` is of type {{BufferSource}}.
         2. Assign a copy of `init.data` to  {{ImageDecoder/[[encoded data]]}}.
         3. Assign `true` to {{ImageDecoder/complete}}
-    15. Queue a control message to [=configure the image decoder=] with
-        |init|.
-    16. return |d|.
+        4. Queue a control message to [=configure the image decoder=] with
+            |init|.
+        5. Queue a control message to [=decode track metadata=].
+    14. return |d|.
 
     [=Running a control message=] to configure the image decoder
     means running these steps:
@@ -2451,6 +2445,10 @@
         {{ImageDecoderInit/desiredWidth}}, and
         {{ImageDecoderInit/desiredHeight}}.
 
+    [=Running a control message=] to decode track metadata means
+    running these steps:
+    1. Run the [=ImageDecoder/Establish Tracks=] algorithm.
+
 ### Attributes ### {#imagedecoder-attributes}
 
 : tracks
@@ -2468,31 +2466,6 @@
 
 
 ### Methods ### {#imagedecoder-methods}
-: decodeMetadata()
-:: Returns a promise which resolves once enough
-    {{ImageDecoder/[[encoded data]]}} has been buffered and decoded to
-    establish {{ImageDecoder/tracks}}.
-
-    NOTE: {{ImageTrack}} {{ImageTrack/frameCount}} may receive subsequent
-        updates until {{ImageDecoder/complete}} is `true`.
-
-    When invoked, run these steps:
-    1. If {{ImageDecoder/[[closed]]}} is `true`, return a {{Promise}}
-        rejected with an {{InvalidStateError}} {{DOMException}}.
-    2. If {{ImageDecoder/[[tracks established]]}} is `true`, return a resolved
-        {{Promise}}.
-    3. Let |promise| be a new {{Promise}}.
-    4. Append |promise| to {{ImageDecoder/[[pending metadata promises]]}}
-    5. If {{ImageDecoder/[[decode metadata requested]]}} is `false`:
-        1. [=Queue a control message=] to [=ImageDecoder/decode track
-            metadata=].
-        2. Assign `true` to {{ImageDecoder/[[decode metadata requested]]}}.
-    5. Return |promise|.
-
-    [=Running a control message=] to decode track
-    metadata means running these steps:
-    1. Run the [=ImageDecoder/Establish Tracks=] algorithm.
-
 : decode(options)
 :: Enqueues a control message to decode the frame according to |options|.
 
@@ -2504,15 +2477,11 @@
         rejected with an {{InvalidStateError}} {{DOMException}}.
     3. If |options| is `undefined`, assign a new {{ImageDecodeOptions}} to
         |options|.
-    4. If {{ImageDecoder/[[decode metadata requested]]}} is `false`:
-        1. [=Queue a control message=] to [=ImageDecoder/decode track
-            metadata=].
-        2. Assign `true` to {{ImageDecoder/[[decode metadata requested]]}}.
-    5. Let |promise| be a new {{Promise}}.
-    6. [=Queue a control message=] to decode the the image with |options|, and
+    4. Let |promise| be a new {{Promise}}.
+    5. [=Queue a control message=] to decode the the image with |options|, and
         |promise|.
-    7. Append |promise| to {{ImageDecoder/[[pending decode promises]]}}.
-    8. Return |promise|.
+    6. Append |promise| to {{ImageDecoder/[[pending decode promises]]}}.
+    7. Return |promise|.
 
     [=Running a control message=] to decode the image means running these
     steps:
@@ -2583,8 +2552,11 @@
 :: Run these steps:
     1. Assert {{ImageDecoder/[[tracks established]]}} is `false`.
     2. If {{ImageDecoder/[[encoded data]]}} does not contain enough data to
-        determine the number of tracks, abort these steps.
-    3. If the number of tracks is found to be '0', queue a task on the
+        determine the number of tracks:
+        1. If {{ImageDecoder/complete}} is `true`, queue a task on the
+            [=control thread=] event loop to run the [=ImageDecoder/Close ImageDecoder=] algorithm.
+        2. Abort these steps.
+    3. If the number of tracks is found to be `0`, queue a task on the
         [=control thread=] event loop to run the
         [=ImageDecoder/Close ImageDecoder=] algorithm and abort these steps.
     4. Let |newTrackList| be a new [=list=].
@@ -2631,9 +2603,7 @@
             {{ImageTrackList/[[track list]]}} internal slot.
         2. Assign |selectedTrackIndex| to {{ImageDecoder/tracks}}
             {{ImageTrackList/[[selected index]]}}.
-        3. For each |promise| in
-            {{ImageDecoder/[[pending metadata promises]]}}, resolve |promise|
-            and remove it from the list.
+        3. Resolve {{ImageTrackList/[[ready promise]]}}.
 
 : Get Default Selected Track Index (with
     |trackList|)
@@ -2722,8 +2692,14 @@
     11. Let |decodeResult| be a new {{ImageDecodeResult}} initialized as
         follows:
         1. Assign 'true' to {{ImageDecodeResult/complete}}.
+        2. Let |timestamp| and |duration| be the presentation timestamp and
+            duration for |output| as described by the
+            {{ImageDecoder/[[encoded data]]}}. If
+            {{ImageDecoder/[[encoded data]]}} does not describe a timestamp or
+            duration, assign `null` to the corresponding variable.
         2. Assign {{ImageDecodeResult/image}} with the result of running the
-            [=Create a VideoFrame=] algorithm with |output|.
+            [=Create a VideoFrame=] algorithm with |output|, |timestamp|, and
+            |duration|.
     12. Run the [=ImageDecoder/Resolve Decode=] algorithm with |promise| and
         |decodeResult|.
 
@@ -2814,12 +2790,7 @@
 : Reset ImageDecoder (with |exception|)
 :: 1. Signal {{ImageDecoder/[[codec implementation]]}} to abort any active
         decoding operation.
-    2. For each |metadataPromise| in
-        {{ImageDecoder/[[pending metadata promises]]}}:
-        1. Reject |metadataPromise| with |exception|.
-        2. Remove |metadataPromise| from
-            {{ImageDecoder/[[pending metadata promises]]}}.
-    3. For each |decodePromise| in
+    2. For each |decodePromise| in
         {{ImageDecoder/[[pending decode promises]]}}:
         1. Reject |decodePromise| with |exception|.
         2. Remove |decodePromise| from
@@ -2972,6 +2943,7 @@
 interface ImageTrackList {
   getter ImageTrack (unsigned long index);
 
+  readonly attribute Promise ready;
   readonly attribute unsigned long length;
   readonly attribute long selectedIndex;
   readonly attribute ImageTrack? selectedTrack;
@@ -2980,6 +2952,12 @@
 

 
 ### Internal Slots ### {#imagetracklist-internal-slots}
+: [[ready promise]]
+:: The promise used to signal when the {{ImageTrackList}} has been poplated
+    with {{ImageTrack}}s.
+
+    NOTE: {{ImageTrack}} {{ImageTrack/frameCount}} may receive subsequent
+        updates until {{ImageDecoder/complete}} is `true`.
 
 : [[track list]]
 :: The list of {{ImageTrack}}s describe by this {{ImageTrackList}}.
@@ -2989,6 +2967,9 @@
     value of `-1` indeicates that no track is selected.
 
 ### Attributes ### {#imagetracklist-attributes}
+: ready
+:: The {{ImageTrackList/ready}} getter steps are to return the
+    {{ImageTrackList/[[ready promise]]}}.
 
 : length
 :: The {{ImageTrackList/length}} getter steps are to return the length of
@@ -3046,7 +3027,7 @@
 
     NOTE: This attribute provides an early indication that
         {{ImageTrack/frameCount}} will ultimately exceed 0 for images where the
-        {{ImageTrack/frameCount}} starts at '0' and later increments as new
+        {{ImageTrack/frameCount}} starts at `0` and later increments as new
         chunks of the {{ReadableStream}} {{ImageDecoderInit/data}} arrive.
 
 : frameCount

From a63ee9bb3ae8d9cb98e67ea573839fb42ba7706a Mon Sep 17 00:00:00 2001
From: Chris Cunningham 
Date: Fri, 23 Apr 2021 12:43:02 -0700
Subject: [PATCH 7/9] Small language fixes

---
 index.src.html | 20 ++++++++++++--------
 1 file changed, 12 insertions(+), 8 deletions(-)

diff --git a/index.src.html b/index.src.html
index 3f0b2767..cfb517f2 100644
--- a/index.src.html
+++ b/index.src.html
@@ -2693,11 +2693,10 @@
         follows:
         1. Assign 'true' to {{ImageDecodeResult/complete}}.
         2. Let |timestamp| and |duration| be the presentation timestamp and
-            duration for |output| as described by the
-            {{ImageDecoder/[[encoded data]]}}. If
-            {{ImageDecoder/[[encoded data]]}} does not describe a timestamp or
+            duration for |output| as described by |encodedFrame|. If
+            |encodedFrame| does not describe a timestamp or
             duration, assign `null` to the corresponding variable.
-        2. Assign {{ImageDecodeResult/image}} with the result of running the
+        3. Assign {{ImageDecodeResult/image}} with the result of running the
             [=Create a VideoFrame=] algorithm with |output|, |timestamp|, and
             |duration|.
     12. Run the [=ImageDecoder/Resolve Decode=] algorithm with |promise| and
@@ -2750,10 +2749,15 @@
             for |output|.
         3. Add a new entry to {{ImageDecoder/[[progressive frame
             generations]]}} with key |frameIndex| and value |frameGeneration|.
-    15. Assign |decodeResult|.{{ImageDecodeResult/image}} with the result of
-        running the [=Create a VideoFrame=] algorithm with |output|.
-    16. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
-    17. Resolve |promise| with |decodeResult|.
+    15. Let |timestamp| and |duration| be the presentation timestamp and
+            duration for |output| as described by |encodedFrame|. If
+            |encodedFrame| does not describe a timestamp or
+            duration, assign `null` to the corresponding variable.
+    16. Assign {{ImageDecodeResult/image}} with the result of running the
+            [=Create a VideoFrame=] algorithm with |output|, |timestamp|, and
+            |duration|.
+    17. Remove |promise| from {{ImageDecoder/[[pending decode promises]]}}.
+    18. Resolve |promise| with |decodeResult|.
 
 : Resolve Decode (with |promise| and |result|)
 :: 1. Queue a task on the [=control thread=] event loop to run these steps:

From a6fe5053637d8b67c3140282f0103dcf32438a9e Mon Sep 17 00:00:00 2001
From: Chris Cunningham 
Date: Wed, 28 Apr 2021 16:00:38 -0700
Subject: [PATCH 8/9] Clarify that ImageDecoder.tracks returns a live object

---
 index.src.html | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/index.src.html b/index.src.html
index cfb517f2..a0fe983b 100644
--- a/index.src.html
+++ b/index.src.html
@@ -41,6 +41,7 @@
         type: attribute; text: resizeWidth; url:#dom-imagebitmapoptions-resizewidth
         type: attribute; text: resizeHeight; url:#dom-imagebitmapoptions-resizeheight
     type: dfn; text: global object; url: webappapis.html#global-object
+    type: dfn; text: live; url: infrastructure.html#live
 
 spec: mediacapture-streams; urlPrefix: https://www.w3.org/TR/mediacapture-streams/
     for: mediaDevices;
@@ -2452,8 +2453,8 @@
 ### Attributes ### {#imagedecoder-attributes}
 
 : tracks
-:: Provides metadata for the available tracks and a mechanism for selecting a
-    track to decode.
+:: Returns a [=live=] {{ImageTrackList}}, which provides metadata
+    for the available tracks and a mechanism for selecting a track to decode.
 
     The {{ImageDecoder/tracks}} getter steps are to return
     {{ImageDecoder/[[ImageTrackList]]}}.

From 051cc10e4a22146a376b50bf4dd3c9d685c8fca5 Mon Sep 17 00:00:00 2001
From: Chris Cunningham 
Date: Thu, 29 Apr 2021 22:27:01 -0700
Subject: [PATCH 9/9] Add completed promise. Fix typo

---
 index.src.html | 33 ++++++++++++++++++++++-----------
 1 file changed, 22 insertions(+), 11 deletions(-)

diff --git a/index.src.html b/index.src.html
index d78f8561..33f7052e 100644
--- a/index.src.html
+++ b/index.src.html
@@ -2410,6 +2410,7 @@
   constructor(ImageDecoderInit init);
 
   readonly attribute boolean complete;
+  readonly attribute Promise completed;
   readonly attribute ImageTrackList tracks;
 
   Promise decode(optional ImageDecodeOptions options);
@@ -2431,6 +2432,10 @@
 :: A boolean indicating whether {{ImageDecoder/[[encoded data]]}} is completely
     buffered.
 
+: [[completed promise]]
+:: The promise used to signal when {{ImageDecoder/[[complete]]}} becomes
+    `true`.
+
 : [[codec implementation]]
 :: An underlying image decoder implementation provided by the User Agent.
 
@@ -2506,10 +2511,11 @@
     13. Otherwise:
         1. Assert that `init.data` is of type {{BufferSource}}.
         2. Assign a copy of `init.data` to  {{ImageDecoder/[[encoded data]]}}.
-        3. Assign `true` to {{ImageDecoder/complete}}
-        4. Queue a control message to [=configure the image decoder=] with
+        3. Assign `true` to {{ImageDecoder/complete}}.
+        4. Reslove {{ImageDecoder/[[completed promise]]}}.
+        5. Queue a control message to [=configure the image decoder=] with
             |init|.
-        5. Queue a control message to [=decode track metadata=].
+        6. Queue a control message to [=decode track metadata=].
     14. return |d|.
 
     [=Running a control message=] to configure the image decoder
@@ -2534,6 +2540,17 @@
     1. Run the [=ImageDecoder/Establish Tracks=] algorithm.
 
 ### Attributes ### {#imagedecoder-attributes}
+: complete
+:: Indicates whether {{ImageDecoder/[[encoded data]]}} is completely buffered.
+
+    The {{ImageDecoder/complete}} getter steps are to return
+    {{ImageDecoder/[[complete]]}}.
+
+: completed
+:: The promise used to signal when {{ImageDecoder/complete}} becomes `true`.
+
+    The {{ImageDecoder/completed}} getter steps are to return
+    {{ImageDecoder/[[completed promise]]}}.
 
 : tracks
 :: Returns a [=live=] {{ImageTrackList}}, which provides metadata
@@ -2542,13 +2559,6 @@
     The {{ImageDecoder/tracks}} getter steps are to return
     {{ImageDecoder/[[ImageTrackList]]}}.
 
-: complete
-:: Indicates whether {{ImageDecoder/[[encoded data]]}} is completely buffered.
-
-    The {{ImageDecoder/complete}} getter steps are to return
-    {{ImageDecoder/[[complete]]}}.
-
-
 ### Methods ### {#imagedecoder-methods}
 : decode(options)
 :: Enqueues a control message to decode the frame according to |options|.
@@ -2624,6 +2634,7 @@
 
         : [=read request/close steps=]
         :: 1. Assign `true` to {{ImageDecoder/complete}}
+            2. Resolve {{ImageDecoder/[[completed promise]]}}.
 
         : [=read request/error steps=]
         :: 1. Queue a task on the [=control thread=] event loop to run the
@@ -3041,7 +3052,7 @@
 
 ### Internal Slots ### {#imagetracklist-internal-slots}
 : [[ready promise]]
-:: The promise used to signal when the {{ImageTrackList}} has been poplated
+:: The promise used to signal when the {{ImageTrackList}} has been populated
     with {{ImageTrack}}s.
 
     NOTE: {{ImageTrack}} {{ImageTrack/frameCount}} may receive subsequent