Skip to content

Commit 73c5464

Browse files
youennfyouennf
authored
Introduce RTCEncodedFrameMetadata dictionary (#250)
1 parent 9745de0 commit 73c5464

File tree

1 file changed

+102
-179
lines changed

1 file changed

+102
-179
lines changed

‎index.bs

Lines changed: 102 additions & 179 deletions
Original file line numberDiff line numberDiff line change
@@ -318,8 +318,104 @@ The <dfn method for="SFrameTransform">setEncryptionKey(|key|, |keyID|)</dfn> met
318318
# RTCRtpScriptTransform # {#scriptTransform}
319319

320320
In this section, the capture system refers to the system where media is sourced from and the sender system
321-
refers to the system that is sending RTP and RTCP packets to the receiver system where {{RTCEncodedVideoFrameMetadata}} data
322-
or {{RTCEncodedAudioFrameMetadata}} data is populated.
321+
refers to the system that is sending RTP and RTCP packets to the receiver system where {{RTCEncodedFrameMetadata}} data is populated.
322+
323+
## <dfn dictionary>RTCEncodedFrameMetadata</dfn> dictionary ## {#RTCEncodedFrameMetadata}
324+
<pre class="idl">
325+
dictionary RTCEncodedFrameMetadata {
326+
unsigned long synchronizationSource;
327+
octet payloadType;
328+
sequence&lt;unsigned long&gt; contributingSources;
329+
unsigned long rtpTimestamp;
330+
DOMHighResTimeStamp receiveTime;
331+
DOMHighResTimeStamp captureTime;
332+
DOMHighResTimeStamp senderCaptureTimeOffset;
333+
DOMString mimeType;
334+
};
335+
</pre>
336+
337+
### Members ### {#RTCEncodedFrameMetadata-members}
338+
339+
<dl dfn-for="RTCEncodedFrameMetadata" class="dictionary-members">
340+
<dt>
341+
<dfn dict-member>synchronizationSource</dfn> <span class="idlMemberType">unsigned long</span>
342+
</dt>
343+
<dd>
344+
<p>
345+
The synchronization source (ssrc) identifier is an unsigned integer value per [[RFC3550]]
346+
used to identify the stream of RTP packets that the encoded frame object is describing.
347+
</p>
348+
</dd>
349+
<dt>
350+
<dfn dict-member>payloadType</dfn> <span class="idlMemberType">octet</span>
351+
</dt>
352+
<dd>
353+
<p>
354+
The payload type is an unsigned integer value in the range from 0 to 127 per [[RFC3550]]
355+
that is used to describe the format of the RTP payload.
356+
</p>
357+
</dd>
358+
<dt>
359+
<dfn dict-member>contributingSources</dfn> <span class=
360+
"idlMemberType">sequence&lt;unsigned long&gt;</span>
361+
</dt>
362+
<dd>
363+
<p>
364+
The list of contribution sources (csrc list) as defined in [[RFC3550]].
365+
</p>
366+
</dd>
367+
<dt>
368+
<dfn dict-member>rtpTimestamp</dfn> <span class=
369+
"idlMemberType">unsigned long</span>
370+
</dt>
371+
<dd>
372+
<p>
373+
The RTP timestamp identifier is an unsigned integer value per [[RFC3550]]
374+
that reflects the sampling instant of the first octet in the RTP data packet.
375+
</p>
376+
</dd>
377+
<dt>
378+
<dfn dict-member>receiveTime</dfn> <span class=
379+
"idlMemberType">DOMHighResTimeStamp</span>
380+
</dt>
381+
<dd>
382+
<p>
383+
For frames coming from an RTCRtpReceiver, represents the timestamp
384+
of the last received packet used to produce this media frame. This
385+
timestamp is relative to {{Performance}}.{{Performance/timeOrigin}}.
386+
</p>
387+
</dd>
388+
<dt>
389+
<dfn dict-member>captureTime</dfn> <span class="idlMemberType">DOMHighResTimeStamp</span>
390+
</dt>
391+
<dd>
392+
<p>
393+
The capture time of this frame in the capture system's clock.
394+
On populating this member, the user agent MUST return the value of the frame's `[[captureTime]]` slot,
395+
shifted to be relative to {{Performance}}.{{Performance/timeOrigin}}.
396+
</p>
397+
</dd>
398+
<dt>
399+
<dfn dict-member>senderCaptureTimeOffset</dfn> <span class="idlMemberType">DOMHighResTimeStamp</span>
400+
</dt>
401+
<dd>
402+
<p>
403+
The {{RTCEncodedFrameMetadata/senderCaptureTimeOffset}} is the sender system's estimate of the offset
404+
between its own NTP clock and the capture system's NTP clock, for the same frame that the
405+
{{RTCEncodedFrameMetadata/captureTime}} was originated from.
406+
On populating this member, the user agent MUST return the value of the frame's `[[senderCaptureTimeOffset]]` slot.
407+
</p>
408+
</dd>
409+
<dt>
410+
<dfn dict-member>mimeType</dfn> <span class="idlMemberType">DOMString</span>
411+
</dt>
412+
<dd>
413+
<p>
414+
The codec MIME media type/subtype defined in the IANA media types registry
415+
[[!IANA-MEDIA-TYPES]], e.g. audio/opus or video/VP8.
416+
</p>
417+
</dd>
418+
</dl>
323419

324420
## <dfn enum>RTCEncodedVideoFrameType</dfn> dictionary ## {#RTCEncodedVideoFrameType}
325421
<pre class="idl">
@@ -374,22 +470,14 @@ enum RTCEncodedVideoFrameType {
374470

375471
## <dfn dictionary>RTCEncodedVideoFrameMetadata</dfn> dictionary ## {#RTCEncodedVideoFrameMetadata}
376472
<pre class="idl">
377-
dictionary RTCEncodedVideoFrameMetadata {
473+
dictionary RTCEncodedVideoFrameMetadata : RTCEncodedFrameMetadata {
378474
unsigned long long frameId;
379475
sequence&lt;unsigned long long&gt; dependencies;
380476
unsigned short width;
381477
unsigned short height;
382478
unsigned long spatialIndex;
383479
unsigned long temporalIndex;
384-
unsigned long synchronizationSource;
385-
octet payloadType;
386-
sequence&lt;unsigned long&gt; contributingSources;
387480
long long timestamp; // microseconds
388-
unsigned long rtpTimestamp;
389-
DOMHighResTimeStamp receiveTime;
390-
DOMHighResTimeStamp captureTime;
391-
DOMHighResTimeStamp senderCaptureTimeOffset;
392-
DOMString mimeType;
393481
};
394482
</pre>
395483

@@ -415,33 +503,6 @@ dictionary RTCEncodedVideoFrameMetadata {
415503
Only present for received frames if the AV1 Dependency Descriptor Header Extension defined in Appendix A of [[AV1-RTP-SPEC]] is present.
416504
</p>
417505
</dd>
418-
<dt>
419-
<dfn dict-member>synchronizationSource</dfn> <span class="idlMemberType">unsigned long</span>
420-
</dt>
421-
<dd>
422-
<p>
423-
The synchronization source (ssrc) identifier is an unsigned integer value per [[RFC3550]]
424-
used to identify the stream of RTP packets that the encoded frame object is describing.
425-
</p>
426-
</dd>
427-
<dt>
428-
<dfn dict-member>payloadType</dfn> <span class="idlMemberType">octet</span>
429-
</dt>
430-
<dd>
431-
<p>
432-
The payload type is an unsigned integer value in the range from 0 to 127 per [[RFC3550]]
433-
that is used to describe the format of the RTP payload.
434-
</p>
435-
</dd>
436-
<dt>
437-
<dfn dict-member>contributingSources</dfn> <span class=
438-
"idlMemberType">sequence&lt;unsigned long&gt;</span>
439-
</dt>
440-
<dd>
441-
<p>
442-
The list of contribution sources (csrc list) as defined in [[RFC3550]].
443-
</p>
444-
</dd>
445506
<dt>
446507
<dfn dict-member>timestamp</dfn> <span class=
447508
"idlMemberType">long long</span>
@@ -452,58 +513,6 @@ dictionary RTCEncodedVideoFrameMetadata {
452513
{{VideoFrame/timestamp}} for raw frames which correspond to this frame.
453514
</p>
454515
</dd>
455-
<dt>
456-
<dfn dict-member>rtpTimestamp</dfn> <span class=
457-
"idlMemberType">unsigned long</span>
458-
</dt>
459-
<dd>
460-
<p>
461-
The RTP timestamp identifier is an unsigned integer value per [[RFC3550]]
462-
that reflects the sampling instant of the first octet in the RTP data packet.
463-
</p>
464-
</dd>
465-
<dt>
466-
<dfn dict-member>receiveTime</dfn> <span class=
467-
"idlMemberType">DOMHighResTimeStamp</span>
468-
</dt>
469-
<dd>
470-
<p>
471-
For frames coming from an RTCRtpReceiver, represents the timestamp
472-
of the last received packet used to produce this video frame. This
473-
timestamp is relative to {{Performance}}.{{Performance/timeOrigin}}.
474-
Only exists for incoming video frames.
475-
</p>
476-
</dd>
477-
<dt>
478-
<dfn dict-member>captureTime</dfn> <span class="idlMemberType">DOMHighResTimeStamp</span>
479-
</dt>
480-
<dd>
481-
<p>
482-
The capture time of this frame in the capture system's clock.
483-
On populating this member, the user agent MUST return the value of the frame's `[[captureTime]]` slot,
484-
shifted to be relative to {{Performance}}.{{Performance/timeOrigin}}.
485-
</p>
486-
</dd>
487-
<dt>
488-
<dfn dict-member>senderCaptureTimeOffset</dfn> <span class="idlMemberType">DOMHighResTimeStamp</span>
489-
</dt>
490-
<dd>
491-
<p>
492-
The {{RTCEncodedVideoFrameMetadata/senderCaptureTimeOffset}} is the sender system's estimate of the offset
493-
between its own NTP clock and the capture system's NTP clock, for the same frame that the
494-
{{RTCEncodedVideoFrameMetadata/captureTime}} was originated from.
495-
On populating this member, the user agent MUST return the value of the frame's `[[senderCaptureTimeOffset]]` slot.
496-
</p>
497-
</dd>
498-
<dt>
499-
<dfn dict-member>mimeType</dfn> <span class="idlMemberType">DOMString</span>
500-
</dt>
501-
<dd>
502-
<p>
503-
The codec MIME media type/subtype defined in the IANA media types registry
504-
[[!IANA-MEDIA-TYPES]], e.g. video/VP8.
505-
</p>
506-
</dd>
507516
</dl>
508517

509518

@@ -567,7 +576,7 @@ interface RTCEncodedVideoFrame {
567576
<p>
568577
The encoded frame data. The format of the data depends on the video codec that is
569578
used to encode/decode the frame which can be determined by looking at the
570-
{{RTCEncodedVideoFrameMetadata/mimeType}}.
579+
{{RTCEncodedFrameMetadata/mimeType}}.
571580
For <a href="https://w3c.github.io/webrtc-svc/">SVC</a>, each spatial layer
572581
is transformed separately.
573582
</p>
@@ -672,48 +681,13 @@ An implementation is therefore free to choose whatever method works best.
672681

673682
## <dfn dictionary>RTCEncodedAudioFrameMetadata</dfn> dictionary ## {#RTCEncodedAudioFrameMetadata}
674683
<pre class="idl">
675-
dictionary RTCEncodedAudioFrameMetadata {
676-
unsigned long synchronizationSource;
677-
octet payloadType;
678-
sequence&lt;unsigned long&gt; contributingSources;
684+
dictionary RTCEncodedAudioFrameMetadata : RTCEncodedFrameMetadata {
679685
short sequenceNumber;
680-
unsigned long rtpTimestamp;
681-
DOMHighResTimeStamp receiveTime;
682-
DOMHighResTimeStamp captureTime;
683-
DOMHighResTimeStamp senderCaptureTimeOffset;
684-
DOMString mimeType;
685686
double audioLevel;
686687
};
687688
</pre>
688689
### Members ### {#RTCEncodedAudioFrameMetadata-members}
689690
<dl dfn-for="RTCEncodedAudioFrameMetadata" class="dictionary-members">
690-
<dt>
691-
<dfn dict-member>synchronizationSource</dfn> <span class="idlMemberType">unsigned long</span>
692-
</dt>
693-
<dd>
694-
<p>
695-
The synchronization source (ssrc) identifier is an unsigned integer value per [[RFC3550]]
696-
used to identify the stream of RTP packets that the encoded frame object is describing.
697-
</p>
698-
</dd>
699-
<dt>
700-
<dfn dict-member>payloadType</dfn> <span class="idlMemberType">octet</span>
701-
</dt>
702-
<dd>
703-
<p>
704-
The payload type is an unsigned integer value in the range from 0 to 127 per [[RFC3550]]
705-
that is used to describe the format of the RTP payload.
706-
</p>
707-
</dd>
708-
<dt>
709-
<dfn dict-member>contributingSources</dfn> <span class=
710-
"idlMemberType">sequence&lt;unsigned long&gt;</span>
711-
</dt>
712-
<dd>
713-
<p>
714-
The list of contribution sources (csrc list) as defined in [[RFC3550]].
715-
</p>
716-
</dd>
717691
<dt>
718692
<dfn dict-member>sequenceNumber</dfn> <span class=
719693
"idlMemberType">short</span>
@@ -726,57 +700,6 @@ dictionary RTCEncodedAudioFrameMetadata {
726700
Comparing two sequence numbers requires serial number arithmetic described in [[RFC1982]].
727701
</p>
728702
</dd>
729-
<dt>
730-
<dfn dict-member>rtpTimestamp</dfn> <span class="idlMemberType">unsigned long</span>
731-
</dt>
732-
<dd>
733-
<p>
734-
The RTP timestamp identifier is an unsigned integer value per [[RFC3550]]
735-
that reflects the sampling instant of the first octet in the RTP data packet.
736-
</p>
737-
</dd>
738-
<dt>
739-
<dfn dict-member>receiveTime</dfn> <span class=
740-
"idlMemberType">DOMHighResTimeStamp</span>
741-
</dt>
742-
<dd>
743-
<p>
744-
For frames coming from an RTCRtpReceiver, represents the timestamp
745-
of the last received packet used to produce this audio frame. This
746-
timestamp is relative to {{Performance}}.{{Performance/timeOrigin}}.
747-
Only exists for incoming audio frames.
748-
</p>
749-
</dd>
750-
<dt>
751-
<dfn dict-member>captureTime</dfn> <span class="idlMemberType">DOMHighResTimeStamp</span>
752-
</dt>
753-
<dd>
754-
<p>
755-
The capture time of this frame in the capture system's clock.
756-
On populating this member, the user agent MUST return the value of the frame's `[[captureTime]]` slot,
757-
shifted to be relative to {{Performance}}.{{Performance/timeOrigin}}.
758-
</p>
759-
</dd>
760-
<dt>
761-
<dfn dict-member>senderCaptureTimeOffset</dfn> <span class="idlMemberType">DOMHighResTimeStamp</span>
762-
</dt>
763-
<dd>
764-
<p>
765-
The {{RTCEncodedAudioFrameMetadata/senderCaptureTimeOffset}} is the sender system's estimate of the offset
766-
between its own NTP clock and the capture system's NTP clock, for the same frame that the
767-
{{RTCEncodedAudioFrameMetadata/captureTime}} was originated from.
768-
On populating this member, the user agent MUST return the value of the frame's `[[senderCaptureTimeOffset]]` slot.
769-
</p>
770-
</dd>
771-
<dt>
772-
<dfn dict-member>mimeType</dfn> <span class="idlMemberType">DOMString</span>
773-
</dt>
774-
<dd>
775-
<p>
776-
The codec MIME media type/subtype defined in the IANA media types registry
777-
[[!IANA-MEDIA-TYPES]], e.g. audio/opus.
778-
</p>
779-
</dd>
780703
<dt>
781704
<dfn dict-member>audioLevel</dfn> <span class="idlMemberType">double</span>
782705
</dt>
@@ -836,7 +759,7 @@ interface RTCEncodedAudioFrame {
836759
<p>
837760
The encoded frame data. The format of the data depends on the audio codec that is
838761
used to encode/decode the frame which can be determined by looking at the
839-
{{RTCEncodedAudioFrameMetadata/mimeType}}.
762+
{{RTCEncodedFrameMetadata/mimeType}}.
840763
The following table gives a number of examples:
841764
</p>
842765
<table class="simple">

0 commit comments

Comments
 (0)