XEP-0180: Jingle Video via RTP

This document defines methods for negotiating Jingle video sessions that use the Real-time Transport Protocol (RTP) for media exchange.


WARNING: This Standards-Track document is Experimental. Publication as an XMPP Extension Protocol does not imply approval of this proposal by the XMPP Standards Foundation. Implementation of the protocol described herein is encouraged in exploratory implementations, but production systems should not deploy implementations of this protocol until it advances to a status of Draft.


Document Information

Series: XEP
Number: 0180
Publisher: XMPP Standards Foundation
Status: Experimental
Type: Standards Track
Version: 0.8
Last Updated: 2007-05-23
Approving Body: XMPP Council
Dependencies: XMPP Core, XEP-0166
Supersedes: None
Superseded By: None
Short Name: TO BE ASSIGNED
Wiki Page: <http://wiki.jabber.org/index.php/Jingle Video via RTP (XEP-0180)>

Author Information

Peter Saint-Andre

Email: stpeter@jabber.org
JabberID: stpeter@jabber.org

Milton Chen

Email: Milton.Chen@vseelab.com

Legal Notice

This XMPP Extension Protocol is copyright 1999 - 2007 by the XMPP Standards Foundation (XSF) and is in full conformance with the XSF's Intellectual Property Rights Policy <http://www.xmpp.org/extensions/ipr-policy.shtml>. This material may be distributed only subject to the terms and conditions set forth in the Creative Commons Attribution License (<http://creativecommons.org/licenses/by/2.5/>).

Discussion Venue

The preferred venue for discussion of this document is the Standards discussion list: <http://mail.jabber.org/mailman/listinfo/standards>.

Relation to XMPP

The Extensible Messaging and Presence Protocol (XMPP) is defined in the XMPP Core (RFC 3920) and XMPP IM (RFC 3921) specifications contributed by the XMPP Standards Foundation to the Internet Standards Process, which is managed by the Internet Engineering Task Force in accordance with RFC 2026. Any protocol defined in this document has been developed outside the Internet Standards Process and is to be understood as an extension to XMPP rather than as an evolution, development, or modification of XMPP itself.

Conformance Terms

The following keywords as used in this document are to be interpreted as described in RFC 2119: "MUST", "SHALL", "REQUIRED"; "MUST NOT", "SHALL NOT"; "SHOULD", "RECOMMENDED"; "SHOULD NOT", "NOT RECOMMENDED"; "MAY", "OPTIONAL".


Table of Contents


1. Introduction
2. Requirements
3. Jingle Conformance
4. Content Description Format
5. Negotiating a Jingle Video Session
6. Mapping to Session Description Protocol
7. Determining Support
8. Informational Messages
9. Implementation Notes
    9.1. Codecs
10. Security Considerations
11. IANA Considerations
12. XMPP Registrar Considerations
    12.1. Protocol Namespaces
    12.2. Jingle Content Description Formats
13. XML Schemas
    13.1. Content Description Format
    13.2. Errors
Notes
Revision History


1. Introduction

Jingle [1] can be used to initiate and negotiate a wide range of peer-to-peer sessions. One session type of interest is video exchange. This document specifies a format for describing Jingle video sessions, where the media exchange occurs using the Real-time Transport Protocol (see RFC 3550 [2]).

2. Requirements

The Jingle content description format defined herein is designed to meet the following requirements:

  1. Enable negotiation of parameters necessary for video exchange.
  2. Map these parameters to the Session Description Protocol (SDP; see RFC 4566 [3]) to enable interoperability.
  3. Define informational messages related to video chat.

3. Jingle Conformance

In accordance with Section 8 of XEP-0166, this document specifies the following information related to the Jingle Video via RTP application type:

  1. The content negotiation process is defined in the Negotiating a Jingle Video Session section of this document.

  2. The semantics of the <description/> element are defined in the Content Description Format section of this document.

  3. A mapping of Jingle semantics to the Session Description Protocol is provided in the Mapping to Session Description Protocol section of this document.

  4. A Jingle video session MUST use a lossy transport method such as Jingle Raw UDP Transport Method [4] or the "ice-udp" method specified in Jingle ICE Transport Method [5].

  5. Content is to be sent and received as follows:

4. Content Description Format

A Jingle video session is described by one or more encodings contained within a wrapper <description/> element qualified by the 'http://www.xmpp.org/extensions/xep-0180.html#ns' namespace (see Protocol Namespaces regarding issuance of one or more permanent namespaces). In the language of RFC 4566 these encodings are payload-types; therefore, each <payload-type/> child element specifies an encoding that can be used for the video stream. Such encodings are often used in the context of the Real-time Transfer Protocol (RTP; see RFC 3550) but may be used in other contexts as well. The most common encodings for the Audio/Video Profile (AVP) of RTP are listed in RFC 3551 [6] (these "static" types are reserved from payload ID 0 through payload ID 95), although other encodings are allowed in accordance with the dynamic assignment rules described in Section 3 of RFC 3551 (these "dynamic" types use payload IDs 96 to 127). The <payload-type/> element's 'id' attribute is REQUIRED and its 'name' attribute is RECOMMENDED. The encodings SHOULD be provided in order of preference.

Example 1. Video Description Format

    <description xmlns='http://www.xmpp.org/extensions/xep-0180.html#ns'>
      <payload-type id='96' name='theora' clockrate='90000' height='720' width='1280'>
        <parameter name='delivery-method' value='inline'/>
        <parameter name='configuration' value='somebase16string'/>
        <parameter name='sampling' value='YCbCr-4:2:2'/>
      </payload-type>
      <payload-type id='28' name='nv' clockrate='90000'/>
      <payload-type id='25' name='CelB' clockrate='90000'/>
      <payload-type id='32' name='MPV' clockrate='90000'/>
    </description>
  

The <description/> element is intended to be a child of a <content/> element as specified in XEP-0166.

The defined attributes of the <payload-type/> element are as follows:

Table 1: Video Description Attributes

Attribute Description Datatype/Units Inclusion
channels The number of channels (e.g., 2 for stereoscopic video) positiveInteger (defaults to 1) OPTIONAL
clockrate The sampling frequency in Hertz positiveInteger RECOMMENDED
height The vertical extent of the displayed video, in pixels positiveInteger RECOMMENDED
id A unique identifier for the payload type positiveInteger REQUIRED
layer The relationship of a layer to the "bottom" of the stack, where 0 = bottom (the first layer) nonNegativeInteger OPTIONAL
name A name for the payload type string RECOMMENDED for static payload types, REQUIRED for dynamic payload types
transparent Whether or not a layer is transparent boolean OPTIONAL
width The horizontal extent of the displayed video, in pixels positiveInteger RECOMMENDED
x The horizontal starting point of a tile, in pixels from the origin point positiveInteger OPTIONAL
y The vertical starting point of a tile, in pixels from the origin point positiveInteger OPTIONAL

Each <payload-type/> element MAY contain one or more child elements that specify particular parameters related to the payload. For example, as described in RTP Payload Format for Theora Encoded Video [7], the "configuration", "configuration-uri", "delivery-method", and "sampling" parameters may be specified in relation to usage of the Theora [8] codec. Where such parameters are encoded via the "fmtp" SDP attribute, they shall be represented in Jingle via the following format, where the <parameter/> element is a child of the <payload-type/> element:

<parameter name='foo' value='bar'/>
  

Note: The parameter names are effectively guaranteed to be unique, since the Internet Assigned Numbers Authority (IANA) [9] maintains a registry of SDP parameters (see <http://www.iana.org/assignments/sdp-parameters>).

5. Negotiating a Jingle Video Session

When the initiator sends a session-initiate stanza to the receiver, the <description/> element includes all of the payload types that the initiator can receive for Jingle video (each one encapsulated in a separate <payload-type/> element):

Example 2. Initiation Example

<iq from='romeo@montague.net/orchard' 
    to='juliet@capulet.com/balcony' 
    id='jinglevideo1' 
    type='set'>
  <jingle xmlns='http://www.xmpp.org/extensions/xep-0166.html#ns'>
          action='session-initiate'
          initiator='romeo@montague.net/orchard'
          sid='v1d30k1ll3dth3r4d10st4r'>
    <content content='initiator' name='this-is-the-video-content' profile='RTP/AVP'>
      <description xmlns='http://www.xmpp.org/extensions/xep-0167.html#ns'>
        <payload-type id='96' name='theora' clockrate='90000' height='720' width='1280'>
          <parameter name='delivery-method' value='inline'/>
          <parameter name='configuration' value='somebase16string'/>
          <parameter name='sampling' value='YCbCr-4:2:2'/>
        </payload-type>
        <payload-type id='28' name='nv' clockrate='90000'/>
        <payload-type id='25' name='CelB' clockrate='90000'/>
        <payload-type id='32' name='MPV' clockrate='90000'/>
      </description>
      <transport xmlns='http://www.xmpp.org/extensions/xep-0176.html#ns-udp'/>
    </content>
  </jingle>
</iq>
  

Upon receiving the session-initiate stanza, the receiver determines whether it can provisionally accept the session and proceed with the negotiation. The general Jingle error cases are specified in XEP-0166. In addition, the receiver must determine if it supports any of the payload types advertised by the initiator; if it does not, it MUST reject the session by sending a <not-acceptable/> error, which SHOULD include a Jingle-specific error condition of <unsupported-codecs/>:

Example 3. Receiver Does Not Support Codecs

<iq type='error' 
    from='juliet@capulet.com/balcony' 
    to='romeo@montague.net/orchard' 
    id='jinglevideo1'>
  <error type='cancel'>
    <not-acceptable xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
    <unsupported-codecs xmlns='http://www.xmpp.org/extensions/xep-0180.html#ns-errors'/>
  </error>
</iq>
  

If there is no error, the receiver provisionally accepts the session:

Example 4. Receiver Provisionally Accepts Session

  <iq from='juliet@capulet.com/balcony' 
      to='romeo@montague.net/orchard' 
      id='jinglevideo1' 
      type='result' />
  

The receiver then should send a list of the payload types that it can receive via a Jingle "content-accept" (or "session-accept") action. The list that the receiver sends MAY include any payload types (not a subset of the payload types sent by the initiator) but SHOULD retain the ID numbers and order specified by the initiator.

Example 5. Receiver Accepts Content Type

<iq from='juliet@capulet.com/balcony' 
    to='romeo@montague.net/orchard' 
    id='jinglevideo2' 
    type='set'>
  <jingle xmlns='http://www.xmpp.org/extensions/xep-0166.html#ns'>
          action='content-accept'
          initiator='romeo@montague.net/orchard'
          sid='v1d30k1ll3dth3r4d10st4r'>
    <content content='initiator' name='this-is-the-video-content' profile='RTP/AVP'>
      <description xmlns='http://www.xmpp.org/extensions/xep-0167.html#ns'>
        <payload-type id='96' name='theora' height='720' width='1280'>
          <parameter name='delivery-method' value='inline'/>
          <parameter name='configuration' value='somebase16string'/>
          <parameter name='sampling' value='YCbCr-4:2:2'/>
        </payload-type>
        <payload-type id='32' name='MPV' clockrate='90000'/>
        <payload-type id='33' name='MP2T' clockrate='90000'/>
      </description> 
      <transport xmlns='http://www.xmpp.org/extensions/xep-0176.html#ns-udp'/>
    </content>
  </jingle>
</iq>
  

The initiator acknowledges the 'content-accept' with an empty IQ result:

Example 6. Initiator Acknowledges Modified Content Description

  <iq from='romeo@montegue.net/orchard' 
      to='juliet@capulet.com/balcony' 
      id='jinglevideo2' 
      type='result'/>
  

After successful transport negotiation (not shown here), the receiver then accepts the session:

Example 7. Receiver Definitively Accepts the Session

<iq type='set' 
    from='juliet@capulet.com/balcony' 
    to='romeo@montague.net/orchard' 
    id='accept1'>
  <jingle xmlns='http://www.xmpp.org/extensions/xep-0166.html#ns'
          action='session-accept'
          initiator='romeo@montague.net/orchard'
          responder='juliet@capulet.com/balcony'
          sid='v1d30k1ll3dth3r4d10st4r'>
    <content creator='initiator' name='this-is-the-video-content' profile='RTP/AVP'>
      <description xmlns='http://www.xmpp.org/extensions/xep-0167.html#ns'/>
      <transport xmlns='http://www.xmpp.org/extensions/xep-0176.html#ns-udp'>
        <candidate component='2'
                   foundation='1'
                   generation='0' 
                   ip='192.0.2.3' 
                   network='1'
                   port='45664'
                   priority='1107821052'
                   protocol='udp'
                   pwd='asd88fgpdd777uzjYhagZg'
                   type='srflx'
                   ufrag='8hhy'/>
      </transport>
    </content>
  </jingle>
</iq>
  

And the initiator acknowledges session acceptance:

Example 8. Initiator Acknowledges Session Acceptance

  <iq from='romeo@montague.net/orchard' 
      to='juliet@capulet.com/balcony' 
      id='accept1' 
      type='result' />
  

6. Mapping to Session Description Protocol

If the payload type is static (payload-type IDs 0 through 95 inclusive), it MUST be mapped to a media field defined in RFC 4566. The generic format for the media field is as follows:

m=<media> <port> <transport> <fmt list>
  

In the context of Jingle video sessions, the <media> is "video", the <port> is the preferred port for such communications (which may be determined dynamically), the <transport> is whatever transport method is negotiated via the Jingle negotiation (e.g., "RTP/AVP"), and the <fmt list> is the payload-type ID.

For example, consider the following static payload-type:

Example 9. Jingle Format for Static Payload-Type

<payload-type id="28" name="nv"/>
  

That Jingle-formatted information would be mapped to SDP as follows:

Example 10. SDP Mapping of Static Payload-Type

m=video 9000 RTP/AVP 28
  

If the payload type is dynamic (payload-type IDs 96 through 127 inclusive), it SHOULD be mapped to an SDP media field plus an SDP attribute field named "rtpmap".

For example, consider a VC-1 payload such as that described in RFC 4425 [10]:

Example 11. Jingle Format for Dynamic Payload-Type

<payload-type id='98' name='vc1' height='288' width='352'/>
  

That Jingle-formatted information would be mapped to SDP as follows:

Example 12. SDP Mapping of Dynamic Payload-Type

m=video 49170 RTP/AVP 98
a=rtpmap:98 vc1/90000
a=fmtp:98 width=352;height=288;
  

As noted, if additional parameters are to be specified, they shall be represented as attributes of the <payload-type/> element or its child <parameter/> element, as in the following example.

Example 13. Jingle Format for Dynamic Payload-Type With Parameters

<payload-type id='96' name='theora' height='720' width='1280'>
  <parameter name='delivery-method' value='inline'/>
  <parameter name='configuration' value='somebase16string'/>
  <parameter name='sampling' value='YCbCr-4:2:2'/>
</payload-type>
  

That Jingle-formatted information would be mapped to SDP as follows:

Example 14. SDP Mapping of Dynamic Payload-Type With Parameters

m=video 49170 RTP/AVP 98
a=rtpmap:96 theora/90000
a=fmtp:96 sampling=YCbCr-4:2:2; width=1280; height=720; 
delivery-method=inline; configuration=somebase16string;
  

7. Determining Support

If an entity supports Jingle video exchanges via RTP, it MUST advertise that fact by returning a feature of "http://www.xmpp.org/extensions/xep-0180.html#ns" in response to Service Discovery [11] information requests (see Protocol Namespaces regarding issuance of one or more permanent namespaces).

Example 15. Service Discovery Information Request

<iq from='romeo@montague.net/orchard' 
    id='disco1'
    to='juliet@capulet.com/balcony' 
    type='get'>
  <query xmlns='http://jabber.org/protocol/disco#info'/>
</iq>
  

Example 16. Service Discovery Information Response

<iq from='juliet@capulet.com/balcony' 
    id='disco1'
    to='romeo@montague.net/orchard' 
    type='result'>
  <query xmlns='http://jabber.org/protocol/disco#info'>
    ...
    <feature var='http://www.xmpp.org/extensions/xep-0166.html#ns'/>
    <feature var='http://www.xmpp.org/extensions/xep-0180.html#ns'/>
    ...
  </query>
</iq>
  

8. Informational Messages

Informational messages may be sent by either party within the context of Jingle to communicate the status of a Jingle video session, device, or principal. The informational message MUST be an IQ-set containing a <jingle/> element of type "session-info". No informational message payload elements have yet been defined, but they may be specified in a future version of this document.

9. Implementation Notes

9.1 Codecs

Support for the Theora codec is RECOMMENDED.

10. Security Considerations

In order to secure the data stream, implementations SHOULD use encryption methods appropriate to the transport method and media being exchanged; for example, in the case of UDP, that would include Datagram Transport Layer Security (DTLS) as specified in RFC 4347 [12]. RTP Over DTLS [13] defines such methods for the Session Description Protocol; the relevant RTP profile (e.g., "UDP/TLS/RTP/AVP" for transporting the RTP stream over DTLS with UDP) shall be specified as the value of the <content/> element's 'profile' attribute.

11. IANA Considerations

This document requires no interaction with the Internet Assigned Numbers Authority (IANA) [14].

12. XMPP Registrar Considerations

12.1 Protocol Namespaces

Until this specification advances to a status of Draft, its associated namespaces shall be "http://www.xmpp.org/extensions/xep-0180.html#ns" and "http://www.xmpp.org/extensions/xep-0180.html#ns-errors"; upon advancement of this specification, the XMPP Registrar [15] shall issue permanent namespaces in accordance with the process defined in Section 4 of XMPP Registrar Function [16].

12.2 Jingle Content Description Formats

The XMPP Registrar shall include "video-rtp" in its registry of Jingle content description formats. The registry submission is as follows:

<content>
  <name>video-rtp</name>
  <desc>Jingle sessions that support video exchange via the Real-time Transport Protocol</desc>
  <transport>lossy</transport>
  <doc>XEP-0180</doc>
</content>
    

13. XML Schemas

13.1 Content Description Format

<?xml version='1.0' encoding='UTF-8'?>

<xs:schema
    xmlns:xs='http://www.w3.org/2001/XMLSchema'
    targetNamespace='http://www.xmpp.org/extensions/xep-0180.html#ns'
    xmlns='http://www.xmpp.org/extensions/xep-0180.html#ns'
    elementFormDefault='qualified'>

  <xs:element name='description'>
    <xs:complexType>
      <xs:sequence>
        <xs:element ref='payload-type' minOccurs='0' maxOccurs='unbounded'/>
      </xs:sequence>
    </xs:complexType>
  </xs:element>

  <xs:element name='payload-type'>
    <xs:complexType>
      <xs:simpleContent>
        <xs:extension base='empty'>
          <xs:attribute name='channels' type='xs:integer' use='optional' default='1'/>
          <xs:attribute name='height' type='xs:nonNegativeInteger' use='optional'/>
          <xs:attribute name='id' type='xs:unsignedByte' use='required'/>
          <xs:attribute name='layer' type='xs:nonNegativeInteger' use='optional'/>
          <xs:attribute name='name' type='xs:string' use='optional'/>
          <xs:attribute name='rate' type='xs:short' use='optional'/>
          <xs:attribute name='transparent' type='xs:boolean' use='optional'/>
          <xs:attribute name='width' type='xs:nonNegativeInteger' use='optional'/>
          <xs:attribute name='x' type='xs:integer' use='optional'/>
          <xs:attribute name='y' type='xs:integer' use='optional'/>
        </xs:extension>
      </xs:simpleContent>
    </xs:complexType>
  </xs:element>

  <xs:simpleType name='empty'>
    <xs:restriction base='xs:string'>
      <xs:enumeration value=''/>
    </xs:restriction>
  </xs:simpleType>

</xs:schema>
    

13.2 Errors

<?xml version='1.0' encoding='UTF-8'?>

<xs:schema
    xmlns:xs='http://www.w3.org/2001/XMLSchema'
    targetNamespace='http://www.xmpp.org/extensions/xep-0180.html#ns-errors'
    xmlns='http://www.xmpp.org/extensions/xep-0180.html#ns-errors'
    elementFormDefault='qualified'>

  <xs:element name='unsupported-codecs' type='empty'/>

  <xs:simpleType name='empty'>
    <xs:restriction base='xs:string'>
      <xs:enumeration value=''/>
    </xs:restriction>
  </xs:simpleType>

</xs:schema>
    

Notes

1. XEP-0166: Jingle <http://www.xmpp.org/extensions/xep-0166.html>.

2. RFC 3550: RTP: A Transport Protocol for Real-Time Applications <http://tools.ietf.org/html/rfc3550>.

3. RFC 4566: SDP: Session Description Protocol <http://tools.ietf.org/html/rfc4566>.

4. XEP-0177: Jingle Raw UDP Transport Method <http://www.xmpp.org/extensions/xep-0177.html>.

5. XEP-0176: Jingle ICE Transport Method <http://www.xmpp.org/extensions/xep-0176.html>.

6. RFC 3551: RTP Profile for Audio and Video Conferences with Minimal Control <http://tools.ietf.org/html/rfc3551>.

7. RTP Payload Format for Theora Encoded Video <http://tools.ietf.org/html/draft-barbato-avt-rtp-theora>. Work in progress.

8. See <http://www.theora.org/>.

9. The Internet Assigned Numbers Authority (IANA) is the central coordinator for the assignment of unique parameter values for Internet protocols, such as port numbers and URI schemes. For further information, see <http://www.iana.org/>.

10. RFC 4425: RTP Payload Format for Video Codec 1 (VC-1) <http://tools.ietf.org/html/rfc4425>.

11. XEP-0030: Service Discovery <http://www.xmpp.org/extensions/xep-0030.html>.

12. RFC 4347: Datagram Transport Layer Security <http://tools.ietf.org/html/rfc4347>.

13. Real-Time Transport Protocol (RTP) over Datagram Transport Layer Security (DTLS) <http://tools.ietf.org/html/draft-fischl-mmusic-sdp-dtls>. Work in progress.

14. The Internet Assigned Numbers Authority (IANA) is the central coordinator for the assignment of unique parameter values for Internet protocols, such as port numbers and URI schemes. For further information, see <http://www.iana.org/>.

15. The XMPP Registrar maintains a list of reserved protocol namespaces as well as registries of parameters used in the context of XMPP extension protocols approved by the XMPP Standards Foundation. For further information, see <http://www.xmpp.org/registrar/>.

16. XEP-0053: XMPP Registrar Function <http://www.xmpp.org/extensions/xep-0053.html>.


Revision History

Version 0.8 (2007-05-23)

Corrected examples to use video codecs; added clockrate attribute.

(psa)

Version 0.7 (2007-05-23)

More completely specified how to include SDP parameters and codec-specific parameters (same approach as in XEP-0167); added and corrected Theora examples.

(psa)

Version 0.6 (2007-04-17)

Specified Jingle conformance, including the need to use a lossy transport and the process of sending and receiving video content.

(psa)

Version 0.5 (2007-03-23)

Added negotiation flow and SDP mapping; renamed to mention RTP as the associated transport; corrected negotiation flow to be consistent with SIP/SDP (each party specifies a list of the payload types it can receive); added profile attribute to content element in order to specify RTP profile in use.

(psa)

Version 0.4 (2006-12-21)

Modified spec to use provisional namespace before advancement to Draft (per XEP-0053).

(psa)

Version 0.3 (2006-08-23)

Modified namespace to track XEP-0166.

(psa)

Version 0.2 (2006-07-12)

Updated to use content type instead of media type.

(psa)

Version 0.1 (2006-03-23)

Initial version.

(psa/mc)

Version 0.0.1 (2006-03-20)

First draft.

(psa/mc)

END