XEP-0266: Codecs for Jingle RTP Sessions

Abstract:This document describes implementation considerations related to voice and video codecs for use in Jingle RTP sessions.
Author:Peter Saint-Andre
Copyright:© 1999 - 2011 XMPP Standards Foundation. SEE LEGAL NOTICES.
Status:Experimental
Type:Standards Track
Version:0.4
Last Updated:2011-06-09

WARNING: This Standards-Track document is Experimental. Publication as an XMPP Extension Protocol does not imply approval of this proposal by the XMPP Standards Foundation. Implementation of the protocol described herein is encouraged in exploratory implementations, but production systems are advised to carefully consider whether it is appropriate to deploy implementations of this protocol before it advances to a status of Draft.

Table of Contents


1. Introduction
2. Basic Considerations
3. Audio Codecs
    3.1. G.711
    3.2. Opus
    3.3. Speex
    3.4. Guidance for Implementers
    3.5. Mandatory-to-Implement Codec
4. Video
    4.1. Dirac
    4.2. H.264
    4.3. Theora
    4.4. Guidance for Implementers
    4.5. Mandatory-to-Implement Codec
5. Security Considerations
6. IANA Considerations
7. XMPP Registrar Considerations
8. Acknowledgements

Appendices
    A: Document Information
    B: Author Information
    C: Legal Notices
    D: Relation to XMPP
    E: Discussion Venue
    F: Requirements Conformance
    G: Notes
    H: Revision History

1. Introduction

Jingle RTP Sessions [1] defines the Jingle [2] signalling exchanges needed to establish voice and video chat using the Real-time Transport Protocol RFC 3550 [3]; however, it does not discuss the matter of voice and video codecs, since the state of codec technologies is more fluid than the signalling interactions. This document fills that gap by providing guidance to Jingle developers regarding voice and video codecs.

Because codec technologies are typically subject to patents, the topics discussed here are controversial. This document attempts to steer a middle path between (1) specifying mandatory-to-implement technologies that realistically will not be implemented and deployed and (2) providing guidelines that, while realistic, do not encourage the implementation and deployment of patent-clear technologies.

Please note that this document is strictly informational and does not (yet) provide binding recommendations to the XMPP developer community regarding mandatory-to-implement technologies; however, it is expected that this document will provide input that the XMPP Standards Foundation (XSF) [4] could use in making such recommendations. Furthermore, it is expected that any recommendations that might be made by the XSF would need to be modified over time as the technology landscape changes.

2. Basic Considerations

The ideal codec would meet the following criteria:

Quality
The encoding quality is acceptable for deployment among XMPP users.
Packetization
The specification of the codec clearly defines packetization of data for sending over RTP.
Availability
The codec can be implemented on a wide variety of computing platforms and is commonly used in Internet or other systems.
Patents
The codec is patent-clear. [5] (Although most XMPP developers would prefer to implement codecs that are patent-clear, such options are not always widely implemented and deployed.)

Unfortunately, not all codecs are ideal. In the remainder of this document we discuss the audio and video codecs that are most appropriate for implementation in Jingle RTP applications.

3. Audio Codecs

3.1 G.711

G.711 refers to the Pulse Code Modulation (PCM) codec defined in International Telecommunication Union (ITU) [6] recommendation G.711, which is widely used on the public switched telephone network (PSTN) and by many voice over Internet Protocol (VoIP) providers. There are two versions: the μ-law ("U-law") version is widely deployed in North America and in Japan, whereas the A-law version is widely deployed in the rest of the world. The following table summarizes the available information about G.711.

Table 1: Codec Considerations for G.711

Quality Packetization Availability Patents
Good quality; no wide-band mode. See RFC 5391 [7]. Commonly deployed in both PSTN and VoIP systems. Developed in 1972; patents have expired.

3.2 Opus

The Opus codec is under development within the IETF's Codec Working Group. In essence it combines the best features of CELT (developed by Jean-Marc Valin, the creator of Speex) and SILK (created by and widely used in the Skype service). The following table summarizes the available information about Opus.

Table 2: Codec Considerations for Opus

Quality Packetization Availability Patents
Extremely high quality; can be used for wide-band audio. Not yet defined. Covered under IETF IPR rules, the intent is for the codec to be covered under a simplified BSD license. See http://tools.ietf.org/html/draft-ietf-codec-opus for details. Not commonly deployed yet, but the SILK codec on which it is partly based is very widely deployed. Designed to be patent-clear, but IPR claims have been filed.

3.3 Speex

According to the speex.org website, the Speex codec is "an Open Source/Free Software patent-free audio compression format designed for speech". Speex was developed by Jean-Marc Valin and is maintained by the Xiph.org Foundation. The following table summarizes the available information about Speex.

Table 3: Codec Considerations for Speex

Quality Packetization Availability Patents
Good quality; optimized for voice; can be used for wide-band audio. See RFC 5574 [8]. Freely downloadable under a revised BSD license at <http://speex.org/> and commonly deployed on Internet (VoIP) systems; not commonly deployed on non-Internet systems. Designed to be patent-clear.

3.4 Guidance for Implementers

Given that both Speex and G.711 are patent-clear, freely implementable, and commonly deployed, implementers are encouraged to consider including support for both codecs in audio applications of Jingle RTP sessions. Discussion on the jingle@xmpp.org mailing list indicates a slight preference for G.711 because it is so widely deployed and easily available. The Opus codec is not yet widely deployed (or even fully developed), but it might become the "codec of the future" for audio applications over the Internet.

3.5 Mandatory-to-Implement Codec

As of June 2011, this document makes the following recommendations:

  1. Jingle clients MUST implement both PCMU and PCMA.
  2. Gateways between Jingle networks and other networks (e.g., SIP networks and the PSTN) MUST implement either PCMA or PCMA.

Naturally, clients and gateways can implement additional codecs, such as those listed in this document.

4. Video

4.1 Dirac

Dirac is a general-purpose video compression technology developed by the BBC that has been licensed in the open. It is used for everything from Internet streaming to HDTV. To date there is no RTP packetization deveintion for Dirac; however, such a format is under development.

Table 4: Codec Considerations for Dirac

Quality Packetization Availability Patents
High quality. Not yet defined. Freely downloadable under both GPL and LGPL at <http://diracvideo.org/>; commonly deployed but not yet in video over IP systems because of the lack of an RTP packetization format. Diract is patent-clear, and the BBC has allowed its related patents to lapse.

4.2 H.264

H.264 is a technology for video compression jointly designed by the ITU and the International Organization for Standardization (ISO) [9]. The following table summarizes the available information about H.264.

Table 5: Codec Considerations for H.264

Quality Packetization Availability Patents
High quality. See RFC 3984 [10]. Commonly deployed in commercial video systems. Not freely downloadable; both software implementations and service deployments can be subject to royalty payments for commercial use. Patented.

4.3 Theora

According to the theora.org website, the Theora codec is "a free and open video compression format". Theora is based on the VP3 codec originally developed by On2 Technologies and is now maintained by the Xiph.org Foundation. The following table summarizes the available information about Theora.

Table 6: Codec Considerations for Theora

Quality Packetization Availability Patents
Acceptable quality. See RTP Payload Format for Theora Encoded Video [11]. Freely downloadable under BSD license at <http://theora.org/>; not yet commonly deployed, especially on devices that have deployed H.264 instead. On2's patents over VP3 were contributed to the Xiph.org Foundation in 2001.

4.4 Guidance for Implementers

The situation regarding video codecs is more murky, and implementers face difficult tradeoffs. Although Theora is patent-clear and freely implementable, it is not yet commonly deployed. Although Dirac is patent-clear and deployed fairly widely, no RTP packetization format has been defined for it. Although deployment of H.264 is fairly common, it is not patent-clear or freely implementable. For many open-source / free software projects and smaller technology vendors, implementation of H.264 is either impossible (because of patents and licensing restrictions) or prohibitively expensive (because of royalty payments). These developers are strongly encouraged to implement Theora or Dirac and also to urge wider adoption of Theora and Dirac among larger technology vendors. However, this document acknowledges that it may take some time before Theora and Dirac are commonly deployed (especially on mobile devices) and that systems based on H.264 might be dominant in the marketplace for several years. This situation is unfortunate but cannot be directly changed by the XMPP developer community.

4.5 Mandatory-to-Implement Codec

Because video codecs are not as mature as audio codecs, it is not yet possible for the XSF to recommend a mandatory-to-implement technology for video codecs. However, it might be possible to recommend Theora or Dirac in the future if they are more widely adopted.

5. Security Considerations

For security considerations related to Jingle RTP sessions, refer to XEP-0167. This document introduces no new security considerations. See also the security considerations described in the relevant codec specifications.

6. IANA Considerations

This document requires no interaction with the Internet Assigned Numbers Authority (IANA) [12].

7. XMPP Registrar Considerations

This document requires no interaction with the XMPP Registrar [13].

8. Acknowledgements

Thanks to Olivier Crête, Dave Cridland, Florian Jensen, Justin Karneges, Evgeniy Khramtsov, Marcus Lundblad, Tobias Markmann, Pedro Melo, Jack Moffitt, Jeff Muller, Jehan Pagès, Arc Riley, Kevin Smith, Remko Tronçon, Justin Uberti, and Paul Witty for their feedback.

Appendices

Appendix A: Document Information

Series: XEP
Number: 0266
Publisher: XMPP Standards Foundation
Status: Experimental
Type: Standards Track
Version: 0.4
Last Updated: 2011-06-09
Approving Body: XMPP Council
Dependencies: XMPP Core, XEP-0167
Supersedes: None
Superseded By: None
Short Name: N/A
Source Control: HTML  RSS
This document in other formats: XML  PDF

Appendix B: Author Information

Peter Saint-Andre

Email: stpeter@jabber.org
JabberID: stpeter@jabber.org
URI: https://stpeter.im/

Appendix C: Legal Notices

Copyright

This XMPP Extension Protocol is copyright © 1999 - 2011 by the XMPP Standards Foundation (XSF).

Permissions

Permission is hereby granted, free of charge, to any person obtaining a copy of this specification (the "Specification"), to make use of the Specification without restriction, including without limitation the rights to implement the Specification in a software program, deploy the Specification in a network service, and copy, modify, merge, publish, translate, distribute, sublicense, or sell copies of the Specification, and to permit persons to whom the Specification is furnished to do so, subject to the condition that the foregoing copyright notice and this permission notice shall be included in all copies or substantial portions of the Specification. Unless separate permission is granted, modified works that are redistributed shall not contain misleading information regarding the authors, title, number, or publisher of the Specification, and shall not claim endorsement of the modified works by the authors, any organization or project to which the authors belong, or the XMPP Standards Foundation.

Disclaimer of Warranty

## NOTE WELL: This Specification is provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. ##

Limitation of Liability

In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall the XMPP Standards Foundation or any author of this Specification be liable for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising from, out of, or in connection with the Specification or the implementation, deployment, or other use of the Specification (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if the XMPP Standards Foundation or such author has been advised of the possibility of such damages.

IPR Conformance

This XMPP Extension Protocol has been contributed in full conformance with the XSF's Intellectual Property Rights Policy (a copy of which can be found at <http://xmpp.org/extensions/ipr-policy.shtml> or obtained by writing to XMPP Standards Foundation, 1899 Wynkoop Street, Suite 600, Denver, CO 80202 USA).

Appendix D: Relation to XMPP

The Extensible Messaging and Presence Protocol (XMPP) is defined in the XMPP Core (RFC 3920) and XMPP IM (RFC 3921) specifications contributed by the XMPP Standards Foundation to the Internet Standards Process, which is managed by the Internet Engineering Task Force in accordance with RFC 2026. Any protocol defined in this document has been developed outside the Internet Standards Process and is to be understood as an extension to XMPP rather than as an evolution, development, or modification of XMPP itself.

Appendix E: Discussion Venue

There exists a special venue for discussion related to the technology described in this document: the <jingle@xmpp.org> mailing list.

The primary venue for discussion of XMPP Extension Protocols is the <standards@xmpp.org> discussion list.

Discussion on other xmpp.org discussion lists might also be appropriate; see <http://xmpp.org/about/discuss.shtml> for a complete list.

Errata can be sent to <editor@xmpp.org>.

Appendix F: Requirements Conformance

The following requirements keywords as used in this document are to be interpreted as described in RFC 2119: "MUST", "SHALL", "REQUIRED"; "MUST NOT", "SHALL NOT"; "SHOULD", "RECOMMENDED"; "SHOULD NOT", "NOT RECOMMENDED"; "MAY", "OPTIONAL".

Appendix G: Notes

1. XEP-0167: Jingle RTP Sessions <http://xmpp.org/extensions/xep-0167.html>.

2. XEP-0166: Jingle <http://xmpp.org/extensions/xep-0166.html>.

3. RFC 3550: RTP: A Transport Protocol for Real-Time Applications <http://tools.ietf.org/html/rfc3550>.

4. The XMPP Standards Foundation (XSF) is an independent, non-profit membership organization that develops open extensions to the IETF's Extensible Messaging and Presence Protocol (XMPP). For further information, see <http://xmpp.org/xsf/>.

5. The term patent-clear does not necessarily mean that no patents have ever been applied for or granted regarding a technology, or that the technology is completely free from patents (since such a judgment is nearly impossible to make, and is outside the purview of the XMPP developer community and the XMPP Standards Foundation); the term means only that those who implement the technology are generally understood to be relatively safe from the threat of patent litigation, either because any relevant patents have expired, were filed in a defensive manner, or are made available under suitable royalty-free licenses.

6. The International Telecommunication Union develops technical and operating standards (such as H.323) for international telecommunication services. For further information, see <http://www.itu.int/>.

7. RFC 5391: RTP Payload Format for ITU-T Recommendation G.711.1 <http://tools.ietf.org/html/rfc5391>.

8. RFC 5574: RTP Payload Format for the Speex Codec <http://tools.ietf.org/html/rfc5574>.

9. The International Organization for Standardization develops standards a wide variety of technical domains. For further information, see <http://www.iso.org/>.

10. RFC 3984: RTP Payload Format for H.264 Video <http://tools.ietf.org/html/rfc3984>.

11. RTP Payload Format for Theora Encoded Video <http://tools.ietf.org/html/draft-barbato-avt-rtp-theora>. Work in progress.

12. The Internet Assigned Numbers Authority (IANA) is the central coordinator for the assignment of unique parameter values for Internet protocols, such as port numbers and URI schemes. For further information, see <http://www.iana.org/>.

13. The XMPP Registrar maintains a list of reserved protocol namespaces as well as registries of parameters used in the context of XMPP extension protocols approved by the XMPP Standards Foundation. For further information, see <http://xmpp.org/registrar/>.

Appendix H: Revision History

Note: Older versions of this specification might be available at http://xmpp.org/extensions/attic/

Version 0.4 (2011-06-09)

Recommended G.711 as mandatory-to-implement for audio.

(psa)

Version 0.3 (2011-01-12)

Added information about the Opus audio codec.

(psa)

Version 0.2 (2009-04-23)

Added information about the Dirac video codec.

(psa)

Version 0.1 (2009-04-08)

Initial published version.

(psa)

Version 0.0.4 (2009-04-04)

Clarified status of H.264.

(psa)

Version 0.0.3 (2009-04-02)

Rewrote document based on developer feedback and Council discussion.

(psa)

Version 0.0.2 (2009-03-04)

Added more information about video codecs.

(psa)

Version 0.0.1 (2009-03-04)

First draft, copied from XEP-0167 with slight revisions and addition of requirements section.

(psa)

END