Codecs for Jingle Audio

Codecs for Jingle Audio This document describes implementation considerations related to audio codecs for use in Jingle RTP sessions. This XMPP Extension Protocol is copyright © 1999 – 2024 by the XMPP Standards Foundation (XSF). Permission is hereby granted, free of charge, to any person obtaining a copy of this specification (the "Specification"), to make use of the Specification without restriction, including without limitation the rights to implement the Specification in a software program, deploy the Specification in a network service, and copy, modify, merge, publish, translate, distribute, sublicense, or sell copies of the Specification, and to permit persons to whom the Specification is furnished to do so, subject to the condition that the foregoing copyright notice and this permission notice shall be included in all copies or substantial portions of the Specification. Unless separate permission is granted, modified works that are redistributed shall not contain misleading information regarding the authors, title, number, or publisher of the Specification, and shall not claim endorsement of the modified works by the authors, any organization or project to which the authors belong, or the XMPP Standards Foundation. ## NOTE WELL: This Specification is provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. ## In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall the XMPP Standards Foundation or any author of this Specification be liable for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising from, out of, or in connection with the Specification or the implementation, deployment, or other use of the Specification (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if the XMPP Standards Foundation or such author has been advised of the possibility of such damages. This XMPP Extension Protocol has been contributed in full conformance with the XSF's Intellectual Property Rights Policy (a copy of which can be found at <https://xmpp.org/about/xsf/ipr-policy> or obtained by writing to XMPP Standards Foundation, P.O. Box 787, Parker, CO 80134 USA). 0266 Draft Standards Track Standards Council XMPP Core XEP-0167 N/A jingle Peter Saint-Andre stpeter@stpeter.im stpeter@jabber.org https://stpeter.im/ 1.1.0-rc.1 2013-03-01 psa

Updated to reflect standardization of the Opus codec; changed client conformance to also recommend (but not require) support for Opus.

1.0 2011-10-04 psa

Per a vote of the XMPP Council, advanced specification from Experimental to Draft.

0.6 2011-06-27 psa

Clarified that the codec descriptions are non-normative.

0.5 2011-06-12 psa

Moved video codecs to XEP-0299.

0.4 2011-06-09 psa

Recommended G.711 as mandatory-to-implement for audio.

0.3 2011-01-12 psa

Added information about the Opus audio codec.

0.2 2009-04-23 psa

Added information about the Dirac video codec.

0.1 2009-04-08 psa

Initial published version.

0.0.4 2009-04-04 psa

Clarified status of H.264.

0.0.3 2009-04-02 psa

Rewrote document based on developer feedback and Council discussion.

0.0.2 2009-03-04 psa

Added more information about video codecs.

0.0.1 2009-03-04 psa

First draft, copied from XEP-0167 with slight revisions and addition of requirements section.

Jingle RTP Sessions (XEP-0167) XEP-0167: Jingle RTP Sessions <https://xmpp.org/extensions/xep-0167.html>. defines the Jingle (XEP-0166) XEP-0166: Jingle <https://xmpp.org/extensions/xep-0166.html>. signalling exchanges needed to establish voice chat and other audio sessions using the Real-time Transport Protocol RFC 3550 RFC 3550: RTP: A Transport Protocol for Real-Time Applications <http://tools.ietf.org/html/rfc3550>.; however, it does not specify which audio codecs are mandatory-to-implement, since the state of codec technologies is more fluid than the signalling interactions. This document fills that gap by providing guidance to Jingle developers regarding audio codecs.

Because codec technologies are typically subject to patents, the topics discussed here are controversial. This document attempts to steer a middle path between (1) specifying mandatory-to-implement technologies that realistically will not be implemented and deployed and (2) providing guidelines that, while realistic, do not encourage the implementation and deployment of patent-clear technologies.

The ideal audio codec would meet the following criteria:

Quality: The encoding quality is acceptable for deployment among XMPP users.
Packetization: The specification of the codec clearly defines packetization of data for sending over RTP.
Availability: The codec can be implemented on a wide variety of computing platforms and is commonly used in Internet or other systems.
Patents: The codec is patent-clear. The term patent-clear does not necessarily mean that no patents have ever been applied for or granted regarding a technology, or that the technology is completely free from patents (since such a judgment is nearly impossible to make, and is outside the purview of the XMPP developer community and the XMPP Standards Foundation); the term means only that those who implement the technology are generally understood to be relatively safe from the threat of patent litigation, either because any relevant patents have expired, were filed in a defensive manner, or are made available under suitable royalty-free licenses. (Although most XMPP developers would prefer to implement codecs that are patent-clear, such options are not always widely implemented and deployed.)

Unfortunately, not all codecs meet those criteria. In the remainder of this document we discuss the audio codecs that are most appropriate for implementation in Jingle RTP applications.

This section is non-normative. Future versions of this specification might provide information about additional codecs not listed here.

G.711 refers to the Pulse Code Modulation (PCM) codec defined in International Telecommunication Union (ITU) The International Telecommunication Union develops technical and operating standards (such as H.323) for international telecommunication services. For further information, see <http://www.itu.int/>. recommendation G.711, which is widely used on the public switched telephone network (PSTN) and by many voice over Internet Protocol (VoIP) providers. There are two versions: the μ-law ("U-law") version is widely deployed in North America and in Japan, whereas the A-law version is widely deployed in the rest of the world. The following table summarizes the available information about G.711.

Quality	Packetization	Availability	Patents
Good quality; no wide-band mode.	See RFC 5391 RFC 5391: RTP Payload Format for ITU-T Recommendation G.711.1 <http://tools.ietf.org/html/rfc5391>..	Commonly deployed in both PSTN and VoIP systems.	Developed in 1972; patents have expired.

The Opus codec was developed within the IETF's Codec Working Group and has been published as RFC 6716 RFC 6716: Definition of the Opus Audio Codec <http://tools.ietf.org/html/rfc6716>.. In essence it combines the best features of CELT (developed by Jean-Marc Valin, the creator of Speex) and SILK (created by and widely used in the Skype service). The following table summarizes the available information about Opus.

Quality	Packetization	Availability	Patents
Extremely high quality; can be used for wide-band audio; very robust in the face of packet loss.	See RTP Payload Format and File Storage Format for Opus Speech and Audio Codec RTP Payload Format and File Storage Format for Opus Speech and Audio Codec <http://tools.ietf.org/html/draft-ietf-payload-rtp-opus>. Work in progress..	In accordance with IETF IPR rules, the codec is covered under a simplified BSD license. See RFC 6716 for details. Starting to be more commonly deployed, and the SILK codec on which it is partly based is very widely deployed.	Designed to be patent-clear, although IPR claims have been filed.

According to the speex.org website, the Speex codec is "an Open Source/Free Software patent-free audio compression format designed for speech". Speex was developed by Jean-Marc Valin and is maintained by the Xiph.org Foundation. The following table summarizes the available information about Speex.

Quality	Packetization	Availability	Patents
Good quality; optimized for voice; can be used for wide-band audio.	See RFC 5574 RFC 5574: RTP Payload Format for the Speex Codec <http://tools.ietf.org/html/rfc5574>..	Freely downloadable under a revised BSD license at <http://speex.org/> and commonly deployed on Internet (VoIP) systems; not commonly deployed on non-Internet systems.	Designed to be patent-clear.

This section is non-normative.

Given that Opus and G.711 are patent-clear, freely implementable, and commonly deployed, implementers are encouraged to consider including support for both codecs in audio applications of Jingle RTP sessions. Discussion on the jingle@xmpp.org mailing list indicates a slight preference for G.711 because it is easily available and so widely deployed (e.g., in SIP networks and the PSTN). Opus has effectively superseded Speex, and implementers are strongly encouraged to include support for Opus rather than Speex among the "open" codecs.

As of January 2013, this document makes the following recommendations:

Jingle clients MUST implement G.711 (i.e., both PCMU and PCMA) and SHOULD implement Opus.
Gateways between Jingle networks and other networks (e.g., SIP networks and the PSTN) MUST implement either PCMU or PCMA (and preferably both).

Naturally, clients and gateways can implement additional codecs, such as those listed in this document.

For security considerations related to Jingle RTP sessions, refer to Jingle RTP Sessions (XEP-0167) XEP-0167: Jingle RTP Sessions <https://xmpp.org/extensions/xep-0167.html>.. This document introduces no new security considerations. See also the security considerations described in the relevant codec specifications.

This document requires no interaction with the Internet Assigned Numbers Authority (IANA) The Internet Assigned Numbers Authority (IANA) is the central coordinator for the assignment of unique parameter values for Internet protocols, such as port numbers and URI schemes. For further information, see <http://www.iana.org/>..

This document requires no interaction with the XMPP Registrar The XMPP Registrar maintains a list of reserved protocol namespaces as well as registries of parameters used in the context of XMPP extension protocols approved by the XMPP Standards Foundation. For further information, see <https://xmpp.org/registrar/>..

Thanks to Olivier Crête, Dave Cridland, Florian Jensen, Justin Karneges, Evgeniy Khramtsov, Marcus Lundblad, Tobias Markmann, Jean-Marc Valin, Pedro Melo, Jack Moffitt, Jeff Muller, Jehan Pagès, Arc Riley, Kevin Smith, Remko Tronçon, Justin Uberti, and Paul Witty for their feedback.