First draft.
Audio/Video calls are possible with a single destination via Jingle (XEP-0166)
A mesh network where each peer contacts each other peer: this is straightforward to implement and doesn't require any extra service, but it is heavy on network bandwidth and doesn't scale.
An MCU (Multipoint Conferencing Unit) which uses a central service that generally mixes the contents of participants: this is lighter on network bandwidth but heavy on computing resources and hard to scale.
An SFU (Selective Forwarding Unit) which uses a service that redistributes streams of participants as efficiently as possible: this is lighter on computing resources, acceptable in terms of network bandwidth, and can scale.
The mesh strategy is covered by Multiparty Jingle (XEP-0272)
This specification proposes a simple way to implement an SFU strategy with a service called in a similar way to one-to-one calls.
The design goals of this XEP are:
A/V conferences work by using a single Jingle session per participant. To join a conference, a client calls the JID of a conference room and sends its own streams. The conference service then delivers the streams of the other participants by adding and removing the corresponding contents of the session.
An <origin> element is used to associate a forwarded stream with the originating entity (see Origin of Forwarded Streams).
Ad-Hoc Commands (XEP-0050)
To join a conference, a user calls the conference room JID using Jingle RTP Sessions (XEP-0167)
A joining entity MAY send the same audio or video stream with different quality for simulcast purposes. It is up to the SFU service to then select which one to forward.
A participant leaves the conference by terminating the session.
The SFU service selects and delivers the streams of other participants by sending a content-add action on the participant's session, including one content per stream (e.g. one audio and one video content). The added contents MUST have their 'senders' attribute set to "responder". The action MUST include an <origin> element identifying the entity the streams originate from (see Origin of Forwarded Streams).
The contents of the streams of the same origin MUST share a single transport, declared with a BUNDLE group, as defined in Jingle Grouping Framework (XEP-0338)
The service MAY send content-add actions while the session is still in the PENDING state (as allowed by Jingle (XEP-0166)
The joining entity SHOULD content-accept each added content. If it does not want to receive the corresponding stream, it MAY content-reject it.
When a stream has to stop being delivered to a participant (e.g. because its origin left the conference, or because the service changed its selection), the service MUST remove the corresponding contents with a content-remove action.
A participant MAY add additional streams of its own to the session, e.g. to share its screen, by sending a content-add action with the corresponding contents (one content per stream, their 'senders' attribute set to "initiator"), as for its initial streams (see Joining a Conference). The participant MAY use a new BUNDLE group for the added streams, or add them to an existing BUNDLE group (see Receiving Streams). The service MAY then deliver the added streams to the other participants, with the participant as origin.
An entity MAY have several concurrent streams in a conference (e.g. streams from its camera and from its screen). Each stream is carried by a single content. How the streams of an entity are rendered (e.g. one tile per source, or combined) is up to the implementation.
Each content-add action carrying streams forwarded by the SFU service MUST include an <origin> element, qualified by the 'urn:xmpp:jingle:av-conferences:0' namespace, as a child of the <jingle> element (i.e. a sibling of the <content> elements), as shown below. Its 'entity' attribute identifies the entity the streams originate from, as a URI (typically an "xmpp:" URL, in the form "xmpp:user@domain" or "xmpp:user@domain/resource").
A content-add action carries the streams of a single origin, so a single <origin> element identifies the origin of all the contents of the action.
A stream MAY originate from a non-XMPP entity (e.g. using the SFU's own user interface or with a gateway to another protocol). In that case, the 'entity' attribute MAY use another URI scheme.
It may be desirable to have configuration options on the SFU service or on one of its rooms. For instance, a conference room may offer the option to record the session. While the definition of those options is beyond the scope of this specification, the configuration MUST be done via Ad-Hoc Commands (XEP-0050)
Only other peers' streams are sent to a joining entity. An SFU service MUST NOT send back a stream to the entity that initially sent it.
An SFU service MUST NOT initiate independent Jingle sessions to deliver streams to a joined entity: the streams are only delivered as contents of the entity's session, via content-add actions.
An SFU service MUST ensure that the names of the contents it adds do not collide with the existing contents of the session, as a BUNDLE group identifies its contents by name only.
If a client or a service supports the protocol specified in this XEP, it MUST advertise it by including the 'urn:xmpp:jingle:av-conferences:0' discovery feature in response to a Service Discovery (XEP-0030)
If an entity is an A/V conferences service as specified by this XEP, it MUST have an identity with a category of "conference" and a value of "audio-video" in a response to a Service Discovery (XEP-0030)
Streams are sent to the SFU service for redistribution. Although it is technically possible to have the stream encrypted end-to-end, this specification assumes that streams are decoded at the SFU level and are therefore not end-to-end encrypted, unlike one-to-one calls. Future specifications may outline a method for maintaining end-to-end encryption, but this is beyond the scope of this XEP; developers and end-users MUST assume that streams are decrypted at the SFU service level.
The general security considerations for one-to-one calls apply, with the added risk that more parties are participating in the call. However, the IP address of a device sending a stream is only known by the SFU service, not by other peers, unlike in one-to-one or Multiparty Jingle (XEP-0272)
The <origin> element is not authenticated: it is provided by the SFU service, which is responsible for having established the identity of the entities that sent the streams (e.g. via the JID of their Jingle sessions). The service could however alter an <origin> or a stream, either intentionally or through a bug.
TODO
TODO
TODO
Thanks to NLNet foundation/NGI Assure for funding the work on this specification.