XEP-0237: Roster Versioning

Abstract:This specification defines a proposed modification to the XMPP roster protocol that enables versioning of rosters such that the server will not send the roster to the client if the roster has not changed, thus saving bandwidth during session establishment.
Author:Peter Saint-Andre
Copyright:© 1999 - 2009 XMPP Standards Foundation. SEE LEGAL NOTICES.
Status:Proposed
Type:Standards Track
Version:0.9
Last Updated:2009-04-22

NOTICE: This document is currently within Last Call or under consideration by the XMPP Council for advancement to the next stage in the XSF standards process.

Table of Contents


1. Introduction
2. Protocol
    2.1. Data Format
    2.2. Client Request
    2.3. Server Response
3. Examples
4. Stream Feature
5. Security Considerations
6. IANA Considerations
7. XMPP Registrar Considerations
    7.1. Protocol Namespaces
8. XML Schemas
    8.1. jabber:iq:roster
    8.2. Stream Feature
9. Acknowledgements

Appendices
    A: Document Information
    B: Author Information
    C: Legal Notices
    D: Relation to XMPP
    E: Discussion Venue
    F: Requirements Conformance
    G: Notes
    H: Revision History

1. Introduction

Although XMPP rosters can become quite large, they change infrequently. Therefore it can be inefficient for the server to send the roster to the client during session establishment if the roster has not changed. This document defines a small modification to the XMPP roster protocol specified in XMPP IM [1] that enables "versioning" of roster information.

The basic model is that if the client specifies a version number when it requests the roster, the server returns an empty IQ-result. If the roster has changed, the server sends numbered roster pushes for each roster item that has been touched in any way since the version specified by the client. The client processes each roster push as it normally would, incrementing its local version number with each roster push. Therefore the client can receive only the items that have been modified, not the entire roster.

Note: This document describes a protocol or best practice that is intended for incorporation into the specification that will supersede RFC 3921 [2] within the Internet Standards Process, i.e., rfc3921bis [3]. This document is provided only for the purpose of open community discussion of the potential modification and will be obsoleted as soon as the relevant RFC is published.

2. Protocol

2.1 Data Format

This document adds a new 'ver' attribute to the <query/> element qualified by the 'jabber:iq:roster' namespace, defined as follows.

Definition: The 'ver' attribute is a strictly increasing sequence number that is increased (but not necessarily incremented-by-one) with any modification to the roster data. The value of the attribute MUST be a non-negative 64-bit integer, MUST be changed only by the server, and MUST be treated by the client as opaque. The server MUST ensure that each change to the roster data will result in a different sequence number and that the sequence number associated with a given roster modification will be greater than the sequence number associated with any previous roster modification for this session.

An XMPP server cannot guarantee that a timestamp will be strictly increasing (e.g., the system time might change because of an adjustment based on an update triggered by the user of the Network Time Protocol (RFC 958 [4]); therefore, because the 'ver' attribute is defined as a strictly increasing sequence number, server implementations are advised to employ a method other than timestamps for generation of the 'ver' attribute.

Definition: A "roster modification" is any change to the roster data that would result in a roster push to a connected client. Therefore internal states related to roster processing within the server that would not result in a roster push do not necessitate a increase in the sequence number. If a series of roster modifications result in a roster item that does not differ from the version cached by the client (e.g., a change to the item's 'name' attribute and then a change back to the original value), the server MUST consider the item to have been modified and therefore MUST send the item to the client (typically via a roster push as described below).

2.2 Client Request

If a client supports roster versioning, it SHOULD include the 'ver' element in its request for the roster, where the 'ver' attribute is set to the sequence number associated with its last cache of the roster.

Example 1. Roster get with sequence number

C: <iq from='romeo@montague.lit/home' id='r1h3vzp7' to='romeo@montague.lit' type='get'>
     <query xmlns='jabber:iq:roster' ver='299'/>
   </iq>

If the client has not yet cached the roster or the cache is lost or corrupted, but the client wishes to bootstrap the use of roster versioning, it MUST set the 'ver' attribute to a value of zero (0).

Naturally, if the client does not support roster versioning or does not wish to bootstrap the use of roster versioning, it will behave like an RFC-3921-compliant client by not including the 'ver' attribute.

2.3 Server Response

Whether or not the roster has changed since the version enumerated by the client, the server MUST either return the complete roster as described in RFC 3921 or return an empty IQ-result (thus indicating that any roster modifications will be sent via roster pushes, as described below). In general, unless returning the complete roster would use less bandwidth than sending individual roster pushes to the client (e.g., if the roster contains only a few items), the server SHOULD send an empty IQ-result and then send the modifications (if any) via roster pushes. In addition, if the client signals a sequence number that is greater than the sequence number currently on file at the server for that JID, the server MUST return the whole current roster as if client announced its version to be zero, thus bootstrapping the client's local cache.

Example 2. Empty roster result

S: <iq from='romeo@montague.lit' id='r1h3vzp7' to='romeo@montague.lit/home' type='result'/>

Note: This empty IQ-result is different from an empty <query/>, thus disambiguating this usage from an empty roster.

If the roster has not changed since the version enumerated by the client, the server will simply not send any roster pushes to the client (until and unless some relevant event triggers a roster push during the lifetime of the client's session).

If the roster has changed since the version enumerated by the client, the server MUST then send one roster push to the client for each roster item that has been modified since the version enumerated by the client.

Example 3. Roster pushes

S: <iq from='romeo@montague.lit' id='ah382g67' to='romeo@montague.lit/home' type='set'>
     <query xmlns='jabber:iq:roster' ver='303'>
       <item jid='shylock@shakespeare.lit' subscription='remove'/>
     </query>
   </iq>

S: <iq from='romeo@montague.lit' id='b2gs90j5' to='romeo@montague.lit/home' type='set'>
     <query xmlns='jabber:iq:roster' ver='307'>
       <item jid='bill@shakespeare.lit' subscription='both'/>
     </query>
   </iq>

S: <iq from='romeo@montague.lit' id='c73gs419' to='romeo@montague.lit/home' type='set'>
     <query xmlns='jabber:iq:roster' ver='311'>
       <item jid='nurse@shakespeare.lit' name='Nurse' subscription='to'>
         <group>Servants</group>
       </item>
     </query>
   </iq>

S: <iq from='romeo@montague.lit' id='dh361f35' to='romeo@montague.lit/home' type='set'>
     <query xmlns='jabber:iq:roster' ver='315'>
       <item jid='juliet@shakespeare.lit' name='Juliet' subscription='both'>
         <group>Friends</group>
       </item>
     </query>
   </iq>

These "interim roster pushes" can be understood as follows:

  1. Imagine that the client had an active presence session for the entire time between its cached roster version (say, 299) and the new roster version (say, 315).
  2. During that time, the client might have received roster pushes related to roster version numbers 301, 303, 305, 307, 309, 311, 313, and 315 (the version numbers must be strictly increasing but there is no requirement that the sequence shall be continuous).
  3. However, some of those roster pushes might have contained intermediate updates to the same roster item (e.g., changes in the subscription state for bill@shakespeare.lit from "none" to "to" and from "to" to "both").
  4. The interim roster pushes would not include all of the intermediate steps, only the final result of all changes applied while the client was in fact offline (say, 303, 307, 311, and 315).

The client MUST handle an "interim roster push" in the same way it handles any roster push (indeed, from the client's perspective it cannot tell the difference between an "interim" roster push and a "live" roster push). If the client's session ends before it receives all of the interim roster pushes, when requesting the roster after reconnection it SHOULD request the version associated with the last roster push it received during the session that was disconnected, not the version associated with the roster result it received at the start of the session that was disconnected.

When roster versioning is enabled, the server MUST include the updated roster sequence number with each roster push. Roster pushes MUST occur in sequence order and the sequence number contained in a roster push MUST be unique.

3. Examples

This section provides a detailed scenario that illustrates the use of roster versioning. In this example the client gets disconnected before the server has had a chance to send all of its roster pushes, but this is immaterial to the synchronization process.

Example 4. The roster synchronization process

C: <iq from='romeo@montague.lit/home' id='r1h3vzp7' to='romeo@montague.lit' type='get'>
     <query xmlns='jabber:iq:roster' ver='299'/>
   </iq>

S: <iq from='romeo@montague.lit' id='r1h3vzp7' to='romeo@montague.lit/home' type='result'/>

S: <iq from='romeo@montague.lit' id='ah382g67' to='romeo@montague.lit/home' type='set'>
     <query xmlns='jabber:iq:roster' ver='303'>
       <item jid='shylock@shakespeare.lit' subscription='remove'/>
     </query>
   </iq>

S: <iq from='romeo@montague.lit' id='b2gs90j5' to='romeo@montague.lit/home' type='set'>
     <query xmlns='jabber:iq:roster' ver='307'>
       <item jid='bill@shakespeare.lit' subscription='both'/>
     </query>
   </iq>

S: </stream:stream>

[ reconnection ]

C: <iq from='romeo@montague.lit/home' id='r2xa7gf9' to='romeo@montague.lit' type='get'>
     <query xmlns='jabber:iq:roster' ver='307'/>
   </iq>

S: <iq from='romeo@montague.lit' id='r2xa7gf9' to='romeo@montague.lit/home' type='result'/>

S: <iq from='romeo@montague.lit' id='c73gs419' to='romeo@montague.lit/home' type='set'>
     <query xmlns='jabber:iq:roster' ver='311'>
       <item jid='nurse@shakespeare.lit' name='Nurse' subscription='to'>
         <group>Servants</group>
       </item>
     </query>
   </iq>

S: <iq from='romeo@montague.lit' id='dh361f35' to='romeo@montague.lit/home' type='set'>
     <query xmlns='jabber:iq:roster' ver='315'>
       <item jid='juliet@shakespeare.lit' name='Juliet' subscription='both'>
         <group>Friends</group>
       </item>
     </query>
   </iq>

4. Stream Feature

If a server supports roster versioning, it MUST inform the connecting entity when returning stream features during the stream negotiation process; at the latest, when informing a client that resource binding is required. This is done by including a <ver/> element qualified by the 'urn:xmpp:features:rosterver' namespace.

Example 5. Stream features

<stream:features>
  <bind xmlns='urn:ietf:params:xml:ns:xmpp-bind'>
    <required/>
  </bind>
  <ver xmlns='urn:xmpp:features:rosterver'>
    <optional/>
  </ver>
</stream:features>

5. Security Considerations

It is possible that client-side caching of roster information across sessions (rather than holding them in memory only for the life of a session) could introduce new vulnerabilities, such as misuse by malware. Implementations are advised to appropriately protect cached roster data.

6. IANA Considerations

This document requires no interaction with the Internet Assigned Numbers Authority (IANA) [5].

7. XMPP Registrar Considerations

7.1 Protocol Namespaces

This specification defines the following XML namespace:

Upon advancement of this specification from a status of Experimental to a status of Draft, the XMPP Registrar [6] shall add the foregoing namespace to the registry located at <http://xmpp.org/registrar/stream-features.html>, as described in Section 4 of XMPP Registrar Function [7].

8. XML Schemas

8.1 jabber:iq:roster

This specification proposes addition of the 'ver' attribute to the schema for the 'jabber:iq:roster' namespace.

8.2 Stream Feature

<?xml version='1.0' encoding='UTF-8'?>

<xs:schema
    xmlns:xs='http://www.w3.org/2001/XMLSchema'
    targetNamespace='urn:xmpp:features:rosterver'
    xmlns='urn:xmpp:features:rosterver'
    elementFormDefault='qualified'>

  <xs:element name='ver'>
    <xs:complexType>
      <xs:choice>
        <xs:element name='optional' type='empty'/>
        <xs:element name='required' type='empty'/>
      </xs:choice>
    </xs:complexType>
  </xs:element>

  <xs:simpleType name='empty'>
    <xs:restriction base='xs:string'>
      <xs:enumeration value=''/>
    </xs:restriction>
  </xs:simpleType>

</xs:schema>

9. Acknowledgements

Thanks to Dave Cridland, Richard Dobson, Leonid Evdokimov, Fabio Forno, Alexander Gnauck, Juha Hartikainen, Joe Hildebrand, Justin Karneges, Sachin Khandelwal, Curtis King, Pedro Melo, Matthew Wild, and Jiří Zárevúcký for their comments. The definition of a sequence number borrows some concepts from the definition of the CONDSTORE extension to IMAP as provided in RFC 4551 [8]

Appendices

Appendix A: Document Information

Series: XEP
Number: 0237
Publisher: XMPP Standards Foundation
Status: Proposed
Type: Standards Track
Version: 0.9
Last Updated: 2009-04-22
Approving Body: XMPP Council
Dependencies: XMPP Core, XMPP IM
Supersedes: None
Superseded By: None
Short Name: N/A
Source Control: HTML  RSS

Appendix B: Author Information

Peter Saint-Andre

JabberID: stpeter@jabber.org
URI: https://stpeter.im/

Appendix C: Legal Notices

Copyright

This XMPP Extension Protocol is copyright © 1999 - 2009 by the XMPP Standards Foundation (XSF).

Permissions

Permission is hereby granted, free of charge, to any person obtaining a copy of this specification (the "Specification"), to make use of the Specification without restriction, including without limitation the rights to implement the Specification in a software program, deploy the Specification in a network service, and copy, modify, merge, publish, translate, distribute, sublicense, or sell copies of the Specification, and to permit persons to whom the Specification is furnished to do so, subject to the condition that the foregoing copyright notice and this permission notice shall be included in all copies or substantial portions of the Specification. Unless separate permission is granted, modified works that are redistributed shall not contain misleading information regarding the authors, title, number, or publisher of the Specification, and shall not claim endorsement of the modified works by the authors, any organization or project to which the authors belong, or the XMPP Standards Foundation.

Disclaimer of Warranty

## NOTE WELL: This Specification is provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. In no event shall the XMPP Standards Foundation or the authors of this Specification be liable for any claim, damages, or other liability, whether in an action of contract, tort, or otherwise, arising from, out of, or in connection with the Specification or the implementation, deployment, or other use of the Specification. ##

Limitation of Liability

In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall the XMPP Standards Foundation or any author of this Specification be liable for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising out of the use or inability to use the Specification (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if the XMPP Standards Foundation or such author has been advised of the possibility of such damages.

IPR Conformance

This XMPP Extension Protocol has been contributed in full conformance with the XSF's Intellectual Property Rights Policy (a copy of which can be found at <http://xmpp.org/extensions/ipr-policy.shtml> or obtained by writing to XSF, P.O. Box 1641, Denver, CO 80201 USA).

Appendix D: Relation to XMPP

The Extensible Messaging and Presence Protocol (XMPP) is defined in the XMPP Core (RFC 3920) and XMPP IM (RFC 3921) specifications contributed by the XMPP Standards Foundation to the Internet Standards Process, which is managed by the Internet Engineering Task Force in accordance with RFC 2026. Any protocol defined in this document has been developed outside the Internet Standards Process and is to be understood as an extension to XMPP rather than as an evolution, development, or modification of XMPP itself.

Appendix E: Discussion Venue

The primary venue for discussion of XMPP Extension Protocols is the <standards@xmpp.org> discussion list.

Discussion on other xmpp.org discussion lists might also be appropriate; see <http://xmpp.org/about/discuss.shtml> for a complete list.

Errata can be sent to <editor@xmpp.org>.

Appendix F: Requirements Conformance

The following requirements keywords as used in this document are to be interpreted as described in RFC 2119: "MUST", "SHALL", "REQUIRED"; "MUST NOT", "SHALL NOT"; "SHOULD", "RECOMMENDED"; "SHOULD NOT", "NOT RECOMMENDED"; "MAY", "OPTIONAL".

Appendix G: Notes

1. RFC 3921: Extensible Messaging and Presence Protocol (XMPP): Instant Messaging and Presence <http://tools.ietf.org/html/rfc3921>.

2. RFC 3921: Extensible Messaging and Presence Protocol (XMPP): Instant Messaging and Presence <http://tools.ietf.org/html/rfc3921>.

3. rfc3921bis: proposed revisions to Extensible Messaging and Presence Protocol (XMPP): Instant Messaging and Presence <http://tools.ietf.org/html/draft-saintandre-rfc3921bis>. (work in progress)

4. RFC 958: Network Time Protocol (NTP) <http://tools.ietf.org/html/rfc0958>.

5. The Internet Assigned Numbers Authority (IANA) is the central coordinator for the assignment of unique parameter values for Internet protocols, such as port numbers and URI schemes. For further information, see <http://www.iana.org/>.

6. The XMPP Registrar maintains a list of reserved protocol namespaces as well as registries of parameters used in the context of XMPP extension protocols approved by the XMPP Standards Foundation. For further information, see <http://xmpp.org/registrar/>.

7. XEP-0053: XMPP Registrar Function <http://xmpp.org/extensions/xep-0053.html>.

8. RFC 4551: IMAP Extension for Conditional STORE Operation or Quick Flag Changes Resynchronization <http://tools.ietf.org/html/rfc4551>.

Appendix H: Revision History

Version 0.9 (2009-04-22)

Further clarified several implementation notes.

(psa)

Version 0.8 (2009-04-20)

Defined schema for stream feature; adjusted some wording for improved clarity.

(psa)

Version 0.7 (2009-04-17)

Modified the underlying model per list consensus; added more detailed scenarios to illustrate usage.

(psa)

Version 0.6 (2009-03-31)

Clarified definition of sequence number.

(psa)

Version 0.5 (2009-02-19)

Reverted to a roster-specific method and modified presentation to enable incorporation into rfc3921bis.

(psa)

Version 0.4 (2008-09-17)

Defined new namespace and generalized to handle service discovery and other use cases in addition to rosters.

(psa)

Version 0.3 (2008-04-21)

Defined protocol solely in terms of full rosters and roster pushes (no more roster diffs); added implementation notes; clarified server behavior if cached version is unavailable.

(psa)

Version 0.2 (2008-03-06)

Renamed to data sequencing; clarified server behavior.

(psa)

Version 0.1 (2008-03-05)

Initial published version; per Council consensus, removed optionality regarding semantics of the version attribute.

(psa)

Version 0.0.3 (2008-03-05)

Corrected semantics of version attribute (should be a strictly increasing sequence number but may be any unique identifier).

(psa)

Version 0.0.2 (2008-03-04)

Clarified description of roster diff; added diff attribute and specified its use in roster results; specified use of version attribute in roster pushes.

(psa)

Version 0.0.1 (2008-03-04)

First draft.

(psa)

END