JEP-0013: Flexible Offline Message Retrieval

A protocol for flexible, POP3-like handling of offline messages in Jabber.

NOTICE: The protocol defined herein is a Draft Standard of the Jabber Software Foundation. Implementations are encouraged and the protocol is appropriate for deployment in production systems, but some changes to the protocol are possible before it becomes a Final Standard.

JEP Information

Status: Draft
Type: Standards Track
Number: 0013
Version: 1.0
Last Updated: 2003-09-09
JIG: Standards JID
Approving Body: Jabber Council
Dependencies: None
Supersedes: None
Superseded By: None
Short Name: offline
Schema for http://jabber.org/protocol/offline: <http://jabber.org/protocol/offline/offline.xsd>

Author Information

Peter Saint-Andre

Email: stpeter@jabber.org
JID: stpeter@jabber.org

Craig Kaes

Email: ckaes@jabber.com
JID: ckaes@corp.jabber.com

Legal Notice

This Jabber Enhancement Proposal is copyright 1999 - 2005 by the Jabber Software Foundation (JSF) and is in full conformance with the JSF's Intellectual Property Rights Policy <http://www.jabber.org/jsf/ipr-policy.shtml>. This material may be distributed only subject to the terms and conditions set forth in the Open Publication License, v1.0 or later (the latest version is presently available at <http://www.opencontent.org/openpub/>).

Discussion Venue

The preferred venue for discussion of this document is the Standards-JIG discussion list: <http://mail.jabber.org/mailman/listinfo/standards-jig>.

Relation to XMPP

The Extensible Messaging and Presence Protocol (XMPP) is defined in the XMPP Core (RFC 3920) and XMPP IM (RFC 3921) specifications contributed by the Jabber Software Foundation to the Internet Standards Process, which is managed by the Internet Engineering Task Force in accordance with RFC 2026. Any protocol defined in this JEP has been developed outside the Internet Standards Process and is to be understood as an extension to XMPP rather than as an evolution, development, or modification of XMPP itself.

Conformance Terms

The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Table of Contents

1. Introduction
2. Use Cases
2.1. Determining Server Support
2.2. Requesting Message Header Information
2.3. Retrieving Specific Messages
2.4. Removing Specific Messages
2.5. Retrieving All Messages
2.6. Removing All Messages
3. Protocol Flow
4. Security Considerations
5. IANA Considerations
6. Jabber Registrar Considerations
7. XML Schema
Notes
Revision History

1. Introduction

The current means for retrieving one's offline messages is simple: one sends available presence to the server and, as a consequence, the server sends a one-time "flood" of all the messages that have been stored while one was offline. This simplification has the following deficiencies:

  1. It can be overwhelming, which is undesirable for the vacationer or heavy user. Many individuals upon returning to work from a weeklong vacation spend the first few hours wading through the dozens, even hundreds, of emails that they received during their absence. Unlucky, however, is this user who then logs onto their Jabber server and is bombarded by hundreds of instant messages, possibly in scores of popup dialogs, simultaneously. Should their client crash, they have lost all communication that occurred while they were away.
  2. It can be difficult to integrate with web-based email clients, which is undesirable for some portals. Several large portals are currently trying to blur the distinction between IM and email -- providing both through one web interface. With offline retrieval semantics so vastly different between the two, this is quite difficult.

What is needed is a flexible semantic for offline message handling, similar to POP3 in the email world. This would enable the wireless user to view header information for all offline messages and select only those from their boss and important clients for viewing. It would enable the vacationer to read and delete their messages one at a time, minimizing the possibility of losing all correspondence. And it would provide for seamless integration with existing web-based email clients.

In particular, such a protocol must support the following use cases:

  1. Client determines server support for this protocol.
  2. Client requests message "header" information (thereby choosing flexible offline message retrieval as opposed to old-fashioned "flood" mode).
  3. Client retrieves specific messages.
  4. Client removes specific messages.
  5. Client retrieves all messages.
  6. Client removes all messages.

2. Use Cases

2.1 Determining Server Support

In order to discover whether one's server supports this protocol, one uses Service Discovery [1].

Example 1. User Requests Service Discovery Information

<iq type='get' to='montague.net'>
  <query xmlns='http://jabber.org/protocol/disco#info'/> 
</iq>

Example 2. Server Reply to Discovery Request

<iq type='result' from='montague.net' to='romeo@montague.net/orchard'>
  <query xmlns='http://jabber.org/protocol/disco#info'/>
    ...
    <feature var='http://jabber.org/protocol/offline'/>
    ...
  </query>
</iq>

If the server supports this protocol, it MUST return a <feature/> element in the disco result with the 'var' attribute set to the namespace name for this protocol: 'http://jabber.org/protocol/offline'.

2.2 Requesting Message Header Information

In order to determine whether to view a full message, a user must be able to inspect the full Jabber ID (JID) of the sender and a timestamp specifying the date and time of the message. In addition, the header information must include a unique identifier for the message within the user's "inbox" so that the user may appropriately manage (view or remove) the message.

In order to retrieve meta-information about the message, the user sends a disco#items request without a 'to' address (i.e., implicitly to the user himself) and with the disco node specified as 'http://jabber.org/protocol/offline'.

Example 3. User Requests Offline Message Headers

<iq type='get'>
  <query 
      xmlns='http://jabber.org/protocol/disco#items' 
      node='http://jabber.org/protocol/offline'/>
</iq>

Upon receiving this request, the server MUST return headers for all of the user's offline messages. The server also MUST NOT send a flood of offline messages if the user subsequently sends initial presence to the server during this session. Thus the user is now free to send initial presence (if desired) and to engage in normal IM activities while continuing to read through offline messages. Note well: once the user sends presence, the user's server MUST deliver in-session messages as usual; this JEP applies to offline messages only! In addition, if the user authenticates and provides presence for another resource while the first (non-flood) resource still has an active session, the server MUST NOT flood the second resource with the offline message queue.

Example 4. Server Provides Offline Message Headers

<iq type='result' to='romeo@montague.net/orchard'>
  <query 
      xmlns='http://jabber.org/protocol/disco#items' 
      node='http://jabber.org/protocol/offline'>
    <item 
        jid='romeo@montague.net'
        node='2003-02-27T22:49:17.008Z'
        name='mercutio@shakespeare.lit/pda'/>
    <item 
        jid='romeo@montague.net'
        node='2003-02-27T22:52:37.225Z'
        name='juliet@capulet.com/balcony'/>
    <item 
        jid='romeo@montague.net'
        node='2003-02-27T22:52:51.270Z'
        name='juliet@capulet.com/balcony'/>
    <item 
        jid='romeo@montague.net'
        node='2003-02-27T22:53:03.421Z'
        name='juliet@capulet.com/balcony'/>
    <item 
        jid='romeo@montague.net'
        node='2003-02-27T22:53:13.925Z'
        name='juliet@capulet.com/balcony'/>
  </query>
</iq>

If the requestor is a JID other than an authorized resource of the user, the server MUST return a 403 (Forbidden) error.

If the requestor is authorized but the node does not exist, the server MUST return a 404 (Not Found) error.

If there are no offline messages for this user, the server MUST return an empty query as defined in JEP-0030.

The syntax and semantics of the attributes are as follows:

2.3 Retrieving Specific Messages

Messages are viewed based on the value of the 'node' attribute as provided for each item returned by the server in the header information. A user may request one or more messages in the same IQ get.

Example 5. User Requests Offline Messages

<iq type='get' id='view1'>
  <offline xmlns='http://jabber.org/protocol/offline'>
    <item action='view'
          node='2003-02-27T22:52:37.225Z'/>
  </offline>
</iq>

If the requestor is a JID other than an authorized resource of the user, the server MUST return a 403 (Forbidden) error.

If the requestor is authorized but the node does not exist, the server MUST return a 404 (Not Found) error.

Otherwise, the server MUST send the requested message(s) plus an IQ result:

Example 6. Server Provides Offline Messages

<message to='romeo@montague.net' from='juliet@capulet.com/balcony'>
  <body>O Romeo, Romeo! wherefore art thou Romeo?</body>
  <offline xmlns='http://jabber.org/protocol/offline'>
    <item node='2003-02-27T22:52:37.225Z'/>
  </offline>
</message>

<iq type='result' to='user@domain/resource' id='view1'/>

In order to distinguish incoming messages, each message MUST contain the node value.

2.4 Removing Specific Messages

A server MUST NOT remove a message simply because it has been requested by and delivered to the user; instead, the user must specifically request to remove a message. This further implies that the user's offline message queue SHOULD NOT be automatically cleared out by the server if there are offline messages remaining when the user's session ends. However, a server or deployment MAY remove messages according to its own algorithms (e.g., storage timeouts based on a "first in first out" rule) or policies (message size limits) if desired.

As with viewing, messages are removed based on the value of the 'node' attribute as provided for each item returned by the server in the header information. The user may request the removal of one or more messages in the same IQ set.

Example 7. User Requests Removal of Offline Messages

<iq type='set' id='remove1'>
  <offline xmlns='http://jabber.org/protocol/offline'>
    <item action='remove'
          node='2003-02-27T22:49:17.008Z'
    <item action='remove'
          node='2003-02-27T22:52:37.225Z'/>
  </offline>
</iq>

If the requestor is a JID other than an authorized resource of the user, the server MUST return a 403 (Forbidden) error.

If the requestor is authorized but the node does not exist, the server MUST return a 404 (Not Found) error.

Otherwise, the server MUST remove the messages and inform the user:

Example 8. Server Informs User of Removal

<iq type='result' to='romeo@montague.net/orchard' id='remove1'/>

2.5 Retrieving All Messages

The user may retrieve all message by sending the "fetch" command:

Example 9. User Retrieval of All Offline Messages

<iq type='get' id='fetch1'>
  <offline xmlns='http://jabber.org/protocol/offline'>
    <fetch/>
  </offline>
</iq>

If the requestor is a JID other than an authorized resource of the user, the server MUST return a 403 (Forbidden) error.

If the requestor is authorized but the node does not exist, the server MUST return a 404 (Not Found) error.

Otherwise, the server MUST retrieve all messages and inform the user:

Example 10. Server Sends All Messages and Informs User of Successful Fetch

<message to='romeo@montague.net' from='juliet@capulet.com/balcony'>
  <body>O Romeo, Romeo! wherefore art thou Romeo?</body>
  <offline xmlns='http://jabber.org/protocol/offline'>
    <item node='2003-02-27T22:52:37.225Z'/>
  </offline>
</message>

...

<iq type='result' to='romeo@montague.net/orchard' id='fetch1'/>

A client MAY retrieve all messages without first requesting message headers. In this case, the server MUST return all of the user's offline messages and also MUST NOT send a flood of offline messages if the user subsequently sends initial presence to the server during this session. That is, the semantics here are the same as for requesting message headers.

2.6 Removing All Messages

The user may remove all message by sending the "purge" command:

Example 11. User Requests Removal of Offline Messages

<iq type='set' id='purge1'>
  <offline xmlns='http://jabber.org/protocol/offline'>
    <purge/>
  </offline>
</iq>

If the requestor is a JID other than an authorized resource of the user, the server MUST return a 403 (Forbidden) error.

If the requestor is authorized but the node does not exist, the server MUST return a 404 (Not Found) error.

Otherwise, the server MUST remove all messages and inform the user:

Example 12. Server Informs User of Successful Purge

<iq type='result' to='romeo@montague.net/orchard' id='purge1'/>

3. Protocol Flow

This section shows the flow of protocol between client (C:) and server (S:) for the existing (flood) scenario and the improved (POP3-like) scenario.

Example 13. Existing Protocol Flow (Flood)

C: <stream:stream ...>

S: <stream:stream ...>

C: authentication (SASL in XMPP, non-SASL in older systems)

S: acknowledge successful authentication 

C: <presence/>

S: send message flood to Client

C: receive flood, send and receive messages, etc.

... and so on

The semantics change with POP-like offline message handling, and server behavior changes as well...

Example 14. Improved Protocol Flow (POP3-Like)

C: <stream:stream ...>

S: <stream:stream ...>

C: authentication (SASL in XMPP, non-SASL in older systems)

S: acknowledge successful authentication 

C: request message headers

S: send message headers to Client

NOTE: Server now MUST NOT flood Client with offline messages.

C: <presence/>

NOTE: Server does not flood Client with offline messages, but 
      sends in-session messages as usual.

C: request and remove offline messages, send and receive messages, etc.

... and so on

4. Security Considerations

A server MUST NOT deliver a user's offline messages to any JID except one of the user's authorized resources.

5. IANA Considerations

This JEP requires no interaction with the Internet Assigned Numbers Authority (IANA) [3].

6. Jabber Registrar Considerations

The 'http://jabber.org/protocol/offline' namespace shall be registered with the Jabber Registrar [4] as a result of this JEP.

7. XML Schema 

<?xml version='1.0' encoding='UTF-8'?>

<xs:schema
    xmlns:xs='http://www.w3.org/2001/XMLSchema'
    targetNamespace='http://jabber.org/protocol/offline'
    xmlns='http://jabber.org/protocol/offline'
    elementFormDefault='qualified'>

  <xs:element name='offline'>
    <xs:complexType>
      <xs:sequence>
        <xs:element ref='item' minOccurs='1' maxOccurs='unbounded'/>
      </xs:sequence>
    </xs:complexType>
  </xs:element>

  <xs:element name='item'>
    <xs:complexType>
      <xs:attribute name='action' use='optional'>
        <xs:simpleType>
          <xs:restriction base='xs:NCName'>
            <xs:enumeration value='remove'/>
            <xs:enumeration value='view'/>
          </xs:restriction>
        </xs:simpleType>
      </xs:attribute>
      <xs:attribute name='jid' type='xs:string' use='optional'/>
      <xs:attribute name='node' type='xs:string' use='required'/>
    </xs:complexType>
  </xs:element>

</xs:schema>

Notes

1. JEP-0030: Service Discovery <http://www.jabber.org/jeps/jep-0030.html>.

2. JEP-0082: Jabber Date and Time Profiles <http://www.jabber.org/jeps/jep-0082.html>.

3. The Internet Assigned Numbers Authority (IANA) is the central coordinator for the assignment of unique parameter values for Internet protocols, such as port numbers and URI schemes. For further information, see <http://www.iana.org/>.

4. The Jabber Registrar maintains a list of reserved Jabber protocol namespaces as well as registries of parameters used in the context of protocols approved by the Jabber Software Foundation. For further information, see <http://www.jabber.org/registrar/>.

Revision History

Version 1.0 (2003-09-09)

Per a vote of the Jabber Council, advanced status to Draft. (psa)

Version 0.8 (2003-08-25)

More changes to address Council feedback: removed bandwidth rationale in requirements; added protocol flow section; adjusted semantics for node values (opaque, but dictionary ordering allowed). Also added <fetch/> and <purge/> elements. (psa)

Version 0.7 (2003-08-21)

Changes to address Council feedback: added error codes; enhanced the security considerations; clarified the nature of the node IDs and removed the protocol URI string (leaving only the timestamp). (psa)

Version 0.6 (2003-06-10)

Slight fixes to JEP-0082 reference and XML schema. (psa)

Version 0.5 (2003-04-28)

Added reference to JEP-0082; changed timestamp format to use milliseconds rather than ten-thousandths of a second; made several small editorial changes throughout. (psa)

Version 0.4 (2003-02-27)

Major overhaul: clarified requirements, incorporated disco, simplified and updated the protocol, specified syntax and semantics for nodes, defined business rules, and added XML schema. (psa)

Version 0.3 (2002-10-02)

Reworked to exclude XDBID performace hack, thereby maximizing palatability. Removed all changes made by psa. (cak)

Version 0.2 (2002-10-02)

Changed type and added information about scope. (psa)

Version 0.1 (2002-01-11)

Initial version (cak)

END