XEP-0313: Message Archive Management

WARNING: This Standards-Track document is Experimental. Publication as an XMPP Extension Protocol does not imply approval of this proposal by the XMPP Standards Foundation. Implementation of the protocol described herein is encouraged in exploratory implementations, but production systems are advised to carefully consider whether it is appropriate to deploy implementations of this protocol before it advances to a status of Draft.

Table of Contents

1. Introduction

It is a common desire for a user using XMPP for IM to want to store their messages in a central archive on their server. This feature allows them to record conversations that take place on clients that do not support local history storage, and also to synchronise their conversation history seamlessly between multiple clients.

2. Requirements

As this extension aims to make things easy for client developers, some research was made into the way clients handle history today. The resulting protocol was designed to allow for the following primary usage scenarios:

• Automatic history synchronization between multiple clients.
• Calendar-based on-demand display of historic messages in a client that doesn't keep local history.
• So-called 'infinite' scrollback, whereby a client automatically fetches and displays historic messages naturally in the message log as the user scrolls back in time.

Another extension for archiving already exists in XMPP, Message Archiving [1]). However implementation experience has shown that the protocol defined therein supports rather more functionality than is typically needed for the above uses, and is significantly more effort to implement.

This specification aims to define a much simpler and modular protocol for working with a server-side message store. Through this it is hoped to boost implementation and deployment of archiving in XMPP. It should be noted that (although not required) a server is free to implement XEP-0136 alongside this protocol if it so chooses, though a mapping between both protocols is beyond the scope of this specification.

Notable functionality in XEP-0136 that is intentionally not defined by this specification for simplicity:

• Support for 'collections'. Few clients even support this concept for local history storage, and it is possible to apply the logic for splitting a stream of messages into conversations on the client-side, thus greatly simplifying the protocol.
• Support for uploading to the archive. On the assumption that a server automatically archives messages to and from the client, it is typically rare for a client to end up with messages that are not already in the archive. Uploading could be useful for bootstrapping an empty archive however, and may be defined in a future specification if there is demand for such functionality.
• Support for 'off the record' chats (OTR; not to be confused with the encryption algorithm of the same name). This allowed complex negotiation for either the user or contact to command specific conversations to bypass the archive. In reality the archiving behaviour of a contact's server cannot be enforced (they could ignore the OTR request and archive the messages anyway without your consent) so this specification does not try and regulate that. Equivalent functionality can be implemented with server logic for not including encrypted or specially-tagged messages in the archive (out of scope for this specification).
• Support per-session configuration. This feature was deemed unnecessary for the majority of implementations, and hence the configuration protocol in this specification is much simplified, which allows for a simple user interface in clients. Advanced configuration however may be performed through ad-hoc commands depending on the server implementation.

3. Message archives

An archive is a collection of messages stored on a user's server. Messages sent to or from a user's account are generally automatically added to a user's archive by the server. The collection is ordered chronologically by the time each message was sent/received.

Exactly which messages a server archives is left up to implementation and deployment policy, but as a minimum servers SHOULD NOT archive messages that do not have a <body/> child tag.

At a minimum, a stored message consists of the following pieces of information:

• A timestamp of when the message was sent (for an outgoing message) or received (for an incoming message).
• The remote JID that the stanza is to (for an outgoing message) or from (for an incoming message).
• A server-assigned UID that MUST be unique within the archive.
• The message stanza itself. The entire original stanza SHOULD be stored, but at a minimum only the <body/> tag MUST be preserved (ie. the server might, at its discretion, strip certain extensions from messages before storage).

A server MAY impose limits on the size of a user's archive. For example a server might begin to discard old messages once the archive reaches a certain size, or only keep messages until they reach a certain age. The UIDs of deleted messages MUST NOT be reused for new messages.

Finally, there is no restriction on where an archive may be hosted. Servers that archive messages on behalf of local users SHOULD expose archives to the user on their bare JID, while a MUC service might allow MAM queries to be sent to the room's bare JID.

4. Querying the archive

A client is able to query the archive for all messages within a certain timespan, optionally restricting results to those to/from a particular JID. To allow limiting the results or paging through them a client may use Result Set Management [2], which MUST be supported by servers.

A query consists of an <iq/> stanza addressed to the account or server entity hosting the archive, with a 'query' payload. On receiving the query, the server pushes to the client a series of messages from the archive that match the client's given criteria, and finally returns the <iq/> result.

Example 1. Querying the archive for messages

<iq type='get' id='juliet1'>
  <query xmlns='urn:xmpp:mam:tmp' queryid='f27' />
</iq>

[... server sends matching messages ...]

<iq type='result' id='juliet1'/>
    

To ensure that the client knows when the results are complete, the server MUST delay the result <iq/> until after it has pushed all the results to the client. An optional 'queryid' attribute allows the client to match results to a certain query.

<iq type='get' id='juliet1'>
  <query xmlns='urn:xmpp:mam:tmp'>
    <with>juliet@capulet.com</with>
  </query>
</iq>
    

<iq type='get' id='juliet1'>
  <query xmlns='urn:xmpp:mam:tmp'>
      <start>2010-06-07T00:00:00Z</start>
      <end>2010-07-07T13:23:54Z</end>
  </query>
</iq>
    

<iq type='get' id='juliet1'>
  <query xmlns='urn:xmpp:mam:tmp'>
    <start>2010-08-07T00:00:00Z</start>
  </query>
</iq>
    

<iq type='get' id='q29302'>
  <query xmlns='urn:xmpp:mam:tmp'>
      <start>2010-08-07T00:00:00Z</start>
      <set xmlns='http://jabber.org/protocol/rsm'>
         <limit>10</limit>
      </set>
  </query>
</iq>
    

<iq type='error' id='q29302'>
  <error type='modify'>
    <policy-violation xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
    <text>Too many results</text>
  </error>
</iq>
    

<message id='aeb213' to='juliet@capulet.com/chamber'>
  <result xmlns='urn:xmpp:mam:tmp' queryid='f27' id='28482-98726-73623' />
  <forwarded xmlns='urn:xmpp:forward:0'>
    <delay xmlns='urn:xmpp:delay' stamp='2010-07-10T23:08:25Z'/>
    <message to='juliet@capulet.com/balcony'
      from='romeo@montague.net/orchard'
      type='chat'>
      <body>Call me but love, and I'll be new baptized; Henceforth I never will be Romeo.</body>
    </message>
  </forwarded>
</message>

<message id='aeb214' to='juliet@capulet.com/chamber'>
  <result xmlns='urn:xmpp:mam:tmp' queryid='f27' id='5d398-28273-f7382'/>
  <forwarded xmlns='urn:xmpp:forward:0'>
    <delay xmlns='urn:xmpp:delay' stamp='2010-07-10T23:09:32Z'/>
    <message to='romeo@montague.net/orchard'
       from='juliet@capulet.com/balcony'
       type='chat' id='8a54s'>
      <body>What man art thou that thus bescreen'd in night so stumblest on my counsel?</body>
    </message>
  </forwarded>
</message>
    

5. Archiving Preferences

Depending on implementation and deployment policies, a server MAY allow the user to have control over the server's archiving behaviour. This specification defines a basic protocol for this, and also allows a server to offer more advanced configuration to a user.

  <iq type='set' id='juliet2'>
    <prefs xmlns='urn:xmpp:mam:tmp' default="roster">
      <always>
        <jid>romeo@montague.net</jid>
      </always>
      <never>
        <jid>montague@montague.net</jid>
      </never>
    </prefs>
  </iq>

  <iq type='result' id='juliet1'>
    <prefs xmlns='urn:xmpp:mam:tmp' default="roster">
      <always>
        <jid>romeo@montague.net</jid>
      </always>
      <never>
        <jid>montague@montague.net</jid>
      </never>
    </prefs>
  </iq>
      

6. Determining support

If a server or other entity hosts archives and supports MAM queries, it MUST advertise that fact by including the feature "urn:xmpp:mam:tmp" in response to a Service Discovery [8] request:

Example 10. Client queries for server features

  <iq type='get' id='disco1' to='capulet.lit' from='juliet@capulet.lit/balcony'>
      <query xmlns='http://jabber.org/protocol/disco#info'/>
  </iq>

Example 11. Server responds with features

  <iq type='result' id='disco1' from='capulet.lit' to='juliet@capulet.lit/balcony'>
      <query xmlns='http://jabber.org/protocol/disco#info'>
          ...
          <feature var='urn:xmpp:mam:tmp'/>
          ...
      </query>
  </iq>

Appendices

Appendix A: Document Information

Series: XEP
Number: 0313
Publisher: XMPP Standards Foundation
Status: Experimental
Type: Standards Track
Version: 0.1
Last Updated: 2012-04-18
Approving Body: XMPP Council
Dependencies: XMPP Core, XEP-0030, XEP-0059, XEP-0297
Supersedes: None
Superseded By: None
Short Name: mam
Schema: <http://www.xmpp.org/schemas/archive-management.xsd>
Source Control: HTML
This document in other formats: XML  PDF

Appendix B: Author Information

Appendix C: Legal Notices

Appendix D: Relation to XMPP

The Extensible Messaging and Presence Protocol (XMPP) is defined in the XMPP Core (RFC 6120) and XMPP IM (RFC 6121) specifications contributed by the XMPP Standards Foundation to the Internet Standards Process, which is managed by the Internet Engineering Task Force in accordance with RFC 2026. Any protocol defined in this document has been developed outside the Internet Standards Process and is to be understood as an extension to XMPP rather than as an evolution, development, or modification of XMPP itself.

Appendix E: Discussion Venue

The primary venue for discussion of XMPP Extension Protocols is the <standards@xmpp.org> discussion list.

Discussion on other xmpp.org discussion lists might also be appropriate; see <http://xmpp.org/about/discuss.shtml> for a complete list.

Errata can be sent to <editor@xmpp.org>.

Appendix F: Requirements Conformance

The following requirements keywords as used in this document are to be interpreted as described in RFC 2119: "MUST", "SHALL", "REQUIRED"; "MUST NOT", "SHALL NOT"; "SHOULD", "RECOMMENDED"; "SHOULD NOT", "NOT RECOMMENDED"; "MAY", "OPTIONAL".

Appendix G: Notes

Appendix H: Revision History

Note: Older versions of this specification might be available at http://xmpp.org/extensions/attic/

END