JEP-0136: Message Archiving

This document defines mechanisms for server-side archiving of XMPP messages.


WARNING: This Standards-Track JEP is Experimental. Publication as a Jabber Enhancement Proposal does not imply approval of this proposal by the Jabber Software Foundation. Implementation of the protocol described herein is encouraged in exploratory implementations, but production systems should not deploy implementations of this protocol until it advances to a status of Draft.


JEP Information

Status: Experimental
Type: Standards Track
Number: 0136
Version: 0.5
Last Updated: 2006-05-03
JIG: Standards JIG
Approving Body: Jabber Council
Dependencies: XMPP Core, XMPP IM, JEP-0030
Supersedes: None
Superseded By: None
Short Name: archive
Wiki Page: <http://wiki.jabber.org/index.php/Message Archiving (JEP-0136)>

Author Information

Justin Karneges

Email: justin@affinix.com
JID: justin@andbit.net

Ian Paterson

Email: ian.paterson@clientside.co.uk
JID: ian@zoofy.com

Jon Perlow

Email: jonp@google.com
JID: jonp@google.com

Peter Saint-Andre

Email: stpeter@jabber.org
JID: stpeter@jabber.org

Legal Notice

This Jabber Enhancement Proposal is copyright 1999 - 2006 by the Jabber Software Foundation (JSF) and is in full conformance with the JSF's Intellectual Property Rights Policy <http://www.jabber.org/jsf/ipr-policy.shtml>. This material may be distributed only subject to the terms and conditions set forth in the Creative Commons Attribution License (<http://creativecommons.org/licenses/by/2.5/>).

Discussion Venue

The preferred venue for discussion of this document is the Standards-JIG discussion list: <http://mail.jabber.org/mailman/listinfo/standards-jig>.

Relation to XMPP

The Extensible Messaging and Presence Protocol (XMPP) is defined in the XMPP Core (RFC 3920) and XMPP IM (RFC 3921) specifications contributed by the Jabber Software Foundation to the Internet Standards Process, which is managed by the Internet Engineering Task Force in accordance with RFC 2026. Any protocol defined in this JEP has been developed outside the Internet Standards Process and is to be understood as an extension to XMPP rather than as an evolution, development, or modification of XMPP itself.

Conformance Terms

The following keywords as used in this document are to be interpreted as described in RFC 2119: "MUST", "SHALL", "REQUIRED"; "MUST NOT", "SHALL NOT"; "SHOULD", "RECOMMENDED"; "SHOULD NOT", "NOT RECOMMENDED"; "MAY", "OPTIONAL".


Table of Contents

1. Introduction
2. Determining Server Support
3. Automated Archiving
3.1. Save Mode
3.1.1. Requirements
3.1.2. Determining Save Mode States
3.1.3. Toggling Default Save Mode State
3.1.4. Toggling Save Mode State for a Contact
3.2. Off-the-Record Mode
3.2.1. Requirements
3.2.2. Determining Off-the-Record Mode for a Contact
3.2.3. Toggling Off-the-Record Mode State for a Contact
3.2.4. Sending Message Stanzas
4. Manual Archiving
4.1. Requirements
4.2. Uploading Messages to a Collection
5. Archive Management
5.1. Obtaining a List of Collections
5.2. Retrieving a Collection
5.3. Removing a Collection
6. File Format
7. Implementation Notes
7.1. Bandwidth Considerations
7.2. Karma Considerations
7.3. Synchronization
7.4. Storage Considerations
7.5. Groupchat Messages
8. Security Considerations
8.1. Encryption of Archived Messages
8.2. Store Headers
8.3. Subject Attributes
9. IANA Considerations
10. Jabber Registrar Considerations
10.1. Protocol Namespaces
10.2. Service Discovery Features
11. XML Schemas
Notes
Revision History


1. Introduction

Many XMPP clients implement some form of client-side message archiving. However, it is not always convenient or even possible to archive messages locally, e.g., because it is easier to keep all archives in one place (not scattered around on multiple computers or devices) or because the client operates in a web browser or resides on a mobile device that does not have sufficient local storage for message archiving. In addition, server-side archiving makes it possible to offer new services such as integration of IM and email. Therefore it is beneficial to define methods for server-side archiving of XMPP messages.

There are two main approaches to this problem:

  1. Enable the server (at the client's request) to archive messages as they pass through the server; we call this automated archiving.
  2. Enable the client to send individual messages or entire conversations to the server for archiving (optionally after encryption); we call this manual archiving.

So that client and server developers can refer to one specification, both approaches are defined in this document. In addition, this document defines common methods for retrieving archived messages.

2. Determining Server Support

A client discovers whether its server supports this protocol using Service Discovery [1].

Example 1. Client Service Discovery request

    
<iq type='get' to='montague.net'>
  <query xmlns='http://jabber.org/protocol/disco#info'/>
</iq>

For each feature defined herein, if the server supports that feature it MUST return a <feature/> element with the 'var' attribute set to 'http://jabber.org/protocol/archive#name', where "name" is "save" for the Save Mode feature, "otr" for the Off-the-Record Mode feature, "manual" for the Manual Archiving feature, or "manage" for the Archive Management feature.

Example 2. Server Service Discovery response

    
<iq type='result'
    from='montague.net'
    to='romeo@montague.net/orchard'>
  <query xmlns='http://jabber.org/protocol/disco#info'/>
    ...
    <feature var='http://jabber.org/protocol/archive#manage'/>
    <feature var='http://jabber.org/protocol/archive#manual'/>
    <feature var='http://jabber.org/protocol/archive#otr'/>
    <feature var='http://jabber.org/protocol/archive#save'/>
    ...
  </query>
</iq>

3. Automated Archiving

There are two features related to automated archiving:

  1. Being able to turn on and turn off automated archiving; this is called "Save Mode".
  2. Being able to take certain conversations (or conversations with certain contacts) "off-the-record" for both the user and the contact with whom the user is chatting (i.e., neither party's server shall archive the conversation); this is called "Off-the-Record Mode".

Requirements and protocol flows for these features are defined below.

3.1 Save Mode

3.1.1 Requirements

Automated server-side archiving of XMPP messages eases the burden on clients. However, not all users may want to archive messages on the server. Therefore, a client should be able to toggle archiving (here referred to as Save Mode) on a default basis (i.e., specify whether by default all conversations are to be saved on the server or not). In addition, a user may want to override the default setting for a particular contact. Therefore this document addresses the following requirements for Save Mode:

  1. A user must be able to toggle default Save Mode.
  2. A user must be able to toggle Save Mode for a particular contact.
  3. A user must be able to determine the current Save Mode (default and for particular contacts).

3.1.2 Determining Save Mode States

In order to determine the current Save Mode states both for the default and for particular contacts, a user sends a query to the user's server:

Example 3. Client Requests Save Mode States

<iq type='get' id='save1' from='juliet@capulet.com/chamber'>
  <save xmlns='http://jabber.org/protocol/archive'/>
</iq>
      

The server sends the Save Mode state(s) to the user.

The state may be set only for the default.

Example 4. Server Returns Save Mode States

<iq type='result' id='save1' to='juliet@capulet.com/chamber'>
  <save xmlns='http://jabber.org/protocol/archive'>
    <default save='true'/>
  </save>
</iq>
      

The state may also include settings for particular contacts.

Example 5. Client Requests Save Mode States

<iq type='result' id='save1' to='juliet@capulet.com/chamber'>
  <save xmlns='http://jabber.org/protocol/archive'>
    <default save='true'/>
    <item jid='romeo@montague.net' save='false'/>
  </save>
</iq>
      

The server also remembers that the requesting resource is "save-mode aware" and will broadcast it any changes in the Save Mode state.

Note: It is STRONGLY RECOMMENDED for the value of the 'jid' attribute to be a bare JID (<node@domain.tld>).

3.1.3 Toggling Default Save Mode State

A user may toggle the default Save Mode state by updating the setting:

Example 6. Client Toggles Default Save Mode State

<iq type='set' id='save2' from='juliet@capulet.com/chamber'>
  <save xmlns='http://jabber.org/protocol/archive'>
    <default save='true'/>
  </save>
</iq>
      

If the server can process the request, it acknowledges the change:

Example 7. Server Acknowledges Change

<iq type='result' id='save2' to='juliet@capulet.com/chamber'/>
      

The server then MUST inform all of the user's connected resources:

Example 8. Server Pushes New Save Mode State

<iq type='set' id='savepush1' to='juliet@capulet.com/chamber'>
  <save xmlns='http://jabber.org/protocol/archive'>
    <default save='true'/>
  </save>
</iq>

<iq type='set' id='savepush2' to='juliet@capulet.com/pda'>
  <save xmlns='http://jabber.org/protocol/archive'>
    <default save='true'/>
  </save>
</iq>
      

Note: Error cases to follow.

3.1.4 Toggling Save Mode State for a Contact

A user may toggle the Save Mode state for a particular contact by updating the setting:

Example 9. Client Toggles Save Mode State for a Contact

<iq type='set' item='save3' from='juliet@capulet.com/chamber'>
  <save xmlns='http://jabber.org/protocol/archive'>
    <item jid='romeo@montague.net' save='false'/>
  </save>
</iq>
      

If the server can process the request, it acknowledges the change:

Example 10. Server Acknowleges Change

<iq type='result' id='save3' to='juliet@capulet.com/chamber'/>
      

The server then MUST inform all of the user's connected resources:

Example 11. Server Pushes New Save Mode State

<iq type='set' id='savepush3' to='juliet@capulet.com/chamber'>
  <save xmlns='http://jabber.org/protocol/archive'>
    <item jid='romeo@montague.net' save='false'/>
  </save>
</iq>

<iq type='set' id='savepush4' to='juliet@capulet.com/pda'>
  <save xmlns='http://jabber.org/protocol/archive'>
    <item jid='romeo@montague.net' save='false'/>
  </save>
</iq>
      

Note: Error cases to follow.

3.2 Off-the-Record Mode

3.2.1 Requirements

Even if a user does not save chat conversations with a particular contact, the user may also want to request that the contact also not save their chat conversations; this is in effect a shared off-the-record state that can be enabled and disabled by either user. Therefore, a client should be able to take a chat "off the record" by requesting use of the Off-the-Record Mode with that contact. Therefore this document addresses the following requirements for Off-the-Record Mode:

  1. A user must be able to determine the current Off-the-Record Mode for a particular contact.
  2. A user must be able to toggle Off-the-Record Mode for a particular contact.
  3. The user's server must be able to inform the contact's server of any change in the Off-the-Record Mode (from true to false or from false to true).
  4. The contact's server must toggle the state appropriately on request and inform the contact's resources.
  5. If the servers get out of sync regarding the Off-the-Record Mode state, they must be able to quickly converge again.

Note: a similar Off-the-Record Mode for Multi-User Chat [2] conversations is out of scope.

It is understood that at this time most clients will not be compatible with this document. It is expected that clients and servers will adopt this specification over time. Servers that save chats in a server-side store should implement this specification as soon as possible since server-side archiving of messages is more sensitive than client-side archiving.

Stanza Headers and Internet Metadata [3] introduces the Store header, which indicates whether the stanza may be stored or archived by the recipient. This is a partial solution to the problem we're trying to solve. First, it doesn't create a way for the two users to negotiate whether a chat conversation can be saved. Second, it only applies to the recipient of a chat, not the recipient's server. In real world scenarios, we want to make sure that the recipient's server also respects the negotiation of Off-the-Record Mode between the two parties.

3.2.2 Determining Off-the-Record Mode for a Contact

The user (romeo@montague.net) wishes to find out which contacts have Off-the-Record Mode enabled. This process is very similar to how roster requests, updates, and broadcasts work.

Example 12. Client Requests the Off-the-Record Mode States

<iq type='get' id='otr1' from='romeo@montague.net/castle'>
  <otr xmlns='http://jabber.org/protocol/archive'/>
</iq>
      

The server responds with the list of contacts whose Off-the-Record Mode state is "true".

Example 13. Server Returns Off-the-Record Mode States

<iq type='result' id='otr1' to='juliet@capulet.com/chamber'>
  <otr xmlns='http://jabber.org/protocol/archive'>
    <record jid='priest@church.org' otr='true'/>
  </otr>
</iq>
      

The server also remembers that the requesting resource is now "OTR aware" and will broadcast it any changes in the OTR states.

Note: Error cases to follow.

3.2.3 Toggling Off-the-Record Mode State for a Contact

The user (juliet@capulet.com) wishes to enter Off-the-Record Mode with contact (romeo@montague.net).

Example 14. Client Enters Off-the-Record Mode

<iq type='set' id='otr2' from='juliet@capulet.com/chamber'>
 <record xmlns='http://jabber.org/protocol/archive'
         jid='romeo@montague.net' otr='true'/>
/iq>
      

The user's server (capulet.com) updates its state so that chats with romeo@montague.net are not saved. It also sends a message to the contact's server that chats with Juliet are not to be saved.

Example 15. User's Server Informs Contact's Server of Change

<iq type='set' from='juliet@capulet.com' to='romeo@montague.net' id='notify1'>
  <record xmlns='http://jabber.org/protocol/archive' otr='true'/>
</iq>
      

The user's server also broadcasts to all of the user's otr-aware resources that the state has changed:

Example 16. OTR Broadcast to User

<iq type='set' to='juliet@capulet.com/chamber' id='update1'>
  <record xmlns='http://jabber.org/protocol/archive'
          jid='romeo@montague.net' otr='true'/>
</iq>

<iq type='set' to='juliet@capulet.com/pda' id='update2'>
  <record xmlns='http://jabber.org/protocol/archive'
          jid='romeo@montague.net' otr='true'/>
</iq>
      

The user's server then sends the result for the IQ-set.

Example 17. IQ Result

<iq type='result' id='otr2' to='juliet@capulet.com/chamber'>
      

The contact's server receives the notification, swaps the JIDs in the <item/> element, and broadcasts the change to all of the contact's otr-aware resources:

Example 18. OTR Broadcast for Contact

<iq type='set' to='romeo@montague.net/orchard' id='update3'>
  <record xmlns='http://jabber.org/protocol/archive'
          jid='juliet@capulet.com' otr='true'/>
</iq>
      

Note: Error cases to follow.

Some deployments (especially in enterprise settings) are required to archive all messages sent through the server, for example to comply with various government regulations. Such a deployment MUST return a <not-allowed/> error in response to a request to enter Off-the-Record mode and SHOULD stamp all message stanzas with <record otr='false'/>.

3.2.4 Sending Message Stanzas

The archiving-aware server stamps all messages with the current OTR state as demonstrated by the following examples. This allows servers that become out of sync to update each other on the current state as soon as a message is sent.

Rome sends a message to Juliet.

Example 19. Client sends a normal text message

<message from='romeo@montague.net/home' to='juliet@capulet.com'>
  <body>Juliet, can you sneak out tonight?</body>
</message>
      

(Optionally the sender's client MAY also include the SHIM Store header.)

Romeo's server adds an element to the message stanza indicating the current Off-the-Record Mode state before routing it to Juliet's server.

Example 20. Server stamps message with current Off-the-Record Mode state.

<message from='romeo@montague.net/home' to='juliet@capulet.com'>
  <body>Juliet, can you sneak out tonight?</body>
  <record xmlns='http://jabber.org/protocol/archive' otr='true'/>
</message>
      

When the contact's server receives the message, it MAY compare the user's Off-the-Record Mode state with its own Off-the-Record Mode state for the contact. If the two states are out of sync, it SHOULD update its state and broadcast an updated IQ set to the contact's connected resources.

4. Manual Archiving

4.1 Requirements

While automated archiving is easy for the client and server to implement, there are contexts in which automated archiving is unacceptable (e.g., because the client does not trust the server) or unworkable (e.g., when messages are encrypted using evanscent keys, as in Encrypted Sessions [4]). Therefore, often a client will want to receive a message, optionally process it (e.g., encrypt it), then ask the server to store the message.

Such messages SHOULD be stored in the form of a "collection", i.e., a set of messages from the same user that are received near each other in time. A collection is intended to mimic the natural flow of human conversations, which in instant messaging (IM) systems tend to occur in bursts (e.g., a five-minute conversation one day, followed by a ten-minute conversation the next). However, sometimes IM message exchange does not follow this conversational pattern; in that case, collections SHOULD be arbitrarily truncated so that transferring them does not violate common rate limiting restrictions (in Jabber systems, often called "karma") or memory limitations for constrained devices.

Messages are stored in message collections on the server. The client uniquely specifies a collection using the pair of attributes: 'with' (the bare JID with which the messages were exchanged) and 'start' (the UTC start time of the conversation thread, which MUST adhere to the DateTime format specified in Jabber Date and Time Profiles [5]).

The content of each individual message MUST be encapsulated in a <to/> or <from/> element. The time in seconds of the message relative to the start-time of the collection SHOULD be specified with a 'secs' attribute. The content SHOULD include a <body/> element. Other elements MAY be included, but they are NOT RECOMMENDED. To conserve bandwidth and storage space, elements qualified by the 'http://jabber.org/protocol/xhtml-im' namespace SHOULD NOT be included. <thread/> elements and elements qualified by the 'jabber:x:delay', 'jabber:x:event' and 'http://jabber.org/protocol/chatstates' namespaces MUST NOT be included.

Complying with XMPP Core, the server MUST respond to all <iq/> element of type 'get' or 'set'. However, most successful responses have been omitted from this document in the interest of conciseness.

All times MUST be set to UTC.

4.2 Uploading Messages to a Collection

The messages to be uploaded are encapsulated in the <store/> element.

Example 21. Storing messages in a collection

<iq type='set' to='montague.net' id='up1'>
  <store xmlns='http://jabber.org/protocol/archive'
         with='juliet@capulet.com'
         start='1469-07-21T02:56:15Z'
         subject='She speaks!'>
    <from secs='0'><body>Art thou not Romeo, and a Montague?</body></from>
    <to secs='11'><body>Neither, fair saint, if either thee dislike.</body></to>
    <from secs='14'><body>How cam'st thou hither, tell me, and wherefore?</body></from>
  </store>
</iq>
    

If the collection does not exist then the server MUST create a new collection. If the collection already exists then the server MUST append the messages to the existing collection.

A friendly name for the collection MAY be specified with a 'subject' attribute. If the collection already has a 'subject' then it is simply replaced. Note the Security Considerations regarding the subject attribute.

Example 22. Successful reply

<iq type='result' to='romeo@montague.net/orchard' id='up1'/>
    

If the server cannot service a store request because the collection is too large then it MUST return a <not-acceptable/> error:

Example 23. Unsuccessful reply

<iq type='error' to='romeo@montague.net/orchard'>
  <error code='406' type='modify'>
    <not-acceptable xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
  </error>
</iq>
    

The client MAY specify an absolute time for any message by providing a longer 'utc' attribute instead of a 'secs' attribute:

Example 24. Storing offline messages in a collection

<iq type='set' to='montague.net' id='up2'>
  <store xmlns='http://jabber.org/protocol/archive'
         with='juliet@capulet.com'
         start='1469-07-21T02:56:15Z'
         subject='She speaks!'>
    <from utc='1469-07-21T00:32:29Z'><body>Art thou not Romeo, and a Montague?</body></from>
    <to secs='11'><body>Neither, fair saint, if either thee dislike.</body></to>
    <from secs='14'><body>How cam'st thou hither, tell me, and wherefore?</body></from>
  </store>
</iq>
    

5. Archive Management

There are three main areas of functionality related to management of saved or uploaded archives:

  1. Obtaining a list of collections.
  2. Retrieving a collection.
  3. Removing a collection.

Requirements and protocol flows for these use cases are defined below.

5.1 Obtaining a List of Collections

To request a list of collections the client sends an empty <list/> element. The 'start' and 'end' attributes MAY be specified to indicate a date range (the values of these attributes MUST adhere to the DateTime format specified in JEP-0082).

If the 'with' attribute is omitted then collections with any JID are returned. If only 'start' is specified then all collections on or after that date should be returned. If only 'end' is specified then all collections prior to that date should be returned.

Example 25. Requesting a list

<iq type='get' to='montague.net'>
  <list xmlns='http://jabber.org/protocol/archive'
        with='juliet@capulet.com'
        start='1469-07-21T02:00:00Z'
        end='1479-07-21T04:00:00Z'/>
</iq>
    

Example 26. Requesting a list of all collections with a JID

<iq type='get' to='montague.net'>
  <list xmlns='http://jabber.org/protocol/archive'
        with='juliet@capulet.com'/>
</iq>
    

The client MAY limit the number of items returned by the server with the 'maxitems' attribute.

Example 27. Requesting a list of all collections

<iq type='get' to='montague.net'>
  <list xmlns='http://jabber.org/protocol/archive'
        maxitems='50'/>
</iq>
    

The collections (empty <store/> elements) in the result MUST be listed in chronological order.

Example 28. Receiving a list

<iq type='result' id='a1' to='romeo@montague.net/orchard'>
  <list xmlns='http://jabber.org/protocol/archive'>
    <store with='juliet@capulet.com'
           start='1469-07-21T02:56:15Z'
           subject='She speaks!'/>
    <store with='balcony@house.capulet.com'
           start='1469-07-21T03:16:37Z'/>
    ...
  </list>
</iq>
    

If the requested list would be too long to return in its entirety without exceeding karma limits (or any other limit specified by an administrator), then the server SHOULD only return the first part of the list. In this case the server MUST indicate that the list is incomplete by setting the optional 'partial' attribute of the <list/> element to 'true'.

Example 29. Receiving a partial list

<iq type='result' id='a1' to='romeo@montague.net/orchard'>
  <list partial='true' xmlns='http://jabber.org/protocol/archive'>
    <store with='juliet@capulet.com'
           start='1469-07-21T02:56:15Z'
           subject='She speaks!'/>
    <store with='balcony@house.capulet.com'
           start='1469-07-21T03:16:37Z'/>
    ...
  </list>
</iq>
    

If it has received a partial list, the client MAY then request the remainder of the list, taking care to set the value of the 'start' attribute to one second after the time of the last collection in the partial list that it received.

5.2 Retrieving a Collection

The client sends an empty <retrieve/> element to request the download of a collection:

Example 30. Requesting a collection

<iq type='get' to='montague.net' id='down1'>
  <retrieve xmlns='http://jabber.org/protocol/archive'
            with='juliet@capulet.com'
            start='1469-07-21T02:56:15Z'/>
</iq>
    

Example 31. Receiving a collection

<iq type='result' to='montague.net'>
  <store xmlns='http://jabber.org/protocol/archive'
         with='juliet@capulet.com'
         start='1469-07-21T02:56:15Z'
         subject='She speaks!'>
    <from secs='0'><body>Art thou not Romeo, and a Montague?</body></from>
    <to secs='11'><body>Neither, fair saint, if either thee dislike.</body></to>
    <from secs='14'><body>How cam'st thou hither, tell me, and wherefore?</body></from>
  </store>
</iq>
    

If the collection does not exist then the server MUST return an <item-not-found/> error:

Example 32. Unsuccessful reply

<iq type='error' to='romeo@montague.net/orchard'>
  <error code='404' type='cancel'>
    <item-not-found xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
  </error>
</iq>
    

If the collection would be too long to return in its entirety without exceeding karma limits (or any other limit specified by an administrator), then the server SHOULD return only the first part of the collection. In this case the server MUST indicate that the collection is incomplete by setting the optional 'partial' attribute of the <list/> element to 'true'.

Example 33. Receiving a partial collection

<iq type='result' id='a1' to='romeo@montague.net/orchard'>
  <store xmlns='http://jabber.org/protocol/archive'
         partial='true'
         start='1469-07-21T02:56:15Z'
         subject='She speaks!'
         with='juliet@capulet.com'>
    <from secs='0'><body>Art thou not Romeo, and a Montague?</body></from>
    <to secs='11'><body>Neither, fair saint, if either thee dislike.</body></to>
    <from secs='14'><body>How cam'st thou hither, tell me, and wherefore?</body></from>
  </store>
</iq>
    

If it has received a partial collection, the client MAY then request the remainder of the collection, taking care to set the value of the 'start' attribute to one second after the time of the last <from/> or <to/> element in the partial collection that it received.

5.3 Removing a Collection

To request the removal of a collection the client sends an empty <remove/> element.

Example 34. Removing a single collection

<iq type='set' to='montague.net'>
  <remove xmlns='http://jabber.org/protocol/archive'
          with='juliet@capulet.com'
          start='1469-07-21T02:56:15Z'/>
</iq>
    

If the collection does not exist then the server MUST return a Not Found error:

Example 35. Unsuccessful reply

<iq type='error' to='romeo@montague.net/orchard'>
  <error code='404' type='cancel'>
    <item-not-found xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
  </error>
</iq>
    

The client may remove several collections at once. The 'start' and 'end' elements MAY be specified to indicate a date range.

Example 36. Removing all collections with a specified JID between two times

<iq type='set' to='montague.net'>
  <remove xmlns='http://jabber.org/protocol/archive'
          with='juliet@capulet.com'
          start='1469-07-21T02:00:00Z'
          end='1469-07-21T04:00:00Z'/>
</iq>
    

If the 'with' attribute is omitted then collections with any JID are removed.

If the end date is in the future then then all collections after the start date are removed.

Example 37. Removing all collections after a date

<iq type='set' to='montague.net'>
  <remove xmlns='http://jabber.org/protocol/archive'
          start='1469-07-21T02:00:00Z'
          end='2038-01-01T00:00:00Z'/>
</iq>
    

If the start date is before all the collections in the archive then all collections prior to the end date are removed.

Example 38. Removing all collections before a date

<iq type='set' to='montague.net'>
  <remove xmlns='http://jabber.org/protocol/archive'
          start='0000-01-01T00:00:00Z'
          end='1469-07-21T04:00:00Z'/>
</iq>
    

Example 39. Removing all collections

<iq type='set' to='montague.net'>
  <remove xmlns='http://jabber.org/protocol/archive'/>
</iq>
    

6. File Format

So that clients can share archived messages, this document specifies a common format for storage on disk (similar to email formats like mbox and Maildir). The file format uses the same XML constructs as the protocol. Each file may contain messages exchanged with a single JID. Any number of items may be stored in an archive file.

Example 40. Example file

<?xml version='1.0'?>
<archive xmlns='http://jabber.org/protocol/archive'
         with='juliet@capulet.com'>
  <store start='1469-07-21T02:56:15Z' subject='She speaks!'>
    <from secs='0'><body>Art thou not Romeo, and a Montague?</body></from>
    <to secs='11'><body>Neither, fair saint, if either thee dislike.</body></to>
    <from secs='14'><body>How cam'st thou hither, tell me, and wherefore?</body></from>
  </store>
</archive>
  

7. Implementation Notes

7.1 Bandwidth Considerations

When uploading messages using manual archiving, a client SHOULD NOT store one message at a time on the server since this increases both bandwidth consumption and the total number of transactions. It is instead RECOMMENDED that clients store messages only when the conversation thread appears to be terminated, i.e. when the user closes the chat window. If the user reopens the window and the thread continues then the client should append the new messages to the collection when the user closes the window again.

7.2 Karma Considerations

When appending messages to a collection clients SHOULD try to ensure that the total size of the collection will not exceed karma limits when it is retrieved later. This may be achieved by starting a new collection whenever a message thread becomes too long.

7.3 Synchronization

It is RECOMMENDED for the client synchronize all the times it sends to the server with server time. The client can achieve this using Entity Time [6] to estimate the difference between the server and client clocks.

7.4 Storage Considerations

Server implementations SHOULD give system administrators the option to disable support for both automated and manual archiving, since archived conversations can consume significant storage space.

7.5 Groupchat Messages

A client MAY archive messages that it receives in the context of Multi-User Chat [7] rooms. The client SHOULD include a 'name' attribute to specify the room nickname of all messages:

Example 41. Storing groupchat messages in a collection

<iq type='set' to='montague.net' id='up3'>
  <store xmlns='http://jabber.org/protocol/archive'
         with='balcony@house.capulet.com'
         start='1469-07-21T03:16:37Z'>
    <from secs='0' name='benvolio'><body>She will indite him to some supper.</body></from>
    <from secs='5' name='mercutio'><body>A bawd, a bawd, a bawd! So ho!</body></from>
    <from secs='11' name='romeo'><body>What hast thou found?</body></from>
  </store>
</iq>
    

8. Security Considerations

8.1 Encryption of Archived Messages

Because a server can be compromised, it is RECOMMENDED to encrypt archived messages. Methods for doing so are under development and will be specified in a separate document or in a new section of this document.

8.2 Store Headers

The client that originates a message MAY specify a 'false' value for the 'store' header (see Stanza Headers and Internet Metadata [8]). The recipient MUST NOT archive such a message or any of the information it contains. If the sender plans to use 'store' headers it MUST use Service Discovery to determine whether or not the recipient supports them. If not, the sender MUST warn its human user (if any) before sending the message.

8.3 Subject Attributes

Since the subject of each collection is not encrypted, the client MUST warn its human user (if any) before including 'subject' attributes on encrypted collections.

9. IANA Considerations

No interaction with the Internet Assigned Numbers Authority (IANA) [9] is required as a result of this JEP.

10. Jabber Registrar Considerations

10.1 Protocol Namespaces

The Jabber Registrar [10] shall include 'http://jabber.org/protocol/archive' in its registry of protocol namespaces (see <http://www.jabber.org/registrar/namespaces.html>):

10.2 Service Discovery Features

The Jabber Registrar shall include the following features in its registry of service discovery features (see <http://www.jabber.org/registrar/disco-features.html>):

11. XML Schemas

<?xml version='1.0' encoding='UTF-8'?>

<xs:schema
    xmlns:xs='http://www.w3.org/2001/XMLSchema'
    targetNamespace='http://jabber.org/protocol/archive'
    xmlns='http://jabber.org/protocol/archive'
    elementFormDefault='qualified'>

  <xs:annotation>
    <xs:documentation>
      The allowable root element for the namespace defined
      herein are:
        - archive
        - list
        - otr
        - record
        - remove
        - retrieve
        - save
        - store
    </xs:documentation>
  </xs:annotation>

  <xs:element name='archive'>
    <xs:complexType>
      <xs:sequence>
        <xs:element ref='store' minOccurs='1' maxOccurs='unbounded'/>
      </xs:sequence>
      <xs:attribute name='with' type='xs:string' use='optional'/>
    </xs:complexType>
  </xs:element>

  <xs:element name='list'>
    <xs:complexType>
      <xs:sequence>
        <xs:element ref='store' minOccurs='0' maxOccurs='unbounded'/>
      </xs:sequence>
      <xs:attribute name='end' type='xs:dateTime' use='optional'/>
      <xs:attribute name='maxitems' type='xs:positiveInteger' use='optional'/>
      <xs:attribute name='partial' type='xs:boolean' use='optional'/>
      <xs:attribute name='start' type='xs:dateTime' use='optional'/>
      <xs:attribute name='with' type='xs:string' use='optional'/>
    </xs:complexType>
  </xs:element>

  <xs:element name='otr'>
    <xs:complexType>
      <xs:sequence>
        <xs:element ref='record' minOccurs='0' maxOccurs='unbounded'/>
      </xs:sequence>
    </xs:complexType>
  </xs:element>

  <xs:element name='record'>
    <xs:complexType>
      <xs:simpleContent>
        <xs:extension base='empty'>
          <xs:attribute name='jid' use='optional' type='xs:string'/>
          <xs:attribute name='otr' use='required' type='xs:boolean'/>
        </xs:extension>
      </xs:simpleContent>
    </xs:complexType>
  </xs:element>

  <xs:element name='remove'>
    <xs:complexType>
      <xs:simpleContent>
        <xs:extension base='empty'>
          <xs:attribute name='end' type='xs:dateTime' use='optional'/>
          <xs:attribute name='start' type='xs:dateTime' use='required'/>
          <xs:attribute name='with' type='xs:string' use='required'/>
        </xs:extension>
      </xs:simpleContent>
    </xs:complexType>
  </xs:element>

  <xs:element name='retrieve'>
    <xs:complexType>
      <xs:simpleContent>
        <xs:extension base='empty'>
          <xs:attribute name='start' type='xs:dateTime' use='required'/>
          <xs:attribute name='with' type='xs:string' use='required'/>
        </xs:extension>
      </xs:simpleContent>
    </xs:complexType>
  </xs:element>

  <xs:element name='save'>
    <xs:complexType>
      <xs:sequence>
        <xs:element ref='default' minOccurs='0' maxOccurs='1'/>
        <xs:element ref='item' minOccurs='0' maxOccurs='unbounded'/>
      </xs:sequence>
    </xs:complexType>
  </xs:element>

  <xs:element name='store'>
    <xs:complexType>
      <xs:choice maxOccurs='unbounded'>
        <xs:element name='from' type='messageType'/>
        <xs:element name='to' type='messageType'/>
      </xs:choice>
      <xs:attribute name='partial' type='xs:boolean' use='optional'/>
      <xs:attribute name='start' type='xs:dateTime' use='required'/>
      <xs:attribute name='subject' type='xs:string' use='optional'/>
      <xs:attribute name='with' type='xs:string' use='required'/>
    </xs:complexType>
  </xs:element>

  <xs:element name='default'>
    <xs:complexType>
      <xs:simpleContent>
        <xs:extension base='empty'>
          <xs:attribute name='save' use='required' type='xs:boolean'/>
        </xs:extension>
      </xs:simpleContent>
    </xs:complexType>
  </xs:element>

  <xs:element name='item'>
    <xs:complexType>
      <xs:simpleContent>
        <xs:extension base='empty'>
          <xs:attribute name='jid' use='required' type='xs:string'/>
          <xs:attribute name='save' use='required' type='xs:boolean'/>
        </xs:extension>
      </xs:simpleContent>
    </xs:complexType>
  </xs:element>

  <xs:complexType name='messageType'>
    <xs:sequence>
      <xs:element name='body' type='xs:string' maxOccurs='unbounded'/>
    </xs:sequence>
    <xs:attribute name='secs' type='xs:nonNegativeInteger' use='required'/>
  </xs:complexType>

  <xs:simpleType name='empty'>
    <xs:restriction base='xs:string'>
      <xs:enumeration value=''/>
    </xs:restriction>
  </xs:simpleType>

</xs:schema>
  


Notes

1. JEP-0030: Service Discovery <http://www.jabber.org/jeps/jep-0030.html>.

2. JEP-0045: Multi-User Chat <http://www.jabber.org/jeps/jep-0045.html>.

3. JEP-0131: Stanza Headers and Internet Metadata <http://www.jabber.org/jeps/jep-0131.html>.

4. JEP-0116: Encrypted Sessions <http://www.jabber.org/jeps/jep-0116.html>.

5. JEP-0082: Jabber Date and Time Profiles <http://www.jabber.org/jeps/jep-0082.html>.

6. JEP-0090: Entity Time <http://www.jabber.org/jeps/jep-0090.html>.

7. JEP-0045: Multi-User Chat <http://www.jabber.org/jeps/jep-0045.html>.

8. JEP-0131: Stanza Headers and Internet Metadata <http://www.jabber.org/jeps/jep-0131.html>.

9. The Internet Assigned Numbers Authority (IANA) is the central coordinator for the assignment of unique parameter values for Internet protocols, such as port numbers and URI schemes. For further information, see <http://www.iana.org/>.

10. The Jabber Registrar maintains a list of reserved Jabber protocol namespaces as well as registries of parameters used in the context of protocols approved by the Jabber Software Foundation. For further information, see <http://www.jabber.org/registrar/>.


Revision History

Version 0.5 (2006-05-03)

Integrated text from server-side archiving proposal; added partial support to collection retrieval; harmonized XML formats and namespaces; defined Jabber Registrar considerations and XML schema.

(psa/jp/ip/jk)

Version 0.4 (2005-12-21)

Added Replication and Searching section, partial attribute; minor improvements

(ip)

Version 0.3 (2005-10-21)

Added more examples to Removing Collections

(ip)

Version 0.2 (2005-04-18)

Complete rewrite.

(ip)

Version 0.1 (2004-06-04)

Initial version.

(jk)


END