JEP-0059: Result Set Manipulation

This document defines an XMPP extension to enable more advanced search functionality.

NOTICE: This JEP is currently within Last Call or under consideration by the Jabber Council for advancement to the next stage in the JSF standards process. For further details, visit <http://www.jabber.org/council/queue.shtml>.

JEP Information

Status: Proposed
Type: Standards Track
Number: 0059
Version: 0.5
Last Updated: 2006-05-02
JIG: Standards JIG
Approving Body: Jabber Council
Dependencies: None
Supersedes: None
Superseded By: None
Short Name: rsm
Wiki Page: <http://wiki.jabber.org/index.php/Result Set Manipulation (JEP-0059)>

Author Information

Jean-Louis Seguineau

Email: jls@antepo.com
JID: jlseguineau@im.antepo.com

Peter Saint-Andre

Email: stpeter@jabber.org
JID: stpeter@jabber.org

Valerie Mercier

Email: valerie.mercier@francetelecom.com
JID: vmercier@jabber.com

Legal Notice

This Jabber Enhancement Proposal is copyright 1999 - 2006 by the Jabber Software Foundation (JSF) and is in full conformance with the JSF's Intellectual Property Rights Policy <http://www.jabber.org/jsf/ipr-policy.shtml>. This material may be distributed only subject to the terms and conditions set forth in the Creative Commons Attribution License (<http://creativecommons.org/licenses/by/2.5/>).

Discussion Venue

The preferred venue for discussion of this document is the Standards-JIG discussion list: <http://mail.jabber.org/mailman/listinfo/standards-jig>.

Relation to XMPP

The Extensible Messaging and Presence Protocol (XMPP) is defined in the XMPP Core (RFC 3920) and XMPP IM (RFC 3921) specifications contributed by the Jabber Software Foundation to the Internet Standards Process, which is managed by the Internet Engineering Task Force in accordance with RFC 2026. Any protocol defined in this JEP has been developed outside the Internet Standards Process and is to be understood as an extension to XMPP rather than as an evolution, development, or modification of XMPP itself.

Conformance Terms

The following keywords as used in this document are to be interpreted as described in RFC 2119: "MUST", "SHALL", "REQUIRED"; "MUST NOT", "SHALL NOT"; "SHOULD", "RECOMMENDED"; "SHOULD NOT", "NOT RECOMMENDED"; "MAY", "OPTIONAL".

Table of Contents

1. Introduction
2. Use Cases
2.1. Getting the Item Count
2.2. Limiting the Size of a Result Set
2.3. Paging Through a Result Set
3. Examples
4. Determining Support
5. Implementation Notes
6. Security Considerations
7. IANA Considerations
8. Jabber Registrar Considerations
8.1. Protocol Namespaces
9. XML Schema
10. Acknowledgements
Notes
Revision History

1. Introduction

In both Jabber Search [1] and Service Discovery [2], it is possible to receive large result sets in response to information requests (e.g., a user directory search on a common first name or a service discovery items request sent to a Multi-User Chat [3] service). Thus it would be helpful to define a protocol extension that enables the following functionality:

  1. Discover the size of a potential result set without retrieving the items themselves.
  2. Limit the size of an actual result set.
  3. Page through a result set by retrieving the items in smaller subsets.

This document defines just such a protocol extension.

2. Use Cases

2.1 Getting the Item Count

In order to get the item count of a result set without retrieving the items themselves, the requesting entity specifies a request type of "count":

Example 1. Requesting the Item Count

<iq type='get' from='stpeter@jabber.org/roundabout' to='users.jabber.org' id='count1'>
  <query xmlns='jabber:iq:search'>
    <first>Peter</first>
    <count xmlns='http://jabber.org/protocol/rsm'/>
  </query>
</iq>

The responding entity then returns the item count, which MAY be approximate rather than precise (since determining the exact number of items may be resource-intensive):

Example 2. Returning the Item Count

<iq type='result' from='users.jabber.org' to='stpeter@jabber.org/roundabout' id='count1'>
  <query xmlns='jabber:iq:search'>
    <count xmlns='http://jabber.org/protocol/rsm'>800</count>
  </query>
</iq>

If no items match, the responding entity MUST return a response that adheres to the definition of the wrapper protocol (e.g., "jabber:iq:search" or "http://jabber.org/protocol/disco#items"); for both JEP-0055 and JEP-0030, that means the responding entity MUST return an empty <query/> element.

2.2 Limiting the Size of a Result Set

In order to limit the number of items to be returned in a result set, the requesting entity specifies a request type of "set" and the maximum size of the desired result set:

Example 3. Requesting a Limit to the Result Set

<iq type='get' from='stpeter@jabber.org/roundabout' to='users.jabber.org' id='limit1'>
  <query xmlns='jabber:iq:search'>
    <first>Peter</first>
    <set xmlns='http://jabber.org/protocol/rsm'>
      <max>10</max>
    </set>
  </query>
</iq>

The responding entity then returns a result set limited to the requested size:

Example 4. Returning a Limited Result Set

<iq type='result' from='users.jabber.org' to='stpeter@jabber.org/roundabout' id='limit1'>
  <query xmlns='jabber:iq:search'>
    <item jid='stpeter@jabber.org'>
      <first>Peter</first>
      <last>Saint-Andre</last>
      <nick>stpeter</nick>
    </item>
    <item jid='pgmillard@jabber.org'>
      <first>Peter</first>
      <last>Millard</last>
      <nick>pgmillard</nick>
    </item>
    .
    [8 more items]
    .
  </query>
</iq>

2.3 Paging Through a Result Set

In order to page through a result set, the requesting entity specifies a request type of "set" as well as the item to start with (initially "0") as well as the maximum number of items to return:

Example 5. Requesting the First Page of a Result Set

<iq type='get' from='stpeter@jabber.org/roundabout' to='users.jabber.org' id='page1'>
  <query xmlns='jabber:iq:search'>
    <first>Peter</first>
    <set xmlns='http://jabber.org/protocol/rsm'>
      <max>10</max>
      <start>0</start>
    </set>
  </query>
</iq>

The responding entity then returns the first page of the result set:

Example 6. Returning the First Page of a Result Set

<iq type='result' from='users.jabber.org' to='stpeter@jabber.org/roundabout' id='page1'>
  <query xmlns='jabber:iq:search'>
    <item jid='stpeter@jabber.org'>
      <first>Peter</first>
      <last>Saint-Andre</last>
      <nick>stpeter</nick>
    </item>
    <item jid='pgmillard@jabber.org'>
      <first>Peter</first>
      <last>Millard</last>
      <nick>pgmillard</nick>
    </item>
    .
    [8 more items]
    .
    <set xmlns='http://jabber.org/protocol/rsm'>
      <max>10</max>
      <start>0</start>
    </set>
    <count xmlns='http://jabber.org/protocol/rsm'>800</count>
  </query>
</iq>

As shown in the previous example, the responding entity MAY include the item count with the first result set (or any future result set, although returning the count with the first result set is probably most helpful).

Naturally, the requesting entity can then ask for the next page in the result set:

Example 7. Requesting the Second Page of a Result Set

<iq type='get' from='stpeter@jabber.org/roundabout' to='users.jabber.org' id='page2'>
  <query xmlns='jabber:iq:search'>
    <first>Peter</first>
    <set xmlns='http://jabber.org/protocol/rsm'>
      <max>10</max>
      <start>10</start>
    </set>
  </query>
</iq>

The responding entity then returns the second page of the result set:

Example 8. Returning the Second Page of a Result Set

<iq type='result' from='users.jabber.org' to='stpeter@jabber.org/roundabout' id='page2'>
  <query xmlns='jabber:iq:search'>
    <item jid='peter@pixyland.org'>
      <first>Peter</first>
      <last>Pan</last>
      <nick>peterpan</nick>
    </item>
    <item jid='peter@rabbit.lit'>
      <first>Peter</first>
      <last>Rabbit</last>
      <nick>prabbit</nick>
    </item>
    .
    [8 more items]
    .
    <set xmlns='http://jabber.org/protocol/rsm'>
      <max>10</max>
      <start>10</start>
    </set>
  </query>
</iq>

If the result set contains the last items that meet the provided criteria, the responding entitity SHOULD include an empty <end/> element:

Example 9. Returning the Last Page of a Result Set

<iq type='result' from='users.jabber.org' to='stpeter@jabber.org/roundabout' id='lastpage'>
  <query xmlns='jabber:iq:search'>
    [the last items]
    <set xmlns='http://jabber.org/protocol/rsm'>
      <max>10</max>
      <start>10</start>
      <end/>
    </set>
  </query>
</iq>

3. Examples

The foregoing examples show the use of result set manipulation in the context of JEP-0055: Jabber Search.. Therefore in the following examples we show the use of result set manipulation in the context of JEP-0030: Service Discovery.

Example 10. Requesting the Item Count

<iq type='get' from='stpeter@jabber.org/roundabout' to='conference.jabber.org' id='ex1'>
  <query xmlns='http://jabber.org/protocol/disco#items'>
    <count xmlns='http://jabber.org/protocol/rsm'/>
  </query>
</iq>

Example 11. Returning the Item Count

<iq type='result' from='conference.jabber.org' to='stpeter@jabber.org/roundabout' id='ex1'>
  <query xmlns='http://jabber.org/protocol/disco#items'>
    <count xmlns='http://jabber.org/protocol/rsm'>150</count>
  </query>
</iq>

Example 12. Requesting a Limit to the Result Set

<iq type='get' from='stpeter@jabber.org/roundabout' to='conference.jabber.org' id='ex2'>
  <query xmlns='http://jabber.org/protocol/disco#items'>
    <set xmlns='http://jabber.org/protocol/rsm'>
      <max>20</max>
    </set>
  </query>
</iq>

Example 13. Returning a Limited Result Set

<iq type='result' from='users.jabber.org' to='stpeter@jabber.org/roundabout' id='ex2'>
  <query xmlns='http://jabber.org/protocol/disco#items'>
    <item jid='12@conference.jabber.org'/>
    <item jid='adium@conference.jabber.org'/>
    <item jid='airhitch@conference.jabber.org'/>
    <item jid='alphaville@conference.jabber.org'/>
    <item jid='apatche@conference.jabber.org'/>
    <item jid='argia@conference.jabber.org'/>
    <item jid='armagetron@conference.jabber.org'/>
    <item jid='atticroom123@conference.jabber.org'/>
    <item jid='banquise@conference.jabber.org'/>
    <item jid='bar_paradise@conference.jabber.org'/>
    <item jid='beer@conference.jabber.org'/>
    <item jid='blondie@conference.jabber.org'/>
    <item jid='bpnops@conference.jabber.org'/>
    <item jid='brasileiros@conference.jabber.org'/>
    <item jid='bulgaria@conference.jabber.org'/>
    <item jid='cantinalivre@conference.jabber.org'/>
    <item jid='casablanca@conference.jabber.org'/>
    <item jid='chinortpcrew@conference.jabber.org'/>
    <item jid='coffeetalk@conference.jabber.org'/>
    <item jid='council@conference.jabber.org'/>
  </query>
</iq>

4. Determining Support

In order for a requesting entity to determine if a responding entity supports result set manipulation, it SHOULD send a Service Discovery information request to the responding entity:

Example 14. Requesting entity queries responding entity regarding protocol support

<iq from='stpeter@jabber.org/roundabout'
    to='conference.jabber.org'
    id='disco1'>
  <query xmlns='http://jabber.org/protocol/disco#info'/>
</iq>

Example 15. Responding entity communicates protocol support

<iq from='conference.jabber.org'
    to='stpeter@jabber.org/roundabout'
    id='disco1'>
  <query xmlns='http://jabber.org/protocol/disco#info'>
    ...
    <feature var='http://jabber.org/protocol/rsm'/>
    ...
  </query>
</iq>

A requesting entity SHOULD NOT include result set manipulation extensions if it does not have positive knowledge that the responding entity supports the protocol defined herein. If the responding entity does not understand result set manipulation, it MUST ignore the extension.

Note: Even if a responding entity understands the result set manipulation protocol, its support for result set management in the context of any given using protocol is OPTIONAL (e.g., an implementation could support it in the context of the 'jabber:iq:search' namespace but not in the context of the 'http://jabber.org/protocol/disco#items' namespace). Currently the only way for a requesting entity to determine if a responding entity supports result set management in the context of a given using protocol is to include result set management extensions in its request. If the responding entity does not include result set management extensions in its response, then the requesting entity SHOULD NOT include such extensions in future requests wrapped by the using protocol namespace.

5. Implementation Notes

If a search is performed over a live store (e.g., a list of chatrooms, where new rooms can be created while a search is ongoing), the number of items in the result set might change between between any two pages. One possible way to handle this is for the responding entity to send some state to the requesting entity in each response, which the requesting entity sends back with each subsequent request. Methods for doing so are currently out of scope but may addressed in a future version of this specification.

6. Security Considerations

Security considerations are the responsibility of the using ("wrapper") protocol, such as JEP-0030 for the 'http://jabber.org/protocol/disco#items' namespace and JEP-0055 for the 'jabber:iq:search' namespace.

7. IANA Considerations

This JEP requires no interaction with the Internet Assigned Numbers Authority (IANA) [4].

8. Jabber Registrar Considerations

8.1 Protocol Namespaces

The Jabber Registrar [5] shall include 'http://jabber.org/protocol/rsm' in its registry of protocol namespaces.

9. XML Schema 

<?xml version='1.0' encoding='UTF-8'?>

<xs:schema
    xmlns:xs='http://www.w3.org/2001/XMLSchema'
    targetNamespace='http://jabber.org/protocol/rsm'
    xmlns='http://jabber.org/protocol/rsm'
    elementFormDefault='qualified'>

  <xs:element name='count' type='xs:string'/>

  <xs:element name='set'>
    <xs:complexType>
      <xs:sequence>
        <xs:element name='max' type='xs:int' minOccurs='1' maxOccurs='1'/>
        <xs:element name='start' type='xs:int' minOccurs='0' maxOccurs='1'/>
        <xs:element name='end' type='empty' minOccurs='0' maxOccurs='1'/>
      </xs:sequence>
    </xs:complexType>
  </xs:element>

  <xs:simpleType name='empty'>
    <xs:restriction base='xs:string'>
      <xs:enumeration value=''/>
    </xs:restriction>
  </xs:simpleType>

</xs:schema>

10. Acknowledgements

Thanks to Jon Perlow for his feedback.

Notes

1. JEP-0055: Jabber Search <http://www.jabber.org/jeps/jep-0055.html>.

2. JEP-0030: Service Discovery <http://www.jabber.org/jeps/jep-0030.html>.

3. JEP-0045: Multi-User Chat <http://www.jabber.org/jeps/jep-0045.html>.

4. The Internet Assigned Numbers Authority (IANA) is the central coordinator for the assignment of unique parameter values for Internet protocols, such as port numbers and URI schemes. For further information, see <http://www.iana.org/>.

5. The Jabber Registrar maintains a list of reserved Jabber protocol namespaces as well as registries of parameters used in the context of protocols approved by the Jabber Software Foundation. For further information, see <http://www.jabber.org/registrar/>.

Revision History

Version 0.5 (2006-05-02)

Clarified error handling, determination of support in the context of using protocols, and security considerations.

(psa)

Version 0.4 (2006-04-24)

Specified that an item count may be approximate; specified that an item count may be returned with a page of results.

(psa)

Version 0.3 (2006-04-21)

Added <end/> element to specify last result set; added service discovery information; added more examples.

(psa/vm)

Version 0.2 (2005-12-22)

Revived and renamed the JEP; modified syntax; added use case for getting number of items; defined XML schema.

(psa/vm)

Version 0.1 (2002-11-12)

Initial version.

(jls)

END