XEP-0433: Extended Channel Search

1. Introduction & Motivation¶

The XMPP instant messaging ecosystem is a federated one. This leads to many different group chat service providers existing and interesting public group chats being spread out across them. In order to provide users with a way to find public group chats (henceforth called channels) of interest to them, there needs to be a way to execute a cross-domain search based on keywords.

The protocol in this document provides a general and extensible search for channels across different domains and service types (e.g. MUC vs. MIX). It provides meta-information right in the result set, which allows searching entities to skip additional Service Discovery (XEP-0030) [1] queries against the channels themselves.

The protocol is not only useful for cross-domain search, but also as an alternative to using a Service Discovery (XEP-0030) [1] disco#items request followed by many disco#info requests on a group chat service.

2. Requirements¶

The protocol:

• must work without state on the server side. This is to allow stateless proxies to be used for pseudonymisation or anonymisation.
• must allow searching the list using a free-text keyword-based search.
• must allow future extensions to the search query and the result.
• must allow retrieving the entire data set (although, for clarification, an operator may choose to turn this off).
• must use completely machine-readable and machine-writable data.

3. Glossary¶

4. Use Cases¶

<iq from='search.service.example' to='client@user.example' id='id1' type='result'>
  <query xmlns='http://jabber.org/protocol/disco#info'>
    <!-- ... -->
    <feature var='urn:xmpp:channel-search:0:search'/>
    <!-- ... -->
  </query>
</iq>

<iq from='client@user.example' to='search.service.example' id='id2' type='get' xml:lang='en'>
  <search xmlns='urn:xmpp:channel-search:0:search'/>
</iq>

<iq from='search.service.example' to='client@user.example' id='id2' type='result' xml:lang='en'>
  <search xmlns='urn:xmpp:channel-search:0:search'>
    <x xmlns='jabber:x:data' type='form'>
      <field type='hidden' var='FORM_TYPE'>
        <value>urn:xmpp:channel-search:0:search-params</value>
      </field>
      <field type='text-single' var='q' label='Search for'/>
      <field type='boolean' var='all' label='Return all entries (ignore search terms)'/>
      <field type='boolean' var='sinname' label='Search in name'>
        <value>true</value>
      </field>
      <field type='boolean' var='sindescription' label='Search in description'>
        <value>true</value>
      </field>
      <field type='boolean' var='sinaddr' label='Search in address'>
        <value>true</value>
      </field>
      <field type='text-single' var='min_users' label='Minimum number of users'>
        <value>1</value>
      </field>
      <field var="types" type="list-multi" label="Service types">
        <value>xep-0045</value>
        <option label='XEP-0045 Multi User Chats'><value>xep-0045</value></option>
        <option label='XEP-0369 MIX channels'><value>xep-0369</value></option>
      </field>
      <field type='list-single' var='key' label='Sort results by'>
        <value>{urn:xmpp:channel-search:0:order}nusers</value>
        <option label='Number of online users'><value>{urn:xmpp:channel-search:0:order}nusers</value></option>
        <option label='Address'><value>{urn:xmpp:channel-search:0:order}address</value></option>
      </field>
    </x>
</search>
</iq>

<iq from='client@user.example' to='search.service.example' id='id3' type='get' xml:lang='en'>
  <search xmlns='urn:xmpp:channel-search:0:search'>
    <set xmlns="http://jabber.org/protocol/rsm">
      <max>5</max>
    </set>
    <x xmlns="jabber:x:data" type="submit">
      <field type='hidden' var='FORM_TYPE'>
        <value>urn:xmpp:channel-search:0:search-params</value>
      </field>
      <field var="q" type="text-single" label="Search for">
        <value>xmpp.org</value>
      </field>
      <field var="key" type="list-single" label="Sort results by">
        <value>{urn:xmpp:channel-search:0:order}address</value>
        <option label='Number of online users'><value>{urn:xmpp:channel-search:0:order}nusers</value></option>
        <option label='Address'><value>{urn:xmpp:channel-search:0:order}address</value></option>
      </field>
    </x>
  </search>
</iq>

<iq from='search.service.example' to='client@user.example' id='id3' type='result' xml:lang='en'>
  <result xmlns='urn:xmpp:channel-search:0:search'>
    <item address='commteam@muc.xmpp.org'>
      <name>commteam</name>
      <nusers>10</nusers>
      <is-open/>
    </item>
    <!-- three more items -->
    <item address='operators@muc.xmpp.org'>
      <name>XMPP Service Operators</name>
      <description>Discussion venue for operators of federated XMPP services</description>
      <nusers>43</nusers>
      <is-open/>
    </item>
    <set xmlns='http://jabber.org/protocol/rsm'>
      <first>opaque-string-1</first>
      <last>opaque-string-2</last>
      <max>5</max>
    </set>
  </result>
</iq>

<iq from='search.service.example' to='client@user.example' id='id3' type='error' xml:lang='en'>
  <error type='modify'>
    <feature-not-implemented xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
    <invalid-sort-key xmlns='urn:xmpp:channel-search:0:error'/>
  </error>
</iq>

<iq from='search.service.example' to='client@user.example' id='id3' type='error' xml:lang='en'>
  <error type='modify'>
    <bad-request xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
    <text xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'>
      Search terms must have at least three characters.
    </text>
    <invalid-search-terms xmlns='urn:xmpp:channel-search:0:error'/>
  </error>
</iq>

<iq from='search.service.example' to='client@user.example' id='id3' type='error' xml:lang='en'>
  <error type='wait'>
    <resource-constraint xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
    <rate-limit xmlns='urn:xmpp:channel-search:0:error' retry-after='30'/>
  </error>
</iq>

<iq from='search.service.example' to='client@user.example' id='id3' type='error' xml:lang='en'>
  <error type='cancel'>
    <not-allowed xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
    <text xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'>
      Retrieval of the full database is not allowed.
    </text>
    <full-set-retrieval-rejected xmlns='urn:xmpp:channel-search:0:error'/>
  </error>
</iq>

<iq from='search.service.example' to='client@user.example' id='id3' type='error' xml:lang='en'>
  <error type='cancel'>
    <not-allowed xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
    <text xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'>
      Cannot both return all results and search by keywords.
    </text>
    <conflicting-fields xmlns='urn:xmpp:channel-search:0:error'>
      <var>all</var>
      <var>q</var>
    </conflicting-fields>
  </error>
</iq>

<iq from='search.service.example' to='client@user.example' id='id3' type='error' xml:lang='en'>
  <error type='modify'>
    <bad-request xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'/>
    <no-search-conditions xmlns='urn:xmpp:channel-search:0:error'/>
  </error>
</iq>

5. Business Rules¶

• When sending the form template, the Search Service MUST include all fields it supports with their respective default values.
• When submitting a form to the Search Service, a Searcher MAY omit all fields it either does not understand or it has left unchanged.
• When submitting a form to the Search Service, a Searcher MAY omit the <option/> elements.
• When receiving a search form, the Search Service MUST ignore fields with a var value it does not understand.
• When executing a keyword search, the service may process the keyword string in implementation-defined ways. This may include interpreting quotes and other "special" characters, removing keywords which do not fit internal criteria for suitability and others.
• If the Searcher receives a <rate-limit/> error, the behaviour of the Searcher depends on the retry-after attribute:
  • If the retry-after attribute is present, the Searcher MUST NOT send another search request before the amount of seconds indicated in the retry-after attribute have elapsed. There is no guarantee that the request will be accepted at that time.
  • If the retry-after attribute is not present, the Searcher should wait for an implementation-defined amount of time and SHOULD back off exponentially on each subsequent <rate-limit/> error.
• If a search request does not yield any results, the Search Service MUST reply with a <result/> without any <item/> children in a type='result' IQ. Specifically, it MUST NOT reply with an <item-not-found/> error.
• If the all field is set to true and the Search Service allows this operation, all results MUST be included in the result set (and then paginated using Result Set Management (XEP-0059) [4]).

6. Implementation Notes¶

7. Security Considerations¶

When sending a search form with a q field, the Searcher transmits potentially sensitive information to a third party.

8. IANA Considerations¶

This specification does not require any interaction with the IANA.

9. XMPP Registrar Considerations¶

This specification should probably create registries for the various fields it defines, as well as register a form type.

10. XML Schema¶

To be done.

11. Design Considerations¶

Instead of rolling a custom protocol for the result items, Jabber Search (XEP-0055) [8] could have been used.

While the result format of Jabber Search (XEP-0055) [8] allows for some generality, it does so in a rather restricted way. It is limited by the data formats and types expressable in Data Forms (XEP-0004) [9]. Sturctured data, beyond lists of text and JIDs, can not be represented with Data Forms (XEP-0004) [9] at all. Machine-readable data would also have to be human-readable at the same time to provide a fallback view for human users. Interationalization of such human-readable data in field values is not possible with Data Forms (XEP-0004) [9].

The advantage of entities being able to process unknown fields in a degraded manner is, principally, still present in the current proposal (although with a different kind of degration).

Given the complexity of fully and correctly processing Data Forms (XEP-0004) [9] report data, the slim benefits did, in the eyes of the authors, not outweigh the costs.

12. Acknowledgements¶

The basis for this protocol was developed for the search.jabber.network public group chat search service. It has been cleaned up for publication as a Standards Track XEP by the author and modified to support more use-cases.

Appendices

Appendix A: Document Information¶

Series

XEP

Number

0433

Publisher

XMPP Standards Foundation

Status

Deferred

Type

Standards Track

Version

0.1.0

Last Updated

2020-02-27

Approving Body

XMPP Council

Dependencies

XMPP Core, XEP-0004, XEP-0030, XEP-0059, XEP-0068

Supersedes

None

Superseded By

None

Short Name

ECS

Source Control

HTML

This document in other formats: XML  PDF

Appendix B: Author Information¶

Jonas Schäfer

Email

jonas@wielicki.name

JabberID

jonas@wielicki.name

Appendix C: Legal Notices¶

Appendix D: Relation to XMPP¶

The Extensible Messaging and Presence Protocol (XMPP) is defined in the XMPP Core (RFC 6120) and XMPP IM (RFC 6121) specifications contributed by the XMPP Standards Foundation to the Internet Standards Process, which is managed by the Internet Engineering Task Force in accordance with RFC 2026. Any protocol defined in this document has been developed outside the Internet Standards Process and is to be understood as an extension to XMPP rather than as an evolution, development, or modification of XMPP itself.

Appendix E: Discussion Venue¶

The primary venue for discussion of XMPP Extension Protocols is the <standards@xmpp.org> discussion list.

Discussion on other xmpp.org discussion lists might also be appropriate; see <https://xmpp.org/community/> for a complete list.

Errata can be sent to <editor@xmpp.org>.

Appendix F: Requirements Conformance¶

The following requirements keywords as used in this document are to be interpreted as described in RFC 2119: "MUST", "SHALL", "REQUIRED"; "MUST NOT", "SHALL NOT"; "SHOULD", "RECOMMENDED"; "SHOULD NOT", "NOT RECOMMENDED"; "MAY", "OPTIONAL".

Appendix G: Notes¶

Appendix H: Revision History¶

Note: Older versions of this specification might be available at https://xmpp.org/extensions/attic/

1. Version 0.1.0 (2020-02-27)¶
   Accepted by vote of Council on 2020-02-26.
   XEP Editor (jsc)
2. Version 0.0.1 (2020-02-19)¶

   First draft.

   jsc

Appendix I: Bib(La)TeX Entry

@report{schäfer2020ecs,
  title = {Extended Channel Search},
  author = {Schäfer, Jonas},
  type = {XEP},
  number = {0433},
  version = {0.1.0},
  institution = {XMPP Standards Foundation},
  url = {https://xmpp.org/extensions/xep-0433.html},
  date = {2020-02-19/2020-02-27},
}

END