XEP-xxxx: Pubsub Caching Hints

Abstract: This specification provides a way to get caching information from a Pubsub node
Author: Jérôme Poisson
Copyright: © 2021 – 2021 XMPP Standards Foundation. SEE LEGAL NOTICES.
Status: ProtoXEP
WARNING: This document has not yet been accepted for consideration or approved in any official manner by the XMPP Standards Foundation, and this document is not yet an XMPP Extension Protocol (XEP). If this document is accepted as a XEP by the XMPP Council, it will be published at <https://xmpp.org/extensions/> and announced on the <standards@xmpp.org> mailing list.
Type: Standards Track
Version: 0.0.1 (2021-07-19)

Document Lifecycle

Experimental
Proposed
Stable
Final

1. Introduction¶

XMPP Pubsub as described in Publish-Subscribe (XEP-0060) [1] is a very powerful and versatile tool, which is used for numerous XMPP features. For many reasons, notably speed improvement and resources optimisation, XMPP clients may want to cache internally Pubsub nodes and keep cache synchronised with cached pubsub service. Unfortunately the flexibility of Pubsub makes the choice of a good caching strategy complicated and non optimal. This XEP standardize a way for the Pubsub service to give extra information to fix this situation.

2. Requirements¶

Caching information must be using base XEP-0060 Pubsub features and be easy to obtain for the client, and easy to add for the pubsub service. The desired goals are:

3. Glossary¶

4. Caching Hints Discovery¶

Hints are advertised using well known fields in Pubsub metadata disco extension. If a Pubsub service implements this XEP, and if it also manages a Personal Eventing Protocol (XEP-0163) [2] service, the fields described here MUST be present for both Pubsub and PEP nodes.

5. Node Persistence¶

To properly cache a node, a client must known if they are kept in storage or not. To advertise that fact, a Pubsub node compliant with this standard MUST use a field named {urn:xmpp:pubsub-caching:0}persistence of type list-single and whose value can be one of:

6. Max Items and Item Expire¶

It may be necessary to cache a node, to know how many items the Pubsub Service is keeping before silently deleting them, or when they do expire. This is done by advertising it using the pubsub#max_items and pubsub#item_expire fields of type integer-or-max (this type is defined in XEP-0060).

7. Consistent Items: items are always identical for all users¶

Pubsub is most often used to let allowed users store and retrieve unmodified items. However, nothing in Publish-Subscribe (XEP-0060) [1] prevents a Pubsub service to return dynamic items with identical IDs depending on factors like the JID of the requestor, time of the day, or something else. For instance, a weather service could return local prevision for following day using item identifier like "tomorrow_forecast", or a machine learning algorithm could return favorite Shakespeare books to a user using "favorite" item id, in which case romeo@montaigu.lit would probably have different results than juliet@capulet.lit

For obvious reason those items are hard to impossible to cache. Pubsub services SHOULD avoid using dynamic items, unless there is a really good reason for it.

If a Pubsub service implements Caching Hints and if items are static (i.e. items with the same ID are identical whatever user is requesting them, time of day it is, or any other variable parameter), then it MUST advertise this fact by using the field named {urn:xmpp:pubsub-caching:0}consistent-items of type boolean with the value of true.

Note that overwriting an items as specified in XEP-0060 Publish an Item to a Node is a normal Pubsub use case which SHOULD result in proper notifications being sent to subscribers, if the item is not otherwise different, this is considered as static item and MUST result in a value of true for the {urn:xmpp:pubsub-caching:0}consistent-items field.

Respectively, if a Pubsub node delivers one or more dynamic items, it MUST avertise the fact by using the value of false for the same field.

8. Consistent Set: All Users Have the Same Series of Items¶

A Pubsub service may have a feature to restrict individual items from a node to some entities (e.g. to have some items only visible to family, friends or coworkers). In this case, we say that the node has inconsistent items set, and this implies that cache must not be shared between users (as some users may have access to some items that other don't).

If a Pubsub node is always returning the same items ids to all allowed users, it MUST advertise this fact by using the value true for the boolean field {urn:xmpp:pubsub-caching:0}consistent-set.

On the other hand, if a Pubsub node may return different items according to the requesting entities (assuming that entities are allowed at the node level), it MUST advertise this fact by using the value false for the same field.

9. Stable Items: Items Won't Appear Out of Order¶

Normally, items are managed like a queue in a node, i.e. new items are appended to one ends, and existing items can only be deleted (or overwritten, in which case an item with the same ID is appended to the end). However, it may be necessary for a Pubsub service to include items out of order (i.e. not appending it at the end), for instance when a Pubsub service is a bridge to a third party protocol which receives items out of order.

Unstable items doesn't change the fact that node can be shared between users or node, but it may have an impact on client implementation, as the caching implementation may have items in a different order, and items may be missed, thus this fact is valuable to know for a client willing to cache the node. Note that unstable items SHOULD be avoided by a Pubsub service whenever it's possible.

If items are always appended to the end of the queue, the Pubsub node MUST advertise this fact using the {urn:xmpp:pubsub-caching:0}stable-items field of type boolean with a value of true.

On the other hand, if items order can't be guaranteed, the Pubsub node MUST advertise this fact by using the value of false for the same field

10. Always Notify¶

To synchronize correctly a Pubsub node, an XMPP client must be aware of any modification that happen to its items or to the node itself. This is possible thanks to the subscription mechanism of Publish-Subscribe (XEP-0060) [1]. However, notification can be skipped, notably item retraction notification must be explicitly requested by the client, and thus may be missing, resulting in cache becoming out of sync with the Pubsub service

To avoid that, the Pubsub service may enforce notifications for all modifying events to a node or its items, even if they are not explicitly requested by the user doing the modification.

If a node always sends notification, including <retact> notifications even if notify attribute is not set, then it must advertise this fact using the field named {urn:xmpp:pubsub-caching:0}alway-notify of type boolean with the value of true. If notifications may be omitted, then the same field must be used with the value of false.

A Pubsub service allowing a node to have notifications always sent SHOULD allow the node owners to activate or deactivate this feature through node configuration, using the well-known field with the same name of {urn:xmpp:pubsub-caching:0}always-notify and the same type of boolean. It's up to the implementation to determine if the default value should be true or false.

11. Public Node¶

If the node is public, i.e. if it as an open access model that means that a client can safely share the cache between users (providing that the consistent items and consistent set fields are also both true).

If a Pubsub node is public, it MUST advertise this fact by exposing it in its metadata using the field named pubsub#access_model of type list-single with a value of open

If the node has an other access model, it is up to the Pubsub implementation to advertise publicly or not this data in the node metadata. It may be a privacy concern to expose any other access model than open

12. The Node can be used for Suggestions¶

An XMPP client may have a feature to suggest new Pubsub nodes to users (for network exploration, and let users find rapidly interesting content). If the discovery feature is not restricted to some users somehow, this SHOULD be done using only public nodes. But even for a public node, the node owner may no be willing to have they node suggested to random users.

To avoid using inappropriately a public node for suggestion, a Pubsub node MUST announce if the node is usable for suggestion or node by using the field named {urn:xmpp:pubsub-caching:0}allowed-for-suggestions of type boolean. It is up to the Pubsub implementation to decide how this field is set, but it SHOULD have a default value of false and it should be modifiable by node owner through node configuration, using the well-known field with the same name of {urn:xmpp:pubsub-caching:0}allowed-for-suggestions and the same type of boolean.

A client MUST NOT use a node with {urn:xmpp:pubsub-caching:0}allowed-for-suggestions set to false for suggestions

13. Purging a Node¶

To make it explicit, a client implementing this specification MUST use the field named {urn:xmpp:pubsub-caching:0}purge-keep-last-item of type boolean with the value of true if the last item is NOT retracted when a node purge is performed. On the opposite, the value of true MUST be used if ALL items are retracted when a node purge is performed, actually leaving the node empty, with no item at all.

14. Summary¶

Here a is a table summarizing all fields to announce when implementing this XEP. All fields but pubsub#access_model are mandatory if a Pubsub service advertise support for this XEP.

name	field	type	meaning	comment
Max Items	pubsub#max_items	integer-or-max	How many items are kept in storage
Item Expire	pubsub#item_expire	integer-or-max	How many seconds items are kept
Node Persistence	{urn:xmpp:pubsub-caching:0}persistence	list-single	Items are kept in persistent storage
Consistent Items	{urn:xmpp:pubsub-caching:0}consistent-items	boolean	Items are static
Consistent Set	{urn:xmpp:pubsub-caching:0}consistent-set	boolean	All users have the same items
Stable Items	{urn:xmpp:pubsub-caching:0}stable-items	boolean	Items are not added out of order
Always Notify	{urn:xmpp:pubsub-caching:0}always-notify	boolean	Modifying Notifications are Always sent to subscribers, even if not explicitly requested by publisher.
Public Node	pubsub#access_model	list-single	Items can be retrieved by anybody	if value is not `open`, it may be omitted
Allowed for Suggestions	{urn:xmpp:pubsub-caching:0}allowed-for-suggestions	boolean	Node and its items can be suggested to random users	SHOULD be settable by node owner, and MUST default to `false`
Purge Keep Last Item	{urn:xmpp:pubsub-caching:0}purge-keep-last-item	boolean	Last item is always kept when a node purge is performed

15. Example¶

Here is an example of Pubsub metadata advertised by a node on a service implementing this XEP. This example is a "happy path", i.e. features announced here are cache friendly.

16. discovering support¶

If a server supports the "Pubsub Caching Hints" protocol, it must advertize it by including the "urn:xmpp:pubsub-caching:0" discovery feature (see Protocol Namespaces regarding issuance of one or more permanent namespaces) in response to a Service Discovery (XEP-0030) [3] information request:

17. Implementation Notes¶

As order of insertion and overwriting of items may be relevant to the client, it is recommended for caching-friendly Pubsub service to implement Order-By (XEP-0413) [4], thus client can cache items using an order by date of creation.

18. Security Considerations¶

19. IANA Considerations¶

20. XMPP Registrar Considerations¶

21. XML Schema¶

Appendices

Appendix A: Document Information¶

Appendix B: Author Information¶

Jérôme Poisson

Appendix C: Legal Notices¶

Copyright

Permissions

Permission is hereby granted, free of charge, to any person obtaining a copy of this specification (the "Specification"), to make use of the Specification without restriction, including without limitation the rights to implement the Specification in a software program, deploy the Specification in a network service, and copy, modify, merge, publish, translate, distribute, sublicense, or sell copies of the Specification, and to permit persons to whom the Specification is furnished to do so, subject to the condition that the foregoing copyright notice and this permission notice shall be included in all copies or substantial portions of the Specification. Unless separate permission is granted, modified works that are redistributed shall not contain misleading information regarding the authors, title, number, or publisher of the Specification, and shall not claim endorsement of the modified works by the authors, any organization or project to which the authors belong, or the XMPP Standards Foundation.

Disclaimer of Warranty

## NOTE WELL: This Specification is provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. ##

Limitation of Liability

In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall the XMPP Standards Foundation or any author of this Specification be liable for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising from, out of, or in connection with the Specification or the implementation, deployment, or other use of the Specification (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if the XMPP Standards Foundation or such author has been advised of the possibility of such damages.

IPR Conformance

This XMPP Extension Protocol has been contributed in full conformance with the XSF's Intellectual Property Rights Policy (a copy of which can be found at <https://xmpp.org/about/xsf/ipr-policy> or obtained by writing to XMPP Standards Foundation, P.O. Box 787, Parker, CO 80134 USA).

Visual Presentation

The HTML representation (you are looking at) is maintained by the XSF. It is based on the YAML CSS Framework, which is licensed under the terms of the CC-BY-SA 2.0 license.

Appendix D: Relation to XMPP¶

The Extensible Messaging and Presence Protocol (XMPP) is defined in the XMPP Core (RFC 6120) and XMPP IM (RFC 6121) specifications contributed by the XMPP Standards Foundation to the Internet Standards Process, which is managed by the Internet Engineering Task Force in accordance with RFC 2026. Any protocol defined in this document has been developed outside the Internet Standards Process and is to be understood as an extension to XMPP rather than as an evolution, development, or modification of XMPP itself.

Appendix E: Discussion Venue¶

The primary venue for discussion of XMPP Extension Protocols is the <standards@xmpp.org> discussion list.

Discussion on other xmpp.org discussion lists might also be appropriate; see <https://xmpp.org/community/> for a complete list.

Appendix F: Requirements Conformance¶

The following requirements keywords as used in this document are to be interpreted as described in RFC 2119: "MUST", "SHALL", "REQUIRED"; "MUST NOT", "SHALL NOT"; "SHOULD", "RECOMMENDED"; "SHOULD NOT", "NOT RECOMMENDED"; "MAY", "OPTIONAL".