XEP-xxxx: Requirements for IM File Transfer

This document describes requirements and approaches for the design of a sustainable XMPP-based file transfer protocol.

WARNING: This document has not yet been accepted for consideration or approved in any official manner by the XMPP Standards Foundation, and this document must not be referred to as an XMPP Extension Protocol (XEP). If this document is accepted as a XEP by the XMPP Council, it will be published at <http://www.xmpp.org/extensions/> and announced on the <standards@xmpp.org> mailing list.

Document Information

Series: XEP
Number: xxxx
Publisher: XMPP Standards Foundation
Status: ProtoXEP
Type: Standards Track
Version: 0.0.1
Last Updated: 2007-09-13
Approving Body: XMPP Council
Dependencies: XMPP Core
Supersedes: None
Superseded By: None
Short Name: N/A

Author Information

Peter Saint-Andre

JabberID: stpeter@jabber.org
URI: https://stpeter.im/

Legal Notice

This XMPP Extension Protocol is copyright 1999 - 2007 by the XMPP Standards Foundation (XSF) and is in full conformance with the XSF's Intellectual Property Rights Policy <http://www.xmpp.org/extensions/ipr-policy.shtml>. This material may be distributed only subject to the terms and conditions set forth in the Creative Commons Attribution License (<http://creativecommons.org/licenses/by/2.5/>).

Discussion Venue

The preferred venue for discussion of this document is the Standards discussion list: <http://mail.jabber.org/mailman/listinfo/standards>.

Relation to XMPP

The Extensible Messaging and Presence Protocol (XMPP) is defined in the XMPP Core (RFC 3920) and XMPP IM (RFC 3921) specifications contributed by the XMPP Standards Foundation to the Internet Standards Process, which is managed by the Internet Engineering Task Force in accordance with RFC 2026. Any protocol defined in this document has been developed outside the Internet Standards Process and is to be understood as an extension to XMPP rather than as an evolution, development, or modification of XMPP itself.

Conformance Terms

The following keywords as used in this document are to be interpreted as described in RFC 2119: "MUST", "SHALL", "REQUIRED"; "MUST NOT", "SHALL NOT"; "SHOULD", "RECOMMENDED"; "SHOULD NOT", "NOT RECOMMENDED"; "MAY", "OPTIONAL".

Table of Contents

1. Introduction
2. Assumptions
3. Existing Technologies
4. Requirements
5. IANA Considerations
6. XMPP Registrar Considerations
7. Acknowledgements
Revision History

1. Introduction

The ability to transfer files is of general interest to instant messaging users. Unfortunately, existing file transfer technologies used in the Jabber community are far from reliable or full-featured. And unless file transfers can be reliably completed, users will revert to using email attachments as their preferred transport mechanism. Furthemore, it is unlikely that IM users will actively switch from email attachments unless the XMPP-based technology offers features that are unavailable in email-based systems. Therefore this document attempts to describe the requirements for more reliable, featureful file transfer methods between XMPP users.

2. Assumptions

  1. It is rarely necessary to stream a file. [1] Instead, in most situations it is perfectly acceptable for the sender to (temporarily) upload the file to a repository of some kind and then communicate the file's location to the recipient so that the recipient can retrieve the file.

  2. The HyperText Transport Protocol (HTTP; RFC 2616 [2]) is a very widely deployed technology for file upload and retrieval. Therefore HTTP is probably suitable for the purpose of XMPP file transfer in most cases (i.e., not every aspect of file transfer between XMPP users needs to occur using XMPP).

  3. Some IM users do not have access to HTTP servers. Therefore it is necessary to specify fallback mechanisms, whether newly defined or already existing.

  4. It is sometimes desirable to transfer multiple files.

  5. It is sometimes desirable for the file owner to advertise one or more files as available for transfer and enable the potential recipient to request the file(s).

3. Existing Technologies

The following technologies are already used for file transfer within in the Jabber community.

  1. Out-of-Band Data [3]. This protocol can be used if the sender already has a URL for the file, but depends on the existence of a file upload/download technology such as HTTP or FTP. Also, XEP-0066 does not include a way to specify meta-data about a file (although that could be done using Stanza Headers and Internet Metadata [4]). However, this protocol can be used even if the recipient is offline and could be integrated with Verifying HTTP Requests via XMPP [5] or a similar technology to provide authorization checking.

  2. In-Band Bytestreams [6]. This protocol provides a way for a sender to "chunk" a file into multiple XMPP message or IQ stanzas and therefore effectively stream a file "in-band" (i.e., over the XMPP band through an XMPP server or two). While this protocol therefore takes advantage of the fact that the IM users already have a communications channel open between them (e.g., through any firewalls or network address translators), it may not be appropriate as a primary transfer mechanism since XMPP servers are not necessarily optimized for transfer of large amounts of such chunked data. However, deployment experience indicates that IBB may be a very useful fallback mechanism.

  3. SOCKS5 Bytestreams [7]. This protocol uses TCP connections to and from a media relay server (implemented using a subset of the SOCKS5 protocol defined in RFC 1928 [8]) as the channel for a mediated connection between the sender and recipient. As the name implies, SOCKS5 Bytestreams results in streaming of a file from the sender to the recipient, which has proved less than desirable in certain circumstances (e.g., when organizational policies require that server-side virus-checking shall be used). Additionally, many Jabber service providers are hesitant to deploy media relay servers because of the bandwidth consumed by such relays.

  4. File Transfer [9]. This protocol provides a "front end" to In-Band Bytestreams and SOCKS5 Bytestreams, enabling a sender and recipient to negotiate which streaming technology will be used. This negotiation protocol is limited somewhat by Stream Initiation [10] -- it can be used only if both parties are online, and the recipient cannot counter-offer in case it wants to use a different media relay (such as one controlled by its own organization). This profile of stream initiation results in sending one file only; work on Tree Transfer Stream Initiation Profile [11] was started in order to define methods for sending multiple files, but was never completed. Publishing Stream Initiation Requests [12] provides a way to advertise the availability of files that can be transferred using this protocol.

  5. Jingle HTTP File Transfer. This protocol (an extension to the Jingle [13] session management protocol) is used by the Google Talk service to transfer one or more files between IM users. However, it is not fully specified and is not well understood by developers outside of Google. Like Stream Initiation, Jingle can be used only if both parties are online.

4. Requirements

  1. It must be possible to transfer one file at a time.
  2. It should be possible to transfer multiple files at a time.
  3. It should be possible to transfer files even if the recipient is offline.
  4. It must be possible to transfer files even if both the sender and recipient are behind restrictive firewalls or network address translators (NATs).
  5. It should be possible for the file owner to advertise the availability of a file.
  6. It should be possible for the potential recipient to request an available file.
  7. Existing technologies should be re-used as much as possible, whether defined by the XMPP Standards Foundation (XSF) [14] or other standards development organizations.

5. IANA Considerations

This document requires no action by the Internet Assigned Numbers Authority (IANA) [15].

6. XMPP Registrar Considerations

This document requires no action by the XMPP Registrar [16].

7. Acknowledgements

Thanks to the participants at the third XMPP DevCon (Portland, Oregon, August 2007) for input that led to the initial version of this document.


1. Compare the frequency of downloading a sound file and then listening to it vs. listening to a streaming broadcast.

2. RFC 2616: Hypertext Transport Protocol -- HTTP/1.1 <http://tools.ietf.org/html/rfc2616>.

3. XEP-0066: Out of Band Data <http://www.xmpp.org/extensions/xep-0066.html>.

4. XEP-0131: Stanza Headers and Internet Metadata <http://www.xmpp.org/extensions/xep-0131.html>.

5. XEP-0070: Verifying HTTP Requests via XMPP <http://www.xmpp.org/extensions/xep-0070.html>.

6. XEP-0047: In-Band Bytestreams <http://www.xmpp.org/extensions/xep-0047.html>.

7. XEP-0065: SOCKS5 Bytestreams <http://www.xmpp.org/extensions/xep-0065.html>.

8. RFC 1928: SOCKS Protocol Version 5 <http://tools.ietf.org/html/rfc1928>.

9. XEP-0096: File Transfer <http://www.xmpp.org/extensions/xep-0096.html>.

10. XEP-0095: Stream Initiation <http://www.xmpp.org/extensions/xep-0095.html>.

11. XEP-0105: Tree Transfer Stream Initiation Profile <http://www.xmpp.org/extensions/xep-0105.html>.

12. XEP-0137: Publishing Stream Initiation Requests <http://www.xmpp.org/extensions/xep-0137.html>.

13. XEP-0166: Jingle <http://www.xmpp.org/extensions/xep-0166.html>.

14. The XMPP Standards Foundation (XSF) is an independent, non-profit membership organization that develops open extensions to the IETF's Extensible Messaging and Presence Protocol (XMPP). For further information, see <http://www.xmpp.org/xsf/>.

15. The Internet Assigned Numbers Authority (IANA) is the central coordinator for the assignment of unique parameter values for Internet protocols, such as port numbers and URI schemes. For further information, see <http://www.iana.org/>.

16. The XMPP Registrar maintains a list of reserved protocol namespaces as well as registries of parameters used in the context of XMPP extension protocols approved by the XMPP Standards Foundation. For further information, see <http://www.xmpp.org/registrar/>.

Revision History

Version 0.0.1 (2007-09-13)

First draft.