XEP-0363: HTTP File Upload

Abstract
This specification defines a protocol to request permissions from another entity to upload a file to a specific path on an HTTP server and at the same time receive a URL from which that file can later be downloaded again.
Author
Daniel Gultsch
Copyright
© 2015 – 2022 XMPP Standards Foundation. SEE LEGAL NOTICES.
Status

Stable

NOTICE: The protocol defined herein is a Stable Standard of the XMPP Standards Foundation. Implementations are encouraged and the protocol is appropriate for deployment in production systems, but some changes to the protocol are possible before it becomes a Final Standard.
Type
Standards Track
Version
1.1.0 (2022-01-04)
Document Lifecycle
  1. Experimental
  2. Proposed
  3. Stable
  4. Final

1. Introduction

XMPP protocol extensions already define methods for peer-to-peer file transfer such as SI File Transfer (XEP-0096) [3] or Jingle File Transfer (XEP-0234) [4] however due to their very nature of being peer-to-peer they don't work very well in scenarios where it is requeried to send a file to multiple recipients or multiple resources of the same recipient at once. They also don’t work alongside offline storage, MUC history and Message Archive Management (XEP-0313) [5].

Uploading files manually to an HTTP server and sharing the link has been a workaround for this for a long time now. While users have a variety of services to choose from the downside of this manual approach is that an XMPP client can not automate this process on behalf of the user since these services don’t share a common API. Furthermore using a third party service would probably require the user to enter additional credentials into their XMPP client specifically for the file upload.

This XEP defines an approach to request permissions from another entity to upload a file to a specific path on an HTTP server and at the same time receive an URL from which that file can later be downloaded again. These tuples consisting of a PUT and a GET-URL are called slots.

2. Requirements

3. Discovering Support

An entity advertises support for this protocol by including the "urn:xmpp:http:upload:0" in its service discovery information features as specified in Service Discovery (XEP-0030) [9] or section 6.3 of Entity Capabilities (XEP-0115) [10]. To avoid unnecessary round trips an entity SHOULD also include the maximum file size as specified in Service Discovery Extensions (XEP-0128) [2] if such a limitation exists. The field name MUST be "max-file-size" and the value MUST be in bytes.

A user’s server SHOULD include any known entities that provide such services into its service discovery items.

Example 1. Client sends service discovery request to server
<iq from='romeo@montague.tld/garden'
    id='step_01'
    to='montague.tld'
    type='get'>
  <query xmlns='http://jabber.org/protocol/disco#items'/>
</iq>
Example 2. Server replies to service discovery request
<iq from='montague.tld'
    id='step_01'
    to='romeo@montague.tld/garden'
    type='result'>
  <query xmlns='http://jabber.org/protocol/disco#items'>
    <item jid='upload.montague.tld' name='HTTP File Upload' />
    <item jid='conference.montague.tld' name='Chatroom Service' />
  </query>
</iq>
Example 3. Client sends service discovery request to upload service
<iq from='romeo@montague.tld/garden'
    id='step_02'
    to='upload.montague.tld'
    type='get'>
  <query xmlns='http://jabber.org/protocol/disco#info'/>
</iq>
Example 4. Upload service replies to service discovery request and reports a maximum file size of 5MiB
<iq from='upload.montague.tld'
    id='step_02'
    to='romeo@montague.tld/garden'
    type='result'>
  <query xmlns='http://jabber.org/protocol/disco#info'>
    <identity category='store'
              type='file'
              name='HTTP File Upload' />
    <feature var='urn:xmpp:http:upload:0' />
    <x type='result' xmlns='jabber:x:data'>
      <field var='FORM_TYPE' type='hidden'>
        <value>urn:xmpp:http:upload:0</value>
      </field>
      <field var='max-file-size'>
        <value>5242880</value>
      </field>
    </x>
  </query>
</iq>

4. Requesting a slot

A client requests a new upload slot by sending an IQ-get to the upload service containing a <request> child element qualified by the urn:xmpp:http:upload:0 namespace. This element MUST include the attributes filename and size containing the file name and size (in bytes) respectively.

An additional attribute content-type containing the Content-Type is OPTIONAL.

Example 5. Client requests a slot on the upload service
<iq from='romeo@montague.tld/garden'
    id='step_03'
    to='upload.montague.tld'
    type='get'>
  <request xmlns='urn:xmpp:http:upload:0'
    filename='très cool.jpg'
    size='23456'
    content-type='image/jpeg' />
</iq>

The upload service responds with both a PUT and a GET URL wrapped by a <slot> element. The service SHOULD keep the file name and especially the file ending intact. Using the same hostname for PUT and GET is OPTIONAL. The host MUST provide Transport Layer Security (RFC 5246 [11]). Both HTTPS URLs MUST adhere to RFC 3986 [12]. Non ASCII characters MUST be percent-encoded.

The <put> element MAY also contain a number of <header> elements which correspond to HTTP header fields. Each <header> element MUST have a name-attribute and a content with the value of the header. Only the following header names are allowed: Authorization, Cookie, Expires. The allowed headers provided in the response MUST be included in the HTTP PUT request. Other header names MUST be ignored by the requesting entity and MUST NOT be included in the HTTP request. The requesting entity MUST strip any newline characters from the header name and value before performing the HTTP request, but MUST preserve the relative order of multiple values for the same header in the request. Each header name MAY be present zero or more times, and are case insensitive (eXpires is the same as Expires).

Example 6. The upload service responds with a slot
<iq from='upload.montague.tld'
    id='step_03'
    to='romeo@montague.tld/garden'
    type='result'>
  <slot xmlns='urn:xmpp:http:upload:0'>
    <put url='https://upload.montague.tld/4a771ac1-f0b2-4a4a-9700-f2a26fa2bb67/tr%C3%A8s%20cool.jpg'>
      <header name='Authorization'>Basic Base64String==</header>
      <header name='Cookie'>foo=bar; user=romeo</header>
    </put>
    <get url='https://download.montague.tld/4a771ac1-f0b2-4a4a-9700-f2a26fa2bb67/tr%C3%A8s%20cool.jpg' />
  </slot>
</iq>

5. Error conditions

Instead of providing the client with a slot the service MAY respond with an error if the requested file size is too large. In addition the entity MAY inform the requester about the maximum file size.

Example 7. Alternative response by the upload service if the file size was too large
<iq from='upload.montague.tld'
    id='step_03'
    to='romeo@montague.tld/garden'
    type='error'>
  <request xmlns='urn:xmpp:http:upload:0'
    filename='très cool.jpg'
    size='23456'
    content-type='image/jpeg' />
  <error type='modify'>
    <not-acceptable xmlns='urn:ietf:params:xml:ns:xmpp-stanzas' />
    <text xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'>File too large. The maximum file size is 20000 bytes</text>
    <file-too-large xmlns='urn:xmpp:http:upload:0'>
      <max-file-size>20000</max-file-size>
    </file-too-large>
  </error>
</iq>

For any other type of error the service SHOULD respond with appropriate error types to indicate temporary or permanent errors.

For temporary errors such as exceeding a personal quota the service MAY include a <retry/> element qualified by the urn:xmpp:http:upload:0 namespace as a child of the <error/> element. The retry element MUST include an attribute 'stamp' which indicates the time at which the requesting entity may try again. The format of the timestamp MUST adhere to the date-time format specified in XMPP Date and Time Profiles (XEP-0082) [13] and MUST be expressed in UTC.

Example 8. Alternative response by the upload service to indicate a temporary error after the client exceeded a quota
<iq from='upload.montague.tld'
    id='step_03'
    to='romeo@montague.tld/garden'
    type='error'>
  <request xmlns='urn:xmpp:http:upload:0'
    filename='très cool.jpg'
    size='23456'
    content-type='image/jpeg' />
  <error type='wait'>
    <resource-constraint xmlns='urn:ietf:params:xml:ns:xmpp-stanzas' />
    <text xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'>Quota reached. You can only upload 5 files in 5 minutes</text>
    <retry xmlns='urn:xmpp:http:upload:0'
      stamp='2017-12-03T23:42:05Z' />
  </error>
</iq>
Example 9. Alternative response by the upload service to indicate an auth error to a client that is not allowed to upload files
<iq from='upload.montague.tld'
    id='step_03'
    to='romeo@montague.tld/garden'
    type='error'>
  <request xmlns='urn:xmpp:http:upload:0'
     filename='très cool.jpg'
     size='23456'
     content-type='image/jpeg' />
  <error type='auth'>
    <forbidden xmlns='urn:ietf:params:xml:ns:xmpp-stanzas' />
    <text xmlns='urn:ietf:params:xml:ns:xmpp-stanzas'>Only premium members are allowed to upload files</text>
  </error>
</iq>

6. Upload

The actual upload of the file happens via HTTP-PUT and is out of scope of this document. The upload service MUST reject the file upload if the Content-Length does not match the size of the slot request. The service SHOULD reject the file if the Content-Type has been specified beforehand and does not match. The service MAY assume application/octet-stream as a Content-Type if it the client did not specify a Content-Type at all.

In addition to the Content-Length and Content-Type header the client MUST include all allowed headers that came with the slot assignment.

There is no further XMPP communication required between the upload service and the client. A HTTP status Code of 201 means that the server is now ready to serve the file via the provided GET URL. If the upload fails for whatever reasons the client MAY request a new slot.

7. Implementation Notes

The upload service SHOULD choose an appropriate timeout for the validity of the PUT URL. Since there is no reason for a client to wait between requesting the slot and starting the upload, relatively low timeout values of around 300s are RECOMMENDED.

To make HTTP Upload work in web clients (including those hosted on a different domain) the upload service SHOULD set appropriate CORS-Headers. The exact headers and values are out of scope of this document but may include: Access-Control-Allow-Origin, Access-Control-Allow-Methods and Access-Control-Allow-Headers. For HTTP upload services that use custom Authorization or Cookie request header the CORS-Header Access-Control-Allow-Credentials might also be of importance.

Example for loose CORS Header
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: OPTIONS, HEAD, GET, PUT
Access-Control-Allow-Headers: Authorization, Content-Type
Access-Control-Allow-Credentials: true

Clients SHOULD NOT interpret headers and treat them as opaque.

8. Security Considerations

8.1 Server side

Note: This section is not normative; it may be updated when general web security recommendations change in the future.

It is recommended to run the HTTP upload domain used for GET requests in appropriate isolation from other HTTP based services to avoid user-generated, malicious scripts to be executed in the context of said services. Isolation techniques can include, but are not limited to, setting the Content-Security-Policy.

HTTP Upload Server Headers for better Content Security Policy
Content-Security-Policy: default-src 'none'; frame-ancestors 'none';

The provided policy will prohibit a browser from executing all active content from the HTTP upload domain (default-src 'none') and forbid embedding it from other pages (frame-ancestors 'none'). More information on Content-Security-Policy can be found on infosec.mozilla.org.

Further isolation can be achieved by hosting those files on an entirely different domain instead of using subdomains.

Headers may be signed so that receiving HTTP entities can verify these haven't been tempered with by clients.

8.2 Uploader

8.3 General

9. IANA Considerations

This document requires no interaction with the Internet Assigned Numbers Authority (IANA).

10. XMPP Registrar Considerations

10.1 Protocol Namespaces

This specification defines the following XML namespace:

Upon advancement of this specification from a status of Experimental to a status of Draft, the XMPP Registrar [14] shall add the foregoing namespace to the registry located at <https://xmpp.org/registrar/namespaces.html>, as described in Section 4 of XMPP Registrar Function (XEP-0053) [15].

11. XML Schema

<xml version="1.0" encoding="utf8">
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
    targetNamespace="urn:xmpp:http:upload:0"
    xmlns="urn:xmpp:http:upload:0">
  <xs:element name="request">
    <xs:complexType>
      <xs:attribute name="filename" type="xs:string" use="required"/>
      <xs:attribute name="size" type="xs:positiveInteger" use="required"/>
      <xs:attribute name="content-type" type="xs:string" use="optional"/>
    </xs:complexType>
  </xs:element>
  <xs:element name="slot">
    <xs:complexType>
     <xs:sequence>
       <xs:element name="put" minOccurs="1" maxOccurs="1">
         <xs:complexType>
           <xs:attribute name="url" type="xs:string" use="required"/>
           <xs:sequence>
             <xs:element name="header" minOccurs="0" maxOccurs="unbounded" type="xs:string">
               <xs:complexType>
                 <xs:attribute name="name" use="required">
                   <xs:simpleType>
                     <xs:restriction base="xs:string">
                       <xs:enumeration value="Authorization"/>
                       <xs:enumeration value="Cookie"/>
                       <xs:enumeration value="Expires"/>
                     </xs:restriction>
                   </xs:simpleType>
                 </xs:attribute>
               </xs:complexType>
             </xs:element>
           </xs:sequence>
         </xs:complexType>
       </xs:element>
       <xs:element name="get" minOccurs="1" maxOccurs="1">
         <xs:complexType>
           <xs:attribute name="url" type="xs:string" use="required"/>
         </xs:complexType>
       </xs:element>
     </xs:sequence>
    </xs:complexType>
  </xs:element>
  <xs:element name="file-too-large">
    <xs:complexType>
      <xs:sequence>
        <xs:element name="max-file-size" type="xs:positiveInteger" minOccurs="0" maxOccurs="1"/>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
  <xs:element name="retry">
    <xs:complexType>
      <xs:attribute name="stamp" type="xs:string" use="required"/>
    </xs:complexType>
  </xs:element>
</xs:schema>

Appendices

Appendix A: Document Information

Series
XEP
Number
0363
Publisher
XMPP Standards Foundation
Status
Stable
Type
Standards Track
Version
1.1.0
Last Updated
2022-01-04
Approving Body
XMPP Council
Dependencies
XMPP Core, XEP-0030, XEP-0128
Supersedes
None
Superseded By
None
Short Name
NOT_YET_ASSIGNED
Source Control
HTML

This document in other formats: XML  PDF

Appendix B: Author Information

Daniel Gultsch
Email
daniel@gultsch.de
JabberID
daniel@gultsch.de

Copyright

This XMPP Extension Protocol is copyright © 1999 – 2024 by the XMPP Standards Foundation (XSF).

Permissions

Permission is hereby granted, free of charge, to any person obtaining a copy of this specification (the "Specification"), to make use of the Specification without restriction, including without limitation the rights to implement the Specification in a software program, deploy the Specification in a network service, and copy, modify, merge, publish, translate, distribute, sublicense, or sell copies of the Specification, and to permit persons to whom the Specification is furnished to do so, subject to the condition that the foregoing copyright notice and this permission notice shall be included in all copies or substantial portions of the Specification. Unless separate permission is granted, modified works that are redistributed shall not contain misleading information regarding the authors, title, number, or publisher of the Specification, and shall not claim endorsement of the modified works by the authors, any organization or project to which the authors belong, or the XMPP Standards Foundation.

Disclaimer of Warranty

## NOTE WELL: This Specification is provided on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. ##

Limitation of Liability

In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall the XMPP Standards Foundation or any author of this Specification be liable for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising from, out of, or in connection with the Specification or the implementation, deployment, or other use of the Specification (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if the XMPP Standards Foundation or such author has been advised of the possibility of such damages.

IPR Conformance

This XMPP Extension Protocol has been contributed in full conformance with the XSF's Intellectual Property Rights Policy (a copy of which can be found at <https://xmpp.org/about/xsf/ipr-policy> or obtained by writing to XMPP Standards Foundation, P.O. Box 787, Parker, CO 80134 USA).

Visual Presentation

The HTML representation (you are looking at) is maintained by the XSF. It is based on the YAML CSS Framework, which is licensed under the terms of the CC-BY-SA 2.0 license.

Appendix D: Relation to XMPP

The Extensible Messaging and Presence Protocol (XMPP) is defined in the XMPP Core (RFC 6120) and XMPP IM (RFC 6121) specifications contributed by the XMPP Standards Foundation to the Internet Standards Process, which is managed by the Internet Engineering Task Force in accordance with RFC 2026. Any protocol defined in this document has been developed outside the Internet Standards Process and is to be understood as an extension to XMPP rather than as an evolution, development, or modification of XMPP itself.

Appendix E: Discussion Venue

The primary venue for discussion of XMPP Extension Protocols is the <standards@xmpp.org> discussion list.

Discussion on other xmpp.org discussion lists might also be appropriate; see <https://xmpp.org/community/> for a complete list.

Errata can be sent to <editor@xmpp.org>.

Appendix F: Requirements Conformance

The following requirements keywords as used in this document are to be interpreted as described in RFC 2119: "MUST", "SHALL", "REQUIRED"; "MUST NOT", "SHALL NOT"; "SHOULD", "RECOMMENDED"; "SHOULD NOT", "NOT RECOMMENDED"; "MAY", "OPTIONAL".

Appendix G: Notes

1. RFC 6120: Extensible Messaging and Presence Protocol (XMPP): Core <http://tools.ietf.org/html/rfc6120>.

2. XEP-0128: Service Discovery Extensions <https://xmpp.org/extensions/xep-0128.html>.

3. XEP-0096: SI File Transfer <https://xmpp.org/extensions/xep-0096.html>.

4. XEP-0234: Jingle File Transfer <https://xmpp.org/extensions/xep-0234.html>.

5. XEP-0313: Message Archive Management <https://xmpp.org/extensions/xep-0313.html>.

6. XEP-0066: Out of Band Data <https://xmpp.org/extensions/xep-0066.html>.

7. XEP-0370: Jingle HTTP Transport Method <https://xmpp.org/extensions/xep-0370.html>.

8. XEP-0084: User Avatar <https://xmpp.org/extensions/xep-0084.html>.

9. XEP-0030: Service Discovery <https://xmpp.org/extensions/xep-0030.html>.

10. XEP-0115: Entity Capabilities <https://xmpp.org/extensions/xep-0115.html>.

11. RFC 5246: The Transport Layer Security (TLS) Protocol Version 1.2 <http://tools.ietf.org/html/rfc5246>.

12. RFC 3986: Uniform Resource Identifiers (URI): Generic Syntax <http://tools.ietf.org/html/rfc3986>.

13. XEP-0082: XMPP Date and Time Profiles <https://xmpp.org/extensions/xep-0082.html>.

14. The XMPP Registrar maintains a list of reserved protocol namespaces as well as registries of parameters used in the context of XMPP extension protocols approved by the XMPP Standards Foundation. For further information, see <https://xmpp.org/registrar/>.

15. XEP-0053: XMPP Registrar Function <https://xmpp.org/extensions/xep-0053.html>.

Appendix H: Revision History

Note: Older versions of this specification might be available at https://xmpp.org/extensions/attic/

  1. Version 1.1.0 (2022-01-04)
    egp, mb
  2. Version 1.0.0 (2020-02-11)
    Accepted as Draft standard by the XMPP Council. Voted on 2020-01-29.
    XEP Editor (jsc)
  3. Version 0.10.0 (2020-01-20)
    dg
  4. Version 0.9.0 (2018-12-18)
    dg
  5. Version 0.8.0 (2018-12-10)
    dg
  6. Version 0.7.0 (2018-05-30)
    dg
  7. Version 0.6.0 (2018-04-21)
    dg
  8. Version 0.5.0 (2018-02-15)
    dg
  9. Version 0.4.0 (2017-12-03)
    dg
  10. Version 0.3.1 (2017-04-21)

    Fixed example

    dg
  11. Version 0.3.0 (2017-02-01)

    Slight syntax changes and added ability to provide additional HTTP header fields on slot assignment.

    dg
  12. Version 0.2.5 (2017-01-08)

    Merge typo fixes suggested by an unnamed user.

    XEP Editor: ssw
  13. Version 0.2.4 (2016-10-28)

    Fix TLS reference

    dg (XEP Editor: ssw)
  14. Version 0.2.3 (2016-07-11)

    Typo fixes

    wjt (XEP Editor: ssw)
  15. Version 0.2.2 (2016-03-30)

    Typo fix

    ssw
  16. Version 0.2.1 (2016-03-16)

    Minor dependency and citation fixes (fs).

    XEP Editor (ssw)
  17. Version 0.2 (2016-03-07)

    Announce maximum file size by means of Service Discovery Extensions (XEP-0128) [2]

    dg
  18. Version 0.1.2 (2016-02-16)

    Fix typo of discovery (Philipp Hancke).

    XEP Editor (mam)
  19. Version 0.1.1 (2016-01-05)

    Fix invalid XML in examples (tpa).

    XEP Editor (ssw)
  20. Version 0.1 (2015-08-27)

    Initial published version approved by the XMPP Council.

    XEP Editor (mam)
  21. Version 0.0.1 (2015-07-25)

    First draft.

    dg

Appendix I: Bib(La)TeX Entry

@report{gultsch2015xep0363,
  title = {HTTP File Upload},
  author = {Gultsch, Daniel},
  type = {XEP},
  number = {0363},
  version = {1.1.0},
  institution = {XMPP Standards Foundation},
  url = {https://xmpp.org/extensions/xep-0363.html},
  date = {2015-07-25/2022-01-04},
}

END