XEP-0214: File Repository and Sharing

While a protocol has been described for initiating a file transfer from one user to another, there is not yet a way for users to designate a set of files as available for retrieval by other users of their choosing. This extension fills that functionality, with additional features such as file versioning, multiple download mirrors across several internet protocols, collaborative modification of the file listing, and all the other benefits of Pubsub.

WARNING: Consideration of this document has been Deferred by the XMPP Standards Foundation. Implementation of the protocol described herein is not recommended.

Document Information

Series: XEP
Number: 0214
Publisher: XMPP Standards Foundation
Status: Deferred
Type: Standards Track
Version: 0.1
Last Updated: 2007-04-20
Approving Body: XMPP Council
Dependencies: XMPP Core, XEP-0001, XEP-0060, XEP-0082, XEP-0137
Supersedes: None
Superseded By: None
Short Name: Not yet assigned
Source Control: HTML RSS

Author Information

Nicholas Parker

Email: nickbp@gmail.com
JabberID: nickp@jabber.org

Legal Notices

Copyright

Permissions

Disclaimer of Warranty

Limitation of Liability

IPR Conformance

Discussion Venue

Relation to XMPP

The Extensible Messaging and Presence Protocol (XMPP) is defined in the XMPP Core (RFC 3920) and XMPP IM (RFC 3921) specifications contributed by the XMPP Standards Foundation to the Internet Standards Process, which is managed by the Internet Engineering Task Force in accordance with RFC 2026. Any protocol defined in this document has been developed outside the Internet Standards Process and is to be understood as an extension to XMPP rather than as an evolution, development, or modification of XMPP itself.

Conformance Terms

The following keywords as used in this document are to be interpreted as described in RFC 2119: "MUST", "SHALL", "REQUIRED"; "MUST NOT", "SHALL NOT"; "SHOULD", "RECOMMENDED"; "SHOULD NOT", "NOT RECOMMENDED"; "MAY", "OPTIONAL".

1. Introduction
2. Requirements
3. Glossary
4. Use Cases
    4.1. File Listing
       4.1.1. Publication
       4.1.2. Subscription
       4.1.3. Addition
       4.1.4. New Revisions
       4.1.5. Modification/Deletion
    4.2. File Requests
5. Implementation Notes
6. Security Considerations
7. IANA Considerations
8. Jabber Registrar Considerations
9. XML Schema
Notes
Revision History

1. Introduction

Describes how a Jabber user may find and retrieve files which other Jabber users have published. The listing of files is done through a Pubsub (XEP-0060) server, allowing multiple users to manage the same listing. Other features include file metadata, revisions, and download mirrors.

Retrieval of the files provided in the listing MAY be performed through any relevant protocol for transferring data (http, ftp, etc), but this protocol emphasizes the use of Stream Initiation (XEP-0137) to establish the connection.

2. Requirements

3. Glossary

Table 1: File Sharing Terms

File Listing	Top-level Pubsub Collection Node, containing information about files and/or subsections which a user or group of users have published.
Subsection	Non-Root Collection Node which containts files and/or other subsections.
File	Pubsub Node, stored within a File Listing, which describes all revisions of a given file. The filename and (optionally) description are provided here.
Revision	Pubsub Item which describes a given file revision. Other metadata which can vary between revisions is provided here (file size, checksum, available mirrors, etc).
Mirror	A location which has a given Revision available for download. Additional information about a given Mirror MAY be provided for protocols that require it. A list of available protocols is provided below.

4. Use Cases

4.1 File Listing

The following use cases describe tasks which are already covered by XEP-0060 in a more generic context. These tasks are explicitly described in order to demonstrate the wide range of capabilities afforded by this system and convey the data structure of the file listing itself. Consult XEP-0060 for the full range of node and user management commands as well as their server responses.

4.1.1 Publication

Juliet wishes to make her sonnets available for retrieval by the public. She creates a Root Pubsub Collection Node which will contain her file listing:

Example 1. Creating a New File Listing


<iq type='set'
    from='juliet@capulet.com/balcony'
    to='pubsub.shakespeare.lit'
    id='create3'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <create node='juliets_sonnets'/>
    <configure>
      <x xmlns='jabber:x:data' type='submit'>
        <field var='FORM_TYPE' type='hidden'>
          <value>http://jabber.org/protocol/pubsub#node_config</value>
        </field>
        <field var='pubsub#node_type'><value>collection</value></field>
      </x>
    </configure>
  </pubsub>
</iq>

<iq type='set'
    from='juliet@capulet.com/balcony'
    to='pubsub.shakespeare.lit'
    id='config2'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <configure node='juliets_sonnets'/>
      <x xmlns='jabber:x:data' type='submit'>
        <field var='FORM_TYPE' type='hidden'>
          <value>http://jabber.org/protocol/pubsub#meta-data</value>
        </field>
        <field var='pubsub#title'><value>Juliet's Sonnets</value></field>
        <field var='pubsub#description'><value>Optional Description</value></field>
      </x>
    </configure>
  </pubsub>
</iq>

Juliet also wishes to add a subsection for her sonnets about Romeo. She creates another Pubsub Collection Node under the Root Node:

Example 2. Adding a Subsection to the Listing


<iq type='set'
    from='juliet@capulet.com/balcony'
    to='pubsub.shakespeare.lit'
    id='create3'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <create node='35227eec194a4f3971a5f3771e9c2271'/>
    <configure>
      <x xmlns='jabber:x:data' type='submit'>
        <field var='FORM_TYPE' type='hidden'>
          <value>http://jabber.org/protocol/pubsub#node_config</value>
        </field>
        <field var='pubsub#collection'><value>juliets_sonnets</value></field>
        <field var='pubsub#node_type'><value>collection</value></field>
      </x>
    </configure>
  </pubsub>
</iq>

<iq type='set'
    from='juliet@capulet.com/balcony'
    to='pubsub.shakespeare.lit'
    id='config2'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <configure node='35227eec194a4f3971a5f3771e9c2271'/>
      <x xmlns='jabber:x:data' type='submit'>
        <field var='FORM_TYPE' type='hidden'>
          <value>http://jabber.org/protocol/pubsub#meta-data</value>
        </field>
        <field var='pubsub#title'><value>Sonnets About Romeo</value></field>
        <field var='pubsub#description'><value>Optional Description</value></field>
      </x>
    </configure>
  </pubsub>
</iq>

4.1.2 Subscription

Romeo wishes to view all of Juliet's shared sonnets. To do this, Romeo subscribes to the Root Collection Node:

Example 3. Subscription to entire File Listing


<iq type='set'
    from='romeo@montague.net/orchard'
    to='pubsub.shakespeare.lit'
    id='collsub2'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <subscribe jid='romeo@montague.net' node='juliets_sonnets'/>
    <options>
      <x xmlns='jabber:x:data'>
        <field var='FORM_TYPE' type='hidden'>
          <value>http://jabber.org/protocol/pubsub#subscribe_options</value>
        </field>
        <field var='pubsub#subscription_type'><value>items</value></field>
        <field var='pubsub#subscription_depth'><value>all</value></field>
      </x>
    </options>
  </pubsub>
</iq>

4.1.3 Addition

Juliet has just finished a new sonnet and wishes to announce its availability on her File Listing. She adds the sonnet as a new Pubsub Node stored in her Collection Node, then inserts a first revision of her sonnet as an Item within that Node:

Example 4. Adding a new File


<iq type='set'
    from='juliet@capulet.com/balcony'
    to='pubsub.shakespeare.lit'
    id='create4'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub#node_config'>
    <create node='a6190c5d38e22452041d1c5798eff3f5'/>
    <configure>
      <x xmlns='jabber:x:data' type='submit'>
        <field var='pubsub#collection'><value>juliets_sonnets</value></field>
      </x>
    </configure>
  </pubsub>
</iq>

<iq type='set'
    from='juliet@capulet.com/balcony'
    to='pubsub.shakespeare.lit'
    id='config2'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <configure node='a6190c5d38e22452041d1c5798eff3f5'>
      <x xmlns='jabber:x:data' type='submit'>
        <field var='FORM_TYPE' type='hidden'>
          <value>http://jabber.org/protocol/pubsub#meta-data</value>
        </field>
        <field var='pubsub#title'><value>sonnet.txt</value></field>
        <field var='pubsub#description'><value>Sonnet 42</value></field>
      </x>
    </configure>
  </pubsub>
</iq>

<iq type='set'
    from='juliet@capulet.com/balcony'
    to='pubsub.shakespeare.lit'
    id='publish1'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <publish node='a6190c5d38e22452041d1c5798eff3f5'>
      <item id='1'>
        <entry xmlns='http://jabber.org/protocol/fileshare#item_meta-data'>
          <size>5623</size>
          <modified>2006-12-13T18:30:02Z</modified>
          <checksum type="sha1">59282c5db190bdc3b152c5b38363442bfda8ebdd</checksum>
          <mime>text/plain</mime>
          <description>My Latest Sonnet!</description>
          <mirrors>
            <mirror type='sipub' description='File Transfer via capulet.com fileserver'>
              <sipub xmlns='http://jabber.org/protocol/si-pub'
                     from='fileserver@capulet.com'
                     id='publish-sonnet.txt'
                     mime-type='text/plain'
                     profile='http://jabber.org/protocol/si/profile/file-transfer'>
                <file xmlns='http://jabber.org/protocol/si/profile/file-transfer'
                      name='sonnet.txt'
                      size='5623'/>
              </sipub>
            </mirror>
            <mirror type='sipub' description='Jingle HTTP File Transfer via capulet.com fileserver'>
              <sipub xmlns='http://jabber.org/protocol/si-pub'
                     from='fileserver-httpft@capulet.com'
                     id='publish-sonnet.txt'
                     mime-type='text/plain'
                     profile='http://jabber.org/protocol/si/profile/jingle-httpft'>
                <description xmlns='http://www.xmpp.org/extensions/xep-XXXX.html#ns'>
                  <manifest>
                    <file>
                      <name>sonnet.txt</name>
                    </file>
                  </manifest>
                  <http>
                    <url name='source-path'>/source/23A53F01/</url>
                    <url name='preview-path'>/preview/90266EA1/</url>
                  </http>
                </description>
              </sipub>
            </mirror>
            <mirror type='http' description='Shakespeare.lit Torrent'
                    address='www.shakespeare.lit'
                    ref='torrents/sonnet.torrent'/>
            <mirror type='http' description='Shakespeare.lit Website'
                    address='www.shakespeare.lit'
                    ref='~juliet/sonnet.txt'/>
            <mirror type='https' description='Shakespeare.lit Website (SSL)'
                    address='ssl.shakespeare.lit'
                    ref='~juliet/sonnet.txt'/>
            <mirror type='ftp' description='Shakespeare.lit FTP'
                    user='guest' pass='guest'
                    address='files.shakespeare.lit' port='21'
                    ref='public/sonnet.txt'/>
            <mirror type='sftp' description='Shakespeare.lit SFTP'
                    user='guest' pass='guest'
                    address='ssh.shakespeare.lit' port='22'
                    ref='public/sonnet.txt'/>
            <mirror type='smb' description='Capulet Intranet SMB Share'
                    user='guest' pass='guest'
                    address='smbfiles.capulet.com'
                    ref='juliet/mysonnets/sonnet.txt'/>
          </mirrors>
        </entry>
      </item>
    </publish>
  </pubsub>
</iq>

The Item ID is set to 1, signifying the first revision for this file. Subsequent revisions/items will have incremented ID values, like one would see in a versioning system such as CVS or SVN. Implementations MAY follow this convention, but are not required to do so. For example, a given implementation may instead mark revisions using version numbers ("Beta 1", "6.2", etc) or use other arbitrary strings. However, no two revisions of a given file may share the same ID.

Here is a listing of the possible metadata in a file revision (Item), each field is OPTIONAL:

Table 2: Revision Metadata

Size	The size, in bytes, of the file.
Modified	The last modified time of the revision. Follows the format described in XEP-0082. If a publisher prefers to only make a single revision available to clients, the publisher MAY instead update this value (and others, such as size and/or checksum) to announce that a new version of the file is available.
Checksum	A checksum of the revision, using the specified hash algorithm. Acceptable types are "sha512", "sha1", "md5", and "crc32".
Mime	The file's MIME type.
Description	Description text for the revision. As an example, could contain release notes.
Mirrors	A list of mirrors; their properties are defined below. If no downloads are available, MAY be left empty or removed entirely.

Because Romeo is now subscribed, he receives notice of Juliet's addition:

Example 5. Notification of Addition


<message from='pubsub.shakespeare.lit' to='romeo@montague.net' id='create4'>
  <event xmlns='http://jabber.org/protocol/pubsub#event'>
    <collection>
      <node id='a6190c5d38e22452041d1c5798eff3f5'>
        <x xmlns='jabber:x:data' type='result'>
          <field var='FORM_TYPE' type='hidden'>
            <value>http://jabber.org/protocol/pubsub#node_config</value>
          </field>
          <field var='pubsub#collection'><value>juliets_sonnets</value></field>
        </x>
      </node>
    </collection>
  </event>
</message>

<message from='pubsub.shakespeare.lit' to='romeo@montague.net' id='config2'>
  <event xmlns='http://jabber.org/protocol/pubsub#event'>
    <configuration node='a6190c5d38e22452041d1c5798eff3f5'>
      <x xmlns='jabber:x:data' type='result'>
        <field var='FORM_TYPE' type='hidden'>
          <value>http://jabber.org/protocol/pubsub#meta-data</value>
        </field>
        <field var='pubsub#description'><var>Sonnet 42</var></field>
        <field var='pubsub#title'><var>sonnet.txt</var></field>
      </x>
    </configuration>
  </event>
</message>

<message from='pubsub.shakespeare.lit' to='romeo@montague.net' id='foo'>
  <event xmlns='http://jabber.org/protocol/pubsub#event'>
    <items node='a6190c5d38e22452041d1c5798eff3f5'>
      <item id='1'>
        <entry xmlns='http://jabber.org/protocol/fileshare#item_config'>
          <size>5623</size>
          <modified>2006-12-13T18:30:02Z</modified>
          <checksum type="sha1">59282c5db190bdc3b152c5b38363442bfda8ebdd</checksum>
          <mime>text/plain</mime>
          <description>My Latest Sonnet!</description>
          <mirrors>
            ... MIRRORS ...
          </mirrors>
        </entry>
      </item>
    </items>
  </event>
</message>

The above examples give a listing of available mirror protocols in probable configurations. As described in the Implementation Notes, only the sipub mirror type is REQUIRED, the others are only given as examples for common protocols which are not already compatible with SI. Here is a full listing of those protocols and their available settings:

Table 3: Mirror Settings

Protocol	Description	Ref	Address	Port (default)	User	Pass
sipub	OPTIONAL
http	OPTIONAL	REQUIRED	REQUIRED	OPTIONAL (80)	OPTIONAL	OPTIONAL
https	OPTIONAL	REQUIRED	REQUIRED	OPTIONAL (443)	OPTIONAL	OPTIONAL
ftp	OPTIONAL	REQUIRED	REQUIRED	OPTIONAL (21)	OPTIONAL	OPTIONAL
sftp	OPTIONAL	REQUIRED	REQUIRED	OPTIONAL (22)	OPTIONAL	OPTIONAL
smb	OPTIONAL	REQUIRED (Incl. Share name)	REQUIRED	OPTIONAL (445)	OPTIONAL	OPTIONAL

The Description field is where an arbitrary description of the mirror MAY be placed. For example, if a File Listing is advertising mirrors which are located in different geographic locations, then this field may be used to specify those locations.

The Ref field is a unique identifier which is used to request the file from the mirror server. In the above examples, it is used as a path to the file.

The address and port fields describe where the file may be retrieved using the specified protocol. If a port is not provided, the default value (specified in parentheses) is assumed.

The User and Pass fields are for providing credentials which, if given by the File Listing, SHOULD be used when requesting the file. For example, an sftp mirror MAY require that the user log in using specified credentials before the file may be retrieved.

4.1.4 New Revisions

Juliet has revised her sonnet and wishes to publish the new version, while still leaving the original copy available for retrieval. To do this, she inserts a new Item, representing her new revision, into the file's Node:

Example 6. Adding a new Revision


<iq type='set'
    from='juliet@capulet.com/balcony'
    to='pubsub.shakespeare.lit'
    id='publish1'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <publish node='a6190c5d38e22452041d1c5798eff3f5'>
      <item id='2'>
        <entry xmlns='http://jabber.org/protocol/fileshare#item_config'>
          <size>6102</size>
          <modified>2007-01-13T18:30:02Z</modified>
          <checksum type="md5">6aaa20212a99548765b3b15f24f19aaa</checksum>
          <checksum type="sha1">97cbc0e445435af94db5cc2133b94ab5faf1399a</checksum>
          <mime>text/plain</mime>
          <description>A revised copy, fixed some spelling errors.</description>
          <mirrors>
            <mirror type='ftp' description='Shakespeare.lit FTP'
                    user='guest' pass='guest'
                    address='files.shakespeare.lit' port='21'
                    ref='public/juliet/sonnet2.txt'/>
            <mirror type='http' description='Shakespeare.lit Website'
                    address='www.shakespeare.lit'
                    ref='~juliet/sonnet2.txt'/>
            <mirror type='sipub' description='File Transfer via capulet.com fileserver'>
              <sipub xmlns='http://jabber.org/protocol/si-pub'
                     from='fileserver@capulet.com'
                     id='publish-sonnet2.txt'
                     mime-type='text/plain'
                     profile='http://jabber.org/protocol/si/profile/file-transfer'>
                <file xmlns='http://jabber.org/protocol/si/profile/file-transfer'
                      name='sonnet2.txt'
                      size='6102'/>
              </sipub>
            </mirror>
          </mirrors>
        </entry>
      </item>
    </publish>
  </pubsub>
</iq>

4.1.5 Modification/Deletion

Juliet has uploaded a copy of her revised sonnet to a new mirror, and wishes to let her subscribers know about this secondary source. She is able to do this by modifying the revision in question to include a reference to her website, overwriting the existing mirrors in the Item with an updated list:

Example 7. Modifying a Revision


<iq type='set'
    from='juliet@capulet.com/balcony'
    to='pubsub.shakespeare.lit'
    id='publish1'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <publish node='a6190c5d38e22452041d1c5798eff3f5'>
      <item id='2'>
        <entry xmlns='http://jabber.org/protocol/fileshare#item_config'>
          <mirrors>
            <mirror type='ftp' description='Shakespeare.lit FTP'
                    user='guest' pass='guest'
                    address='files.shakespeare.lit' port='21'
                    ref='public/juliet/sonnet2.txt'/>
            <mirror type='http' description='Shakespeare.lit Website'
                    address='www.shakespeare.lit'
                    ref='~juliet/sonnet2.txt'/>
            <mirror type='sipub' description='File Transfer via capulet.com fileserver'>
              <sipub xmlns='http://jabber.org/protocol/si-pub'
                     from='fileserver@capulet.com'
                     id='publish-sonnet2.txt'
                     mime-type='text/plain'
                     profile='http://jabber.org/protocol/si/profile/file-transfer'>
                <file xmlns='http://jabber.org/protocol/si/profile/file-transfer'
                      name='sonnet2.txt'
                      size='6102'/>
              </sipub>
            </mirror>
            <mirror type='http' description='Shakespeare.lit Boston Mirror'
                    address='www.capulet.com'
                    ref='~juliet/sonnet2.txt'/>
          </mirrors>
        </entry>
      </item>
    </publish>
  </pubsub>
</iq>

Juliet now wishes to allow others to contribute to her sonnet collection. She gives owner access for the entire Listing to Romeo, and publisher access to her nurse:

Example 8. Modifying Users


<iq type='set'
    from='juliet@capulet.com/balcony'
    to='pubsub.shakespeare.lit'
    id='ent3'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub#owner'>
    <affiliations node='juliets_sonnets'>
      <affiliation jid='nurse@capulet.com' affiliation='publisher'/>
      <affiliation jid='romeo@montague.net' affiliation='owner'/>
    </affiliations>
  </pubsub>
</iq>

Romeo uses his owner access to remove the older revision of Juliet's sonnet:

Example 9. Deleting a Revision


<iq type='set'
    from='romeo@montague.net/orchard'
    to='pubsub.shakespeare.lit'
    id='retract1'>
  <pubsub xmlns='http://jabber.org/protocol/pubsub'>
    <retract node='a6190c5d38e22452041d1c5798eff3f5'>
      <item id='1'/>
    </retract>
  </pubsub>
</iq>

Other deletion, modification, and user management operations are available as described in XEP-0060. These examples are provided here to explicitly illustrate the capabilities offered by this system.

4.2 File Requests

Romeo is interested in seeing what files Juliet has made available. To do this, Romeo sends Juliet a request for repositories which she is associated with:

Example 10. Request for File Repository listing


<iq type='get'
    from='romeo@montague.net/orchard'
    to='juliet@capulet.com'
    id='repolistreq'>
  <fileshare xmlns='http://jabber.org/protocol/si/profile/fileshare'>
    <list/>
  </fileshare>
</iq>

Juliet responds with a list of Pubsub nodes where she has published files or which she believes would be interesting to Romeo. If no such locations exist, Juliet SHOULD respond with an empty list.

Example 11. File Repository listing


<iq type='get'
    from='romeo@montague.net/orchard'
    to='juliet@capulet.com'
    id='repolist'>
  <fileshare xmlns='http://jabber.org/protocol/si/profile/fileshare'>
    <list>
      <repo address='pubsub.shakespeare.lit'
            node='juliets_sonnets' description='My Sonnets'/>
    </list>
  </fileshare>
</iq>

After browsing Juliet's repository, Romeo has chosen to download her sonnet. The most recent revision of this file contains a listing of available mirrors, and one of them is an SI stream. Romeo sends an SI request to that mirror:

Example 12. Request that a file be sent


<iq type='get'
    id='sipub-request-0'
    from='romeo@montague.net/orchard'
    to='fileserver@capulet.com'>
  <start xmlns='http://jabber.org/protocol/sipub'
         id='publish-sonnet2.txt'/>
</iq>

The rest of the negotiation and transfer occurs as described in XEP-0137.

5. Implementation Notes

Since Pubsub is used for the File Listing, the access models described in XEP-0060 MUST be followed. Users MUST NOT be able to view or control information in the File Listing to which they do not have access.

Node IDs MAY take the form of "path/to/file.ext", rather than the randomized strings provided in the above use cases. For example, Juliet's sonnet MAY use a Node ID of "juliets_sonnets/sonnet.txt" rather than "a6190c5d38e22452041d1c5798eff3f5", as long as this ID is unique to the PubSub server. Randomized strings were used in order to explicitly illustrate that Node IDs SHOULD NOT be depended upon for storing information about files.

The "sipub" mirror type MUST be implemented by the client, while the other mirror types are entirely OPTIONAL, and are provided only as examples for custom client implementations which desire use of common protocols which are currently incompatible with SI requests.

If user access to files is restricted, the mirror servers and the File Listing server MUST be able to synchronize these restrictions between them. See Security Considerations.

6. Security Considerations

When restricted files are being distributed, mirrors need to know which users have permission to access which files. If mirrors are not provided this information by the File Listing (or some other entity), unauthorized users could request files from mirrors directly, thus bypassing any such restrictions.

7. IANA Considerations

No interaction with the Internet Assigned Numbers Authority (IANA) is required as a result of this XEP.