docs/extension_protocol.html

   1 <?xml version="1.0" encoding="utf-8" ?>
   2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
   3 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
   4 <head>
   5 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
   6 <meta name="generator" content="Docutils 0.5: http://docutils.sourceforge.net/" />
   7 <title></title>
   8 <meta name="author" content="Arvid Norberg, arvid&#64;rasterbar.com Ludvig Strigeus, ludde&#64;utorrent.com" />
   9 <link rel="stylesheet" href="style.css" type="text/css" />
  10 </head>
  11 <body>
  12 <div class="document">
  13
  14 <table class="docinfo" frame="void" rules="none">
  15 <col class="docinfo-name" />
  16 <col class="docinfo-content" />
  17 <tbody valign="top">
  18 <tr><th class="docinfo-name">Author:</th>
  19 <td>Arvid Norberg, <a class="reference" href="mailto:arvid&#64;rasterbar.com">arvid&#64;rasterbar.com</a>
  20 Ludvig Strigeus, <a class="last reference" href="mailto:ludde&#64;utorrent.com">ludde&#64;utorrent.com</a></td></tr>
  21 </tbody>
  22 </table>
  23 <div class="section">
  24 <h1><a id="extension-protocol-for-bittorrent" name="extension-protocol-for-bittorrent">extension protocol for bittorrent</a></h1>
  25 <p>The intention of this protocol is to provide a simple and thin transport
  26 for extensions to the bittorrent protocol. Supporting this protocol makes
  27 it easy to add new extensions without interfering with the standard
  28 bittorrent protocol or clients that don't support this extension or the
  29 one you want to add.</p>
  30 <p>To advertise to other clients that you support, one bit from the reserved
  31 bytes is used.</p>
  32 <p>The bit selected for the extension protocol is bit 20 from the right (counting
  33 starts at 0). So (reserved_byte[5] &amp; 0x10) is the expression to use for checking
  34 if the client supports extended messaging.</p>
  35 <p>Once support for the protocol is established, the client is supposed to
  36 support 1 new message:</p>
  37 <table border="1" class="docutils">
  38 <colgroup>
  39 <col width="86%" />
  40 <col width="14%" />
  41 </colgroup>
  42 <thead valign="bottom">
  43 <tr><th class="head">name</th>
  44 <th class="head">id</th>
  45 </tr>
  46 </thead>
  47 <tbody valign="top">
  48 <tr><td><tt class="docutils literal"><span class="pre">extended</span></tt></td>
  49 <td>20</td>
  50 </tr>
  51 </tbody>
  52 </table>
  53 <p>This message is sent as any other bittorrent message, with a 4 byte length
  54 prefix and a single byte identifying the message (the single byte being 20
  55 in this case). At the start of the payload of the message, is a single byte
  56 message identifier. This identifier can refer to different extension messages
  57 and only one ID is specified, 0. If the ID is 0, the message is a handshake
  58 message which is described below. The layout of a general <tt class="docutils literal"><span class="pre">extended</span></tt> message
  59 follows (including the message headers used by the bittorrent protocol):</p>
  60 <table border="1" class="docutils">
  61 <colgroup>
  62 <col width="15%" />
  63 <col width="85%" />
  64 </colgroup>
  65 <thead valign="bottom">
  66 <tr><th class="head">size</th>
  67 <th class="head">description</th>
  68 </tr>
  69 </thead>
  70 <tbody valign="top">
  71 <tr><td>uint32_t</td>
  72 <td>length prefix. Specifies the number of bytes for the
  73 entire message. (Big endian)</td>
  74 </tr>
  75 <tr><td>uint8_t</td>
  76 <td>bittorrent message ID, = 20</td>
  77 </tr>
  78 <tr><td>uint8_t</td>
  79 <td>extended message ID. 0 = handshake, &gt;0 = extended
  80 message as specified by the handshake.</td>
  81 </tr>
  82 </tbody>
  83 </table>
  84 <div class="section">
  85 <h2><a id="handshake-message" name="handshake-message">handshake message</a></h2>
  86 <p>The payload of the handshake message is a bencoded dictionary. All items
  87 in the dictionary are optional. Any unknown names should be ignored
  88 by the client. All parts of the dictionary are case sensitive.
  89 This is the defined item in the dictionary:</p>
  90 <table border="1" class="docutils">
  91 <colgroup>
  92 <col width="11%" />
  93 <col width="89%" />
  94 </colgroup>
  95 <thead valign="bottom">
  96 <tr><th class="head">name</th>
  97 <th class="head">description</th>
  98 </tr>
  99 </thead>
 100 <tbody valign="top">
 101 <tr><td>m</td>
 102 <td><p class="first">Dictionary of supported extension messages which maps
 103 names of extensions to an extended message ID for each
 104 extension message. The only requirement on these IDs
 105 is that no extension message share the same one. Setting
 106 an extension number to zero means that the extension is
 107 not supported/disabled. The client should ignore any
 108 extension names it doesn't recognize.</p>
 109 <p class="last">The extension message IDs are the IDs used to send the
 110 extension messages to the peer sending this handshake.
 111 i.e. The IDs are local to this particular peer.</p>
 112 </td>
 113 </tr>
 114 </tbody>
 115 </table>
 116 <p>Here are some other items that an implementation may choose to support:</p>
 117 <table border="1" class="docutils">
 118 <colgroup>
 119 <col width="12%" />
 120 <col width="88%" />
 121 </colgroup>
 122 <thead valign="bottom">
 123 <tr><th class="head">name</th>
 124 <th class="head">description</th>
 125 </tr>
 126 </thead>
 127 <tbody valign="top">
 128 <tr><td>p</td>
 129 <td>Local TCP listen port. Allows each side to learn about
 130 the TCP port number of the other side. Note that there is
 131 no need for the receiving side of the connection to send
 132 this extension message, since its port number is already
 133 known.</td>
 134 </tr>
 135 <tr><td>v</td>
 136 <td>Client name and version (as a utf-8 string).
 137 This is a much more reliable way of identifying the
 138 client than relying on the peer id encoding.</td>
 139 </tr>
 140 <tr><td>yourip</td>
 141 <td>A string containing the compact representation of the ip
 142 address this peer sees you as. i.e. this is the
 143 receiver's external ip address (no port is included).
 144 This may be either an IPv4 (4 bytes) or an IPv6
 145 (16 bytes) address.</td>
 146 </tr>
 147 <tr><td>ipv6</td>
 148 <td>If this peer has an IPv6 interface, this is the compact
 149 representation of that address (16 bytes). The client may
 150 prefer to connect back via the IPv6 address.</td>
 151 </tr>
 152 <tr><td>ipv4</td>
 153 <td>If this peer has an IPv4 interface, this is the compact
 154 representation of that address (4 bytes). The client may
 155 prefer to connect back via this interface.</td>
 156 </tr>
 157 <tr><td>reqq</td>
 158 <td>An integer, the number of outstanding request messages
 159 this client supports without dropping any. The default in
 160 in libtorrent is 250.</td>
 161 </tr>
 162 </tbody>
 163 </table>
 164 <p>The handshake dictionary could also include extended handshake
 165 information, such as support for encrypted headers or anything
 166 imaginable.</p>
 167 <p>An example of what the payload of a handshake message could look like:</p>
 168 <table border="1" class="docutils">
 169 <colgroup>
 170 <col width="36%" />
 171 <col width="64%" />
 172 </colgroup>
 173 <thead valign="bottom">
 174 <tr><th class="head" colspan="2">Dictionary</th>
 175 </tr>
 176 </thead>
 177 <tbody valign="top">
 178 <tr><td><tt class="docutils literal"><span class="pre">m</span></tt></td>
 179 <td><table border="1" class="first last docutils">
 180 <colgroup>
 181 <col width="88%" />
 182 <col width="12%" />
 183 </colgroup>
 184 <thead valign="bottom">
 185 <tr><th class="head" colspan="2">Dictionary</th>
 186 </tr>
 187 </thead>
 188 <tbody valign="top">
 189 <tr><td><tt class="docutils literal"><span class="pre">LT_metadata</span></tt></td>
 190 <td>1</td>
 191 </tr>
 192 <tr><td><tt class="docutils literal"><span class="pre">ut_pex</span></tt></td>
 193 <td>2</td>
 194 </tr>
 195 </tbody>
 196 </table>
 197 </td>
 198 </tr>
 199 <tr><td><tt class="docutils literal"><span class="pre">p</span></tt></td>
 200 <td>6881</td>
 201 </tr>
 202 <tr><td><tt class="docutils literal"><span class="pre">v</span></tt></td>
 203 <td>&quot;µTorrent 1.2&quot;</td>
 204 </tr>
 205 </tbody>
 206 </table>
 207 <p>and in the encoded form:</p>
 208 <p><tt class="docutils literal"><span class="pre">d1:md11:LT_metadatai1e6:µT_PEXi2ee1:pi6881e1:v13:\xc2\xb5Torrent</span> <span class="pre">1.2e</span></tt></p>
 209 <p>To make sure the extension names do not collide by mistake, they should be
 210 prefixed with the two (or one) character code that is used to identify the
 211 client that introduced the extension. This applies for both the names of
 212 extension messages, and for any additional information put inside the
 213 top-level dictionary. All one and two byte identifiers are invalid to use
 214 unless defined by this specification.</p>
 215 <p>This message should be sent immediately after the standard bittorrent handshake
 216 to any peer that supports this extension protocol. It is valid to send the
 217 handshake message more than once during the lifetime of a connection,
 218 the sending client should not be disconnected. An implementation may choose
 219 to ignore the subsequent handshake messages (or parts of them).</p>
 220 <p>Subsequent handshake messages can be used to enable/disable extensions
 221 without restarting the connection. If a peer supports changing extensions
 222 at run time, it should note that the <tt class="docutils literal"><span class="pre">m</span></tt> dictionary is additive.
 223 It's enough that it contains the actual <em>CHANGES</em> to the extension list.
 224 To disable the support for <tt class="docutils literal"><span class="pre">LT_metadata</span></tt> at run-time, without affecting
 225 any other extensions, this message should be sent:
 226 <tt class="docutils literal"><span class="pre">d11:LT_metadatai0ee</span></tt>.
 227 As specified above, the value 0 is used to turn off an extension.</p>
 228 <p>The extension IDs must be stored for every peer, becuase every peer may have
 229 different IDs for the same extension.</p>
 230 <p>This specification, deliberately, does not specify any extensions such as
 231 peer-exchange or metadata exchange. This protocol is merely a transport
 232 for the actual extensions to the bittorrent protocol and the extensions
 233 named in the example above (such as <tt class="docutils literal"><span class="pre">p</span></tt>) are just examples of possible
 234 extensions.</p>
 235 </div>
 236 <div class="section">
 237 <h2><a id="rationale" name="rationale">rationale</a></h2>
 238 <p>The reason why the extension messages' IDs would be defined in the handshake
 239 is to avoid having a global registry of message IDs. Instead the names of the
 240 extension messages requires unique names, which is much easier to do without
 241 a global registry. The convention is to use a two letter prefix on the
 242 extension message names, the prefix would identify the client first
 243 implementing the extension message. e.g. <tt class="docutils literal"><span class="pre">LT_metadata</span></tt> is implemented by
 244 libtorrent, and hence it has the <tt class="docutils literal"><span class="pre">LT</span></tt> prefix.</p>
 245 <p>If the client supporting the extensions can decide which numbers the messages
 246 it receives will have, it means they are constants within that client. i.e.
 247 they can be used in <tt class="docutils literal"><span class="pre">switch</span></tt> statements. It's easy for the other end to
 248 store an array with the ID's we expect for each message and use that for
 249 lookups each time it sends an extension message.</p>
 250 <p>The reason for having a dictionary instead of having an array (using
 251 implicitly assigned index numbers to the extensions) is that if a client
 252 want to disable some extensions, the ID numbers would change, and it wouldn't
 253 be able to use constants (and hence, not use them in a <tt class="docutils literal"><span class="pre">switch</span></tt>). If the
 254 messages IDs would map directly to bittorrent message IDs, It would also make
 255 it possible to map extensions in the handshake to existing extensions with
 256 fixed message IDs.</p>
 257 <p>The reasoning behind having a single byte as extended message identifier is
 258 to follow the the bittorrent spec. with its single byte message identifiers.
 259 It is also considered to be enough. It won't limit the total number of
 260 extensions, only the number of extensions used simultaneously.</p>
 261 <p>The reason for using single byte identifiers for the standardized handshake
 262 identifiers is 1) The mainline DHT uses single byte identifiers. 2) Saves
 263 bandwidth. The only advantage of longer messages is that it makes the
 264 protocol more readable for a human, but the BT protocol wasn't designed to
 265 be a human readable protocol, so why bother.</p>
 266 </div>
 267 </div>
 268 </div>
 269 </body>
 270 </html>