www/iscurlsick.html

   1 <!DOCTYPE html>
   2 <!--
   3 "Is Your cURL Sick?" Page
   4 Copyright (C) 2015 Kyle J. McKay (mackyle a_t g_ma_il d_o_t _co_m)
   5 All rights reserved
   6
   7 Redistribution and use in source and binary forms, with or without
   8 modification, are permitted provided that the following conditions are met:
   9
  10  * Redistributions of source code must retain the above copyright notice, this
  11    list of conditions and the following disclaimer.
  12  * Redistributions in binary form must reproduce the above copyright notice,
  13    this list of conditions and the following disclaimer in the documentation
  14    and/or other materials provided with the distribution.
  15  * The names of the copyright holders or contributors may not be used to
  16    endorse or promote products derived from this software without specific
  17    prior written permission.
  18
  19 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
  20 ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
  21 WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
  22 DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
  23 ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
  24 (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
  25 LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
  26 ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
  27 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
  28 SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  29 -->
  30 <html xmlns="http://www.w3.org/1999/xhtml">
  31 <!--
  32 <head>
  33 -->
  34 <head>
  35   <meta charset='utf-8' />
  36   <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
  37   <script type="text/javascript">var NREUMQ=[];NREUMQ.f=function(){}</script>
  38
  39   <title>Is Your cURL Sick?</title>
  40
  41   <style type="text/css">
  42     body {
  43       margin-top: 1.0em;
  44       background-color: #f0efe7;
  45       font-family: Times, "Times New Roman", serif;
  46       color: black;
  47     }
  48     #container {
  49       margin: 0 auto;
  50       max-width: 90ex;
  51       position: relative;
  52     }
  53     @media screen {
  54       .last { margin-bottom: -1em;}
  55       .aftermargin {
  56         position: absolute;
  57         height: 100%;
  58         width: 1px;
  59       }
  60     }
  61     h1 { font-size: 2em; color: black; margin-bottom: 0.1em; }
  62     h1 .small { font-size: 0.4em; }
  63     h1 a { text-decoration: none; }
  64     h2 { font-size: 1.5em; color: black; }
  65     h3 { /* text-align: center;*/ color: black; }
  66     h1, h2, h3, th { font-family: Helvetica, Arial, FreeSans, sans-serif; font-weight: normal; }
  67     a { color: black; }
  68     .description { font-size: 1.2em; margin-bottom: 1em; margin-top: 1em; font-style: italic; margin-left: 3em; }
  69     .download { float: right; }
  70     tt, .nowrap { white-space: nowrap; font-size: 90%; }
  71     .breakword { word-wrap: break-word; }
  72     pre.breakword { white-space: pre-wrap; }
  73     pre, ttx { font-family: Menlo, Monaco, Consolas, Courier, "Courier New", monospace;}
  74     pre { font-size: 85%; padding: 0.5em 1ex; border: thin dotted silver; }
  75     hr { border: 0; width: 80%; border-bottom: thin solid silver; }
  76     p { text-align: justify; }
  77     p.left { text-align: left; }
  78     .inset { margin-left: 1ex; margin-right: 1ex; }
  79     .footer { text-align:center; padding-top:2em; font-style: italic; }
  80     ul.ulnone { list-style-type: none; }
  81     li { margin-top: 0; margin-bottom: 0; }
  82     .indent { display: block; margin-left: 5ex; margin-right: 5ex; text-align: left; }
  83     .spacer05 { display: inline-block; height: 1em; width: 0.5ex; }
  84     .spacer25 { display: inline-block; height: 1em; width: 2.5ex; }
  85
  86     .nobreak { white-space: nowrap; }
  87     .hash td { font-size: 80%; line-height: 1.0; font-family: monospace; }
  88     .hash tr > td:first-child { text-align: right; }
  89     .hash tr > td:first-child:after { content: ':'; }
  90
  91     table.download-list {
  92       border-collapse: separate;
  93       border-spacing: 1.25ex 0;
  94       width: 100%;
  95       margin-bottom: 0.5em;
  96     }
  97
  98     table.download-list > * > tr > td:first-child,
  99     table.download-list > * > tr > td:first-child + td,
 100     table.download-list > * > tr > td:first-child + td + td,
 101     table.download-list > * > tr > td:first-child + td + td + td {
 102       white-space: nowrap;
 103     }
 104
 105     tr.header th {
 106       text-align: left;
 107       border-bottom: thin solid black;
 108     }
 109     tr.separator {
 110       height: 0.25em;
 111     }
 112
 113     #credit {
 114       margin-top: 1.75em;
 115       display: block;
 116       width: 100%;
 117       text-align: center;
 118     }
 119     #credit span {
 120       color: gray;
 121       font-size: 60%;
 122     }
 123   </style>
 124 </head>
 125
 126 <body>
 127
 128 <div id="container">
 129
 130 <h1>Is Your cURL Sick?</h1>
 131
 132 <div class="description">
 133 <p>How well does it digest standards-format certificates &amp; keys?</p>
 134 </div>
 135
 136 <h2 id="summary">Summary</h2>
 137
 138 <p><a href="http://curl.haxx.se/">cURL</a> (typically via its library,
 139 <a href="http://curl.haxx.se/libcurl/">libcurl</a>) provides connection, verification and data transport services to many different pieces of software including <a href="http://git-scm.com/">Git</a>.</p>
 140
 141 <p>As part of the connection and verification step when creating a secure connection, certificates, certificate chains and certificate signatures may need to be verified.  Most versions of cURL include suitable defaults that allow secure connections to many servers to be verified without any special additional configuration.  But what happens if you need to supplement the defaults?</p>
 142
 143 <p>This page discusses how easily (or not) cURL (and libcurl)&#x2019;s default configuration can be supplemented with additional certificates and, when necessary, the corresponding certificate keys.</p>
 144
 145 <h2 id="quick">PKI Primer</h2>
 146
 147 <p>If you already understand how the Public Key Infrastructure (PKI) works, proceed to the next section.  Otherwise you may find <a href="pkiprimer.html">this companion document</a> to be a helpful read.</p>
 148
 149 <h2 id="backend">cURL&#x2019;s Backend</h2>
 150
 151 <p>To perform the actual cryptographic operations needed for secure connections, cURL relies on a cryptographic library.  cURL may be built to use one of several cryptographic libraries, but once cURL has been built, the cryptographic library selection is "baked in" and may not be changed.</p>
 152
 153 <p>To see which cryptographic backend cURL was built to use, run this command:</p>
 154 <pre>curl --version | sed -ne 1p | awk '{print $5}'</pre>
 155
 156 <p>cURL backends differ in their support for user-supplied certificates and keys.</p>
 157
 158 <h2 id="operation">Typical Operation and Defaults</h2>
 159
 160 <p>When curl connects securely to a server, the server sends its credentials and curl verifies them against its pre-configured set of root certificates and aborts the connection if verification fails.  By default curl never sends any client certificate credentials.</p>
 161
 162 <p>These defaults can be changed.  Curl can be told to ignore the server's credentials and connect no matter what, to use a different set of root certificates and/or to send client certificate credentials.</p>
 163
 164 <p>Client certificate credentials are not widely used but if required by the server and a satisfactory set of client certificate credentials are not sent, the connection will be dropped immediately by the server.</p>
 165
 166 <h2 id="roots">Planting New Roots</h2>
 167
 168 <p>cURL generally supports user-selected root certificates.  No matter what backend is in use, they all support designating a single root certificate in standard DER or PEM format as well as multiple root certificates in concatenated PEM (PEMSEQ) format.</p>
 169
 170 <p>Some backends support specifying multiple root certificates in a special directory format that depends on which backend is in use.</p>
 171
 172 <h2 id="yanking">Yanking Your Chain</h2>
 173
 174 <p>The X.509 PKIX Certificate standard (<a href="http://tools.ietf.org/html/rfc5280">RFC 5280</a>) requires a certificate chain to consist of at least two certificates &#x2013; a leaf certificate and a root certificate &#x2013; in order to be valid.</p>
 175
 176 <p>The Transport Layer Security (TLS) 1.2 standard (<a href="http://tools.ietf.org/html/rfc5246">RFC 5246</a> formerly known as the Secure Sockets Layer (SSL) standard) requires the server to send the entire server certificate chain (leaf through root) to the client and, if the client is providing a client certificate, the client must send the entire client certificate chain to the server.  There is, however, an exception.  The root certificate (of either chain) may be omitted (presumably the receiver already has it in its list of trusted roots, but it will have to search for it when omitted).</p>
 177
 178 <p>It&#x2019;s unlikely in the extreme that any commercially purchased leaf certificate would have a full certificate chain consisting of only two certificates (the leaf and root).  While client leaf certificates need not be commercially purchased, good security practices will make it similarly unlikely that such a client leaf certificate has a full certificate chain consisting of only two certificates.</p>
 179
 180 <h2 id="client">Certifiable Clients</h2>
 181
 182 <p>cURL allows the client certificate(s) to be configured.  However, some cURL backends only support setting a single client certificate (the leaf).</p>
 183
 184 <p>Even if the root certificate of the client certificate chain is omitted (as permitted by the standard), many client certificate chains will still have at least two client certificates left (the leaf and one (or more) intermediate certificates).</p>
 185
 186 <p>Attempting to make a secure connection to a server that requires such a client certificate chain using cURL built with a backend that only supports a single client certificate will <em>always</em> fail.</p>
 187
 188 <h2 id="problem">Problem Backends</h2>
 189
 190 <p>A cURL backend may have two different kinds of problems with client certificates:</p>
 191 <ol>
 192 <li>Client certificates and/or keys are not supported in the PEM or DER standard formats.</li>
 193 <li>Only the client leaf certificate may be specified, not the rest of the chain as required by the standard.</li>
 194 </ol>
 195
 196 <p>While the first problem can be worked around (assuming one can figure out how to munge the PEM/DER standards-format certificate(s) and key data into whatever format the backend expects), the second problem cannot and is a fatal flaw.</p>
 197
 198 <p>While it&#x2019;s relatively easy to test for standard formats support, checking for full client certificate chain support requires access to a suitably configured server.</p>
 199
 200 <p>It turns out, however, that when the first problem is present, the second is also likely present.  So testing for the first problem will provide a good indication of whether or not the second is present.</p>
 201
 202 <p>In order to provide multiple certificates for the client chain, the "PEM" format is required as the "DER" format can only provide a single certificate (or key).</p>
 203
 204 <h4 id="curltest">cURL PEM Client Certificate Support Test</h4>
 205
 206 <p>The following can be used to check your cURL to see what happens when you feed it PEM format client certificates:</p>
 207
 208 <pre>
 209 touch /tmp/mt
 210 nc -l localhost 2443&amp;
 211 curl -sv --cert-type PEM --cert /tmp/mt --key /tmp/mt https://localhost:2443/
 212 </pre>
 213
 214 <p id="results">If your cURL spews messages similar to these:</p>
 215
 216 <pre id="fail" class="breakword">
 217 * WARNING: SSL: CURLOPT_SSLKEY is ignored by Secure Transport. The private key must be in the Keychain.
 218 * WARNING: SSL: The Security framework only supports loading identities that are in PKCS#12 format.
 219 </pre>
 220
 221 <p>Then your cURL got sick and rejected the good PEM food.</p>
 222
 223 <p>Users of OS X 10.9 (Mavericks) and later (including 10.10 etc.) please note that the system&#x2019;s cURL library (installed as <tt>/usr/lib/libcurl.4.dylib</tt>) is known to suffer from this malady.  A version of cURL that does not have this problem on OS X 10.9 (and later) can be downloaded and installed as part of the <a href="http://mackyle.github.io/git-osx-installer/">Git OS X Installer</a>.</p>
 224
 225 <p>On the other hand, if you see messages like these:</p>
 226
 227 <pre id="success">
 228 * Connected to localhost (::1) port 2443 (#0)
 229 * unable to use client certificate (no key found or wrong pass phrase?)
 230 </pre>
 231
 232 <p>Or even like these:</p>
 233
 234 <pre>
 235 * Connected to localhost (127.0.0.1) port 2443 (#0)
 236 * unable to load certificate data file '/tmp/mt'
 237 </pre>
 238
 239 <p>Then your cURL is perfectly happy on a diet of PEM client certificates.</p>
 240
 241 <h4 id="gittest">Git Tested Too</h4>
 242
 243 <p>Git uses the cURL library for secure connections.  If you wish to determine whether or
 244 not Git can properly consume PEM format client certificates, the actual cURL library that
 245 Git&#x2019;s been linked with must be tested.  Testing the cURL library that&#x2019;s
 246 linked to the cURL command line client may give different results as that may, in fact,
 247 not be the same version of the cURL library that Git is linked to.</p>
 248
 249 <p>The following can be used to check the cURL library your Git client is linked with to see what happens when you feed it PEM format client certificates:</p>
 250
 251 <pre>
 252 touch /tmp/mt
 253 nc -l localhost 2443&amp;
 254 GIT_SSL_CERT=/tmp/mt GIT_SSL_KEY=/tmp/mt GIT_CURL_VERBOSE=1 \
 255   git ls-remote https://localhost:2443/
 256 </pre>
 257
 258 <p class="last">The same <a href="#results">results shown above</a> apply to this test as well.  If your Git suffers from PEM client certificate indigestion and you are using OS X (any version of Git that links to the system&#x2019;s cURL library on OS X 10.9 or later will have this problem), a version of Git that does not suffer from this malady can be downloaded and installed using the <a href="http://mackyle.github.io/git-osx-installer/">Git OS X Installer</a>.</p>
 259
 260 </div>
 261
 262 <span class="aftermargin"></span></body>
 263 <!--
 264 </body>
 265 -->
 266 </html>