[SyncFS] Build indexes from FileTracker entries on disk.

[chromium-blink-merge.git] / third_party / tlslite / README

tlslite version 0.4.6                                            Mar 20 2013
Trevor Perrin <tlslite at trevp.net>
http://trevp.net/tlslite/
============================================================================


Table of Contents
==================
1  Introduction
2  License/Acknowledgements
3  Installation
4  Getting Started with the Command-Line Tools
5  Getting Started with the Library
6  Using TLS Lite with httplib
7  Using TLS Lite with poplib or imaplib
8  Using TLS Lite with smtplib
9 Using TLS Lite with SocketServer
10 Using TLS Lite with asyncore
11 SECURITY CONSIDERATIONS
12 History


1 Introduction 
=============== 
TLS Lite is an open source python library that implements SSL and TLS. TLS
Lite supports RSA and SRP ciphersuites. TLS Lite is pure python, however it
can use other libraries for faster crypto operations. TLS Lite integrates with
several stdlib neworking libraries.

API documentation is available in the 'docs' directory.

If you have questions or feedback, feel free to contact me.  For discussing
improvements to tlslite, also see 'tlslite-dev@googlegroups.com'.


2 Licenses/Acknowledgements
============================
TLS Lite is written (mostly) by Trevor Perrin. It includes code from Bram
Cohen, Google, Kees Bos, Sam Rushing, Dimitris Moraitis, Marcelo Fernandez,
Martin von Loewis, and Dave Baggett.

All code in TLS Lite has either been dedicated to the public domain by its
authors, or placed under a BSD-style license. See the LICENSE file for
details.

Thanks to Edward Loper for Epydoc, which generated the API docs.


3 Installation
===============
Requirements:
  Python 2.6 or higher is required. Python 3 is supported.

Options:
  - If you have the M2Crypto interface to OpenSSL, this will be used for fast
    RSA operations and fast ciphers.

  - If you have pycrypto this will be used for fast RSA operations and fast
    ciphers.

  - If you have the GMPY interface to GMP, this will be used for fast RSA and
    SRP operations.

  - These modules don't need to be present at installation - you can install
    them any time.

Run 'python setup.py install'

Test the Installation:
  - From the distribution's ./tests subdirectory, run:
      ./tlstest.py server localhost:4443 .
  - While the test server is waiting, run:
      ./tlstest.py client localhost:4443 .

  If both say "Test succeeded" at the end, you're ready to go.


4 Getting Started with the Command-Line Tools
==============================================
tlslite installs two command-line scripts: 'tlsdb.py' and 'tls.py'.

'tls.py' lets you run test clients and servers. It can be used for testing
other TLS implementations, or as example code. Note that 'tls.py server' runs
an HTTPS server which will serve files rooted at the current directory by
default, so be careful.

'tlsdb.py' lets you manage SRP verifier databases. These databases are used by
a TLS server when authenticating clients with SRP.

X.509
------
To run an X.509 server, go to the ./tests directory and do:

  tls.py server -k serverX509Key.pem -c serverX509Cert.pem localhost:4443

Try connecting to the server with a web browser, or with:

  tls.py client localhost:4443

X.509 with TACK
----------------
To run an X.509 server using a TACK, install TACKpy, then run the same server
command as above with added arguments:

 ... -t TACK1.pem localhost:4443

SRP
----
To run an SRP server, try something like:

  tlsdb.py createsrp verifierDB
  tlsdb.py add verifierDB alice abra123cadabra 1024
  tlsdb.py add verifierDB bob swordfish 2048

  tls.py server -v verifierDB localhost:4443

Then try connecting to the server with:

  tls.py client localhost:4443 alice abra123cadabra

HTTPS
------
To run an HTTPS server with less typing, run ./tests/httpsserver.sh.

To run an HTTPS client, run ./tests/httpsclient.py.


5 Getting Started with the Library
===================================
Whether you're writing a client or server, there are six steps:

1) Create a socket and connect it to the other party.
2) Construct a TLSConnection instance with the socket.
3) Call a handshake function on TLSConnection to perform the TLS handshake.
4) Check the results to make sure you're talking to the right party.
5) Use the TLSConnection to exchange data.
6) Call close() on the TLSConnection when you're done.

TLS Lite also integrates with several stdlib python libraries. See the
sections following this one for details.

5 Step 1 - create a socket
---------------------------
Below demonstrates a socket connection to Amazon's secure site.

  from socket import *
  sock = socket(AF_INET, SOCK_STREAM)
  sock.connect( ("www.amazon.com", 443) )

5 Step 2 - construct a TLSConnection
-------------------------------------
You can import tlslite objects individually, such as:
  from tlslite import TLSConnection

Or import the most useful objects through:
  from tlslite.api import *

Then do:
  connection = TLSConnection(sock)

5 Step 3 - call a handshake function (client)
----------------------------------------------
If you're a client, there's two different handshake functions you can call,
depending on how you want to authenticate:

  connection.handshakeClientCert()
  connection.handshakeClientCert(certChain, privateKey)

  connection.handshakeClientSRP("alice", "abra123cadabra")

The ClientCert function without arguments is used when connecting to a site
like Amazon, which doesn't require client authentication, but which will
authenticate itself using an X.509 certificate chain.

The ClientCert function can also be used to do client authentication with an
X.509 certificate chain and corresponding private key. To use X.509 chains,
you'll need some way of creating these, such as OpenSSL (see
http://www.openssl.org/docs/HOWTO/ for details).

Below is an example of loading an X.509 chain and private key:
  
  from tlslite import X509, X509CertChain, parsePEMKey
  s = open("./test/clientX509Cert.pem").read()
  x509 = X509()
  x509.parse(s)
  certChain = X509CertChain([x509])
  s = open("./test/clientX509Key.pem").read()
  privateKey = parsePEMKey(s, private=True)

The SRP function does mutual authentication with a username and password - see
RFC 5054 for details.

If you want more control over the handshake, you can pass in a
HandshakeSettings instance. For example, if you're performing SRP, but you
only want to use SRP parameters of at least 2048 bits, and you only want to
use the AES-256 cipher, and you only want to allow TLS (version 3.1), not SSL
(version 3.0), you can do:

  settings = HandshakeSettings()
  settings.minKeySize = 2048
  settings.cipherNames = ["aes256"]
  settings.minVersion = (3,1)
  settings.useExperimentalTACKExtension = True  # Needed for TACK support

  connection.handshakeClientSRP("alice", "abra123cadabra", settings=settings)

If you want to check the server's certificate using TACK, you should set the
"useExperiementalTACKExtension" value in HandshakeSettings. (Eventually, TACK
support will be enabled by default, but for now it is an experimental feature
which relies on a temporary TLS Extension number, and should not be used for
production software.) This will cause the client to request the server to send
you a TACK (and/or any TACK Break Signatures):

Finally, every TLSConnection has a session object. You can try to resume a
previous session by passing in the session object from the old session. If the
server remembers this old session and supports resumption, the handshake will
finish more quickly. Otherwise, the full handshake will be done. For example:

  connection.handshakeClientSRP("alice", "abra123cadabra")
  .
  .
  oldSession = connection.session
  connection2.handshakeClientSRP("alice", "abra123cadabra", session=
  oldSession)

5 Step 3 - call a handshake function (server)
----------------------------------------------
If you're a server, there's only one handshake function, but you can pass it
several different parameters, depending on which types of authentication
you're willing to perform.

To perform SRP authentication, you have to pass in a database of password
verifiers.  The VerifierDB class manages an in-memory or on-disk verifier
database.

  verifierDB = VerifierDB("./test/verifierDB")
  verifierDB.open()
  connection.handshakeServer(verifierDB=verifierDB)

To perform authentication with a certificate and private key, the server must
load these as described in the previous section, then pass them in.  If the
server sets the reqCert boolean to True, a certificate chain will be requested
from the client.

  connection.handshakeServer(certChain=certChain, privateKey=privateKey,
                             reqCert=True)

You can pass in a verifier database and/or a certificate chain+private key.
The client will use one or both to authenticate the server.

You can also pass in a HandshakeSettings object, as described in the last
section, for finer control over handshaking details.

If you are passing in a certificate chain+private key, you may additionally
provide a TACK to assist the client in authenticating your certificate chain.
This requires the TACKpy library. Load a TACKpy.TACK object, then do:

  settings = HandshakeSettings()
  settings.useExperimentalTACKExtension = True  # Needed for TACK support

  connection.handshakeServer(certChain=certChain, privateKey=privateKey,
                             tack=tack, settings=settings)

Finally, the server can maintain a SessionCache, which will allow clients to
use session resumption:

  sessionCache = SessionCache()
  connection.handshakeServer(verifierDB=verifierDB, sessionCache=sessionCache)

It should be noted that the session cache, and the verifier databases, are all
thread-safe.

5 Step 4 - check the results
-----------------------------
If the handshake completes without raising an exception, authentication
results will be stored in the connection's session object.  The following
variables will be populated if applicable, or else set to None:

  connection.session.srpUsername       # string
  connection.session.clientCertChain   # X509CertChain
  connection.session.serverCertChain   # X509CertChain
  connection.session.tackExt           # TACKpy.TACK_Extension

X.509 chain objects return the end-entity fingerprint via getFingerprint(),
and ignore the other certificates.

TACK objects return the (validated) TACK ID via getTACKID().

To save yourself the trouble of inspecting certificates and/or TACKs after the
handshake, you can pass a Checker object into the handshake function. The
checker will be called if the handshake completes successfully. If the other
party isn't approved by the checker, a subclass of TLSAuthenticationError will
be raised.

If the handshake fails for any reason, including a Checker error, an exception
will be raised and the socket will be closed. If the socket timed out or was
unexpectedly closed, a socket.error or TLSAbruptCloseError will be raised.

Otherwise, either a TLSLocalAlert or TLSRemoteAlert will be raised, depending
on whether the local or remote implementation signalled the error. The
exception object has a 'description' member which identifies the error based
on the codes in RFC 2246. A TLSLocalAlert also has a 'message' string that may
have more details.

Example of handling a remote alert:

  try:
      [...]
  except TLSRemoteAlert as alert:
      if alert.description == AlertDescription.unknown_psk_identity:
          print "Unknown user."
  [...]

Below are some common alerts and their probable causes, and whether they are
signalled by the client or server.

Client handshake_failure:
 - SRP parameters are not recognized by client
 - Server's TACK was unrelated to its certificate chain

Client insufficient_security:
 - SRP parameters are too small

Client protocol_version:
 - Client doesn't support the server's protocol version

Server protocol_version:
 - Server doesn't support the client's protocol version

Server bad_record_mac:
 - bad SRP username or password

Server unknown_psk_identity
 - bad SRP username (bad_record_mac could be used for the same thing)

Server handshake_failure:
 - no matching cipher suites

5 Step 5 - exchange data
-------------------------
Now that you have a connection, you can call read() and write() as if it were
a socket.SSL object. You can also call send(), sendall(), recv(), and
makefile() as if it were a socket. These calls may raise TLSLocalAlert,
TLSRemoteAlert, socket.error, or TLSAbruptCloseError, just like the handshake
functions.

Once the TLS connection is closed by the other side, calls to read() or recv()
will return an empty string. If the socket is closed by the other side without
first closing the TLS connection, calls to read() or recv() will return a
TLSAbruptCloseError, and calls to write() or send() will return a
socket.error.

5 Step 6 - close the connection
--------------------------------
When you're finished sending data, you should call close() to close the
connection and socket. When the connection is closed properly, the session
object can be used for session resumption.

If an exception is raised the connection will be automatically closed; you
don't need to call close(). Furthermore, you will probably not be able to
re-use the socket, the connection object, or the session object, and you
shouldn't even try.

By default, calling close() will close the underlying socket. If you set the
connection's closeSocket flag to False, the socket will remain open after
close. (NOTE: some TLS implementations will not respond properly to the
close_notify alert that close() generates, so the connection will hang if
closeSocket is set to True.)


6 Using TLS Lite with httplib
==============================
TLS Lite comes with an HTTPTLSConnection class that extends httplib to work
over SSL/TLS connections.  Depending on how you construct it, it will do
different types of authentication.

  #No authentication whatsoever
  h = HTTPTLSConnection("www.amazon.com", 443)
  h.request("GET", "")
  r = h.getresponse()
  [...]

  #Authenticate server based on its TACK ID
  h = HTTPTLSConnection("localhost", 4443,
          tackID="B3ARS.EQ61B.F34EL.9KKLN.3WEW5", hardTack=False)
  [...]

  #Mutually authenticate with SRP
  h = HTTPTLSConnection("localhost", 443,
          username="alice", password="abra123cadabra")
  [...]


7 Using TLS Lite with poplib or imaplib
========================================
TLS Lite comes with POP3_TLS and IMAP4_TLS classes that extend poplib and
imaplib to work over SSL/TLS connections.  These classes can be constructed
with the same parameters as HTTPTLSConnection (see previous section), and 
behave similarly.

  #To connect to a POP3 server over SSL and display its fingerprint:
  from tlslite.api import *
  p = POP3_TLS("---------.net", port=995)
  print p.sock.session.serverCertChain.getFingerprint()
  [...]

  #To connect to an IMAP server once you know its fingerprint:
  from tlslite.api import *
  i = IMAP4_TLS("cyrus.andrew.cmu.edu",
          x509Fingerprint="00c14371227b3b677ddb9c4901e6f2aee18d3e45")
  [...]  
  

8 Using TLS Lite with smtplib
==============================
TLS Lite comes with an SMTP_TLS class that extends smtplib to work
over SSL/TLS connections.  This class accepts the same parameters as
HTTPTLSConnection (see previous section), and behaves similarly.  Depending 
on how you call starttls(), it will do different types of authentication.

  #To connect to an SMTP server once you know its fingerprint:
  from tlslite.api import *
  s = SMTP_TLS("----------.net", port=587)
  s.ehlo()
  s.starttls(x509Fingerprint="7e39be84a2e3a7ad071752e3001d931bf82c32dc")
  [...]


9 Using TLS Lite with SocketServer
====================================
You can use TLS Lite to implement servers using Python's SocketServer
framework.  TLS Lite comes with a TLSSocketServerMixIn class.  You can combine
this with a TCPServer such as HTTPServer.  To combine them, define a new class
that inherits from both of them (with the mix-in first). Then implement the
handshake() method, doing some sort of server handshake on the connection
argument.  If the handshake method returns True, the RequestHandler will be
triggered.  See the tests/httpsserver.py example.


10 Using TLS Lite with asyncore
================================
TLS Lite can be used with subclasses of asyncore.dispatcher.  See the comments
in TLSAsyncDispatcherMixIn.py for details.  This is still experimental, and
may not work with all asyncore.dispatcher subclasses.


11 Security Considerations
===========================
TLS Lite is beta-quality code. It hasn't received much security analysis. Use
at your own risk.

TLS Lite is probably vulnerable to the "Lucky 13" timing attack if AES or 3DES
are used.  Thus, TLS Lite prefers the RC4 cipher.


12 History
===========
0.4.6 - 3/20/2013
 - **API CHANGE**: TLSClosedConnectionError instead of ValueError when writing
   to a closed connection.  This inherits from socket.error, so should 
   interact better with SocketServer (see http://bugs.python.org/issue14574)
   and other things expecting a socket.error in this situation.
 - Added support for RC4-MD5 ciphersuite (if enabled in settings)
   - This is allegedly necessary to connect to some Internet servers.
 - Added TLSConnection.unread() function 
 - Switched to New-style classes (inherit from 'object')
 - Minor cleanups
 
0.4.5 - (release engineering problem, skipped!) 

0.4.4 - 2/25/2013
 - Added Python 3 support (Martin von Loewis)
 - Added NPN client support (Marcelo Fernandez)
 - Switched to RC4 as preferred cipher
   - faster in Python, avoids "Lucky 13" timing attacks
 - Fixed bug when specifying ciphers for anon ciphersuites
 - Made RSA hashAndVerify() tolerant of sigs w/o encoded NULL AlgorithmParam
   - (this function is not used for TLS currently, and this tolerance may
      not even be necessary)
0.4.3 - 9/27/2012
 - Minor bugfix (0.4.2 doesn't load tackpy)
0.4.2 - 9/25/2012
 - Updated TACK (compatible with tackpy 0.9.9)
0.4.1 - 5/22/2012
 - Fixed RSA padding bugs (w/help from John Randolph)
 - Updated TACK (compatible with tackpy 0.9.7)
 - Added SNI
 - Added NPN server support (Sam Rushing/Google)
 - Added AnonDH (Dimitris Moraitis)
 - Added X509CertChain.parsePemList
 - Improved XML-RPC (Kees Bos)

0.4.0 - 2/11/2012
 - Fixed pycrypto support
 - Fixed python 2.6 problems
 
0.3.9.x - 2/7/2012

Much code cleanup, in particular decomposing the handshake functions so they
are readable. The main new feature is support for TACK, an experimental
authentication method that provides a new way to pin server certificates (See
https://github.com/moxie0/Convergence/wiki/TACK ).

Also:

 - Security Fixes
   - Sends SCSV ciphersuite as per RFC 5746, to signal non-renegotiated
     Client Hello.  Does not support renegotiation (never has).
   - Change from e=3 to e=65537 for generated RSA keys, not strictly 
     necessary but mitigates risk of sloppy verifier.
   - 1/(n-1) countermeasure for BEAST.

 - Behavior changes:
   - Split cmdline into tls.py and tlstest.py, improved options.
   - Formalized LICENSE.
   - Defaults to closing socket after sending close_notify, fixes hanging.
     problem that would occur sometime when waiting for other party's    
     close_notify.
   - Update SRP to RFC 5054 compliance.
   - Removed client handshake "callbacks", no longer support the SRP 
     re-handshake idiom within a single handshake function.

 - Bugfixes
   - Added hashlib support, removes Deprecation Warning due to sha and md5.
   - Handled GeneratorExit exceptions that are a new Python feature, and
     interfere with the async code if not handled.
 
 - Removed:
   - Shared keys (it was based on an ancient I-D, not TLS-PSK).
   - cryptlib support, it wasn't used much, we have enough other options.
   - cryptoIDs (TACK is better).
   - win32prng extension module, as os.urandom is now available.
   - Twisted integration (unused?, slowed down loading).
   - Jython code (ancient, didn't work).
   - Compat support for python versions < 2.7.

 - Additions
   - Support for TACK via TACKpy.
   - Support for CertificateRequest.certificate_authorities ("reqCAs")
   - Added TLSConnection.shutdown() to better mimic socket.
   - Enabled Session resumption for XMLRPCTransport.

0.3.8 - 2/21/2005
 - Added support for poplib, imaplib, and smtplib
 - Added python 2.4 windows installer
 - Fixed occassional timing problems with test suite
0.3.7 - 10/05/2004
 - Added support for Python 2.2
 - Cleaned up compatibility code, and docs, a bit
0.3.6 - 9/28/2004
 - Fixed script installation on UNIX
 - Give better error message on old Python versions
0.3.5 - 9/16/2004
 - TLS 1.1 support
 - os.urandom() support
 - Fixed win32prng on some systems
0.3.4 - 9/12/2004
 - Updated for TLS/SRP draft 8
 - Bugfix: was setting _versioncheck on SRP 1st hello, causing problems
   with GnuTLS (which was offering TLS 1.1)
 - Removed _versioncheck checking, since it could cause interop problems
 - Minor bugfix: when cryptlib_py and and cryptoIDlib present, cryptlib
   was complaining about being initialized twice
0.3.3 - 6/10/2004
 - Updated for TLS/SRP draft 7
 - Updated test cryptoID cert chains for cryptoIDlib 0.3.1
0.3.2 - 5/21/2004
 - fixed bug when handling multiple handshake messages per record (e.g. IIS)
0.3.1 - 4/21/2004
 - added xmlrpclib integration
 - fixed hanging bug in Twisted integration
 - fixed win32prng to work on a wider range of win32 sytems
 - fixed import problem with cryptoIDlib
 - fixed port allocation problem when test scripts are run on some UNIXes
 - made tolerant of buggy IE sending wrong version in premaster secret
0.3.0 - 3/20/2004
 - added API docs thanks to epydoc
 - added X.509 path validation via cryptlib
 - much cleaning/tweaking/re-factoring/minor fixes
0.2.7 - 3/12/2004
 - changed Twisted error handling to use connectionLost()
 - added ignoreAbruptClose
0.2.6 - 3/11/2004
 - added Twisted errorHandler
 - added TLSAbruptCloseError
 - added 'integration' subdirectory
0.2.5 - 3/10/2004
 - improved asynchronous support a bit
 - added first-draft of Twisted support
0.2.4 - 3/5/2004
 - cleaned up asyncore support
 - added proof-of-concept for Twisted
0.2.3 - 3/4/2004
 - added pycrypto RSA support
 - added asyncore support
0.2.2 - 3/1/2004
 - added GMPY support
 - added pycrypto support
 - added support for PEM-encoded private keys, in pure python
0.2.1 - 2/23/2004
 - improved PRNG use (cryptlib, or /dev/random, or CryptoAPI)
 - added RSA blinding, to avoid timing attacks
 - don't install local copy of M2Crypto, too problematic
0.2.0 - 2/19/2004
 - changed VerifierDB to take per-user parameters
 - renamed tls_lite -> tlslite
0.1.9 - 2/16/2004
 - added post-handshake 'Checker'
 - made compatible with Python 2.2
 - made more forgiving of abrupt closure, since everyone does it:
   if the socket is closed while sending/recv'ing close_notify,
   just ignore it.
0.1.8 - 2/12/2004
 - TLSConnections now emulate sockets, including makefile()
 - HTTPTLSConnection and TLSMixIn simplified as a result
0.1.7 - 2/11/2004
 - fixed httplib.HTTPTLSConnection with multiple requests
 - fixed SocketServer to handle close_notify
 - changed handshakeClientNoAuth() to ignore CertificateRequests
 - changed handshakeClient() to ignore non-resumable session arguments
0.1.6 - 2/10/2004
 - fixed httplib support
0.1.5 - 2/09/2004
 - added support for httplib and SocketServer
 - added support for SSLv3
 - added support for 3DES
 - cleaned up read()/write() behavior
 - improved HMAC speed
0.1.4 - 2/06/2004
 - fixed dumb bug in tls.py
0.1.3 - 2/05/2004
 - change read() to only return requested number of bytes
 - added support for shared-key and in-memory databases
 - added support for PEM-encoded X.509 certificates
 - added support for SSLv2 ClientHello
 - fixed shutdown/re-handshaking behavior
 - cleaned up handling of missing_srp_username
 - renamed readString()/writeString() -> read()/write()
 - added documentation
0.1.2 - 2/04/2004
 - added clienttest/servertest functions
 - improved OpenSSL cipher wrappers speed
 - fixed server when it has a key, but client selects plain SRP
 - fixed server to postpone errors until it has read client's messages
 - fixed ServerHello to only include extension data if necessary
0.1.1 - 2/02/2004
 - fixed close_notify behavior
 - fixed handling of empty application data packets
 - fixed socket reads to not consume extra bytes
 - added testing functions to tls.py
0.1.0 - 2/01/2004
 - first release