doc/src/sgml/runtime.sgml

   1 <!-- $PostgreSQL$ -->
   2
   3 <chapter Id="runtime">
   4  <title>Server Setup and Operation</title>
   5
   6  <para>
   7   This chapter discusses how to set up and run the database server
   8   and its interactions with the operating system.
   9  </para>
  10
  11  <sect1 id="postgres-user">
  12   <title>The <productname>PostgreSQL</productname> User Account</title>
  13
  14   <indexterm>
  15    <primary>postgres user</primary>
  16   </indexterm>
  17
  18   <para>
  19    As with any other server daemon that is accessible to the outside world,
  20    it is advisable to run <productname>PostgreSQL</productname> under a
  21    separate user account. This user account should only own the data
  22    that is managed by the server, and should not be shared with other
  23    daemons. (For example, using the user <literal>nobody</literal> is a bad
  24    idea.) It is not advisable to install executables owned by this
  25    user because compromised systems could then modify their own
  26    binaries.
  27   </para>
  28
  29   <para>
  30    To add a Unix user account to your system, look for a command
  31    <command>useradd</command> or <command>adduser</command>. The user
  32    name <systemitem>postgres</systemitem> is often used, and is assumed
  33    throughout this book, but you can use another name if you like.
  34   </para>
  35  </sect1>
  36
  37  <sect1 id="creating-cluster">
  38   <title>Creating a Database Cluster</title>
  39
  40   <indexterm>
  41    <primary>database cluster</primary>
  42   </indexterm>
  43
  44   <indexterm>
  45    <primary>data area</primary>
  46    <see>database cluster</see>
  47   </indexterm>
  48
  49   <para>
  50    Before you can do anything, you must initialize a database storage
  51    area on disk. We call this a <firstterm>database cluster</firstterm>.
  52    (<acronym>SQL</acronym> uses the term catalog cluster.) A
  53    database cluster is a collection of databases that is managed by a
  54    single instance of a running database server. After initialization, a
  55    database cluster will contain a database named <literal>postgres</literal>,
  56    which is meant as a default database for use by utilities, users and third
  57    party applications.  The database server itself does not require the
  58    <literal>postgres</literal> database to exist, but many external utility
  59    programs assume it exists.  Another database created within each cluster
  60    during initialization is called
  61    <literal>template1</literal>.  As the name suggests, this will be used
  62    as a template for subsequently created databases; it should not be
  63    used for actual work.  (See <xref linkend="managing-databases"> for
  64    information about creating new databases within a cluster.)
  65   </para>
  66
  67   <para>
  68    In file system terms, a database cluster will be a single directory
  69    under which all data will be stored. We call this the <firstterm>data
  70    directory</firstterm> or <firstterm>data area</firstterm>. It is
  71    completely up to you where you choose to store your data.  There is no
  72    default, although locations such as
  73    <filename>/usr/local/pgsql/data</filename> or
  74    <filename>/var/lib/pgsql/data</filename> are popular. To initialize a
  75    database cluster, use the command <xref
  76    linkend="app-initdb">,<indexterm><primary>initdb</></> which is
  77    installed with <productname>PostgreSQL</productname>. The desired
  78    file system location of your database cluster is indicated by the
  79    <option>-D</option> option, for example:
  80 <screen>
  81 <prompt>$</> <userinput>initdb -D /usr/local/pgsql/data</userinput>
  82 </screen>
  83    Note that you must execute this command while logged into the
  84    <productname>PostgreSQL</productname> user account, which is
  85    described in the previous section.
  86   </para>
  87
  88   <tip>
  89    <para>
  90     As an alternative to the <option>-D</option> option, you can set
  91     the environment variable <envar>PGDATA</envar>.
  92     <indexterm><primary><envar>PGDATA</envar></primary></indexterm>
  93    </para>
  94   </tip>
  95
  96   <para>
  97    <command>initdb</command> will attempt to create the directory you
  98    specify if it does not already exist. It is likely that it will not
  99    have the permission to do so (if you followed our advice and created
 100    an unprivileged account). In that case you should create the
 101    directory yourself (as root) and change the owner to be the
 102    <productname>PostgreSQL</productname> user. Here is how this might
 103    be done:
 104 <screen>
 105 root# <userinput>mkdir /usr/local/pgsql/data</userinput>
 106 root# <userinput>chown postgres /usr/local/pgsql/data</userinput>
 107 root# <userinput>su postgres</userinput>
 108 postgres$ <userinput>initdb -D /usr/local/pgsql/data</userinput>
 109 </screen>
 110   </para>
 111
 112   <para>
 113    <command>initdb</command> will refuse to run if the data directory
 114    looks like it has already been initialized.</para>
 115
 116   <para>
 117    Because the data directory contains all the data stored in the
 118    database, it is essential that it be secured from unauthorized
 119    access. <command>initdb</command> therefore revokes access
 120    permissions from everyone but the
 121    <productname>PostgreSQL</productname> user.
 122   </para>
 123
 124   <para>
 125    However, while the directory contents are secure, the default
 126    client authentication setup allows any local user to connect to the
 127    database and even become the database superuser. If you do not
 128    trust other local users, we recommend you use one of
 129    <command>initdb</command>'s <option>-W</option>, <option>--pwprompt</option>
 130    or <option>--pwfile</option> options to assign a password to the
 131    database superuser.<indexterm><primary>password</><secondary>of the
 132    superuser</></indexterm>  Also, specify <option>-A md5</> or
 133    <option>-A password</> so that the default <literal>trust</> authentication
 134    mode is not used; or modify the generated <filename>pg_hba.conf</filename>
 135    file after running <command>initdb</command>,
 136    <emphasis>before</> you start the server for the first time. (Other
 137    reasonable approaches include using <literal>ident</literal> authentication
 138    or file system permissions to restrict connections. See <xref
 139    linkend="client-authentication"> for more information.)
 140   </para>
 141
 142   <para>
 143    <command>initdb</command> also initializes the default
 144    locale<indexterm><primary>locale</></> for the database cluster.
 145    Normally, it will just take the locale settings in the environment
 146    and apply them to the initialized database.  It is possible to
 147    specify a different locale for the database; more information about
 148    that can be found in <xref linkend="locale">.  The default sort order used
 149    within the particular database cluster is set by
 150    <command>initdb</command>, and while you can create new databases using
 151    different sort order, the order used in the template databases that initdb
 152    creates cannot be changed without dropping and recreating them.
 153    There is also a performance impact for using locales
 154    other than <literal>C</> or <literal>POSIX</>. Therefore, it is
 155    important to make this choice correctly the first time.
 156   </para>
 157
 158   <para>
 159    <command>initdb</command> also sets the default character set encoding
 160    for the database cluster.  Normally this should be chosen to match the
 161    locale setting.  For details see <xref linkend="multibyte">.
 162   </para>
 163
 164   <sect2 id="creating-cluster-nfs">
 165    <title>Network File Systems</title>
 166
 167    <indexterm zone="creating-cluster-nfs">
 168     <primary>Network File Systems</primary>
 169    </indexterm>
 170    <indexterm><primary><acronym>NFS</></><see>Network File Systems</></>
 171    <indexterm><primary>Network Attached Storage (<acronym>NAS</>)</><see>Network File Systems</></>
 172
 173    <para>
 174     Many installations create database clusters on network file systems.
 175     Sometimes this is done directly via <acronym>NFS</>, or by using a
 176     Network Attached Storage (<acronym>NAS</>) device that uses
 177     <acronym>NFS</> internally.  <productname>PostgreSQL</> does nothing
 178     special for <acronym>NFS</> file systems, meaning it assumes
 179     <acronym>NFS</> behaves exactly like locally-connected drives
 180     (<acronym>DAS</>, Direct Attached Storage).  If client and server
 181     <acronym>NFS</> implementations have non-standard semantics, this can
 182     cause reliability problems (see <ulink
 183     url="http://www.time-travellers.org/shane/papers/NFS_considered_harmful.html"></ulink>).
 184     Specifically, delayed (asynchronous) writes to the <acronym>NFS</>
 185     server can cause reliability problems;   if possible, mount
 186     <acronym>NFS</> file systems synchronously (without caching) to avoid
 187     this.  Also, soft-mounting <acronym>NFS</> is not recommended.
 188     (Storage Area Networks (<acronym>SAN</>) use a low-level
 189     communication protocol rather than <acronym>NFS</>.)
 190    </para>
 191
 192   </sect2>
 193
 194  </sect1>
 195
 196  <sect1 id="server-start">
 197   <title>Starting the Database Server</title>
 198
 199   <para>
 200    Before anyone can access the database, you must start the database
 201    server. The database server program is called
 202    <command>postgres</command>.<indexterm><primary>postgres</></>
 203    The <command>postgres</command> program must know where to
 204    find the data it is supposed to use. This is done with the
 205    <option>-D</option> option. Thus, the simplest way to start the
 206    server is:
 207 <screen>
 208 $ <userinput>postgres -D /usr/local/pgsql/data</userinput>
 209 </screen>
 210    which will leave the server running in the foreground. This must be
 211    done while logged into the <productname>PostgreSQL</productname> user
 212    account. Without <option>-D</option>, the server will try to use
 213    the data directory named by the environment variable <envar>PGDATA</envar>.
 214    If that variable is not provided either, it will fail.
 215   </para>
 216
 217   <para>
 218    Normally it is better to start <command>postgres</command> in the
 219    background.  For this, use the usual shell syntax:
 220 <screen>
 221 $ <userinput>postgres -D /usr/local/pgsql/data &gt;logfile 2&gt;&amp;1 &amp;</userinput>
 222 </screen>
 223    It is important to store the server's <systemitem>stdout</> and
 224    <systemitem>stderr</> output somewhere, as shown above. It will help
 225    for auditing purposes and to diagnose problems. (See <xref
 226    linkend="logfile-maintenance"> for a more thorough discussion of log
 227    file handling.)
 228   </para>
 229
 230   <para>
 231    The <command>postgres</command> program also takes a number of other
 232    command-line options. For more information, see the
 233    <xref linkend="app-postgres"> reference page
 234    and <xref linkend="runtime-config"> below.
 235   </para>
 236
 237   <para>
 238    This shell syntax can get tedious quickly.  Therefore the wrapper
 239    program
 240    <xref linkend="app-pg-ctl"><indexterm><primary>pg_ctl</primary></indexterm>
 241    is provided to simplify some tasks.  For example:
 242 <programlisting>
 243 pg_ctl start -l logfile
 244 </programlisting>
 245    will start the server in the background and put the output into the
 246    named log file. The <option>-D</option> option has the same meaning
 247    here as for <command>postgres</command>. <command>pg_ctl</command>
 248    is also capable of stopping the server.
 249   </para>
 250
 251   <para>
 252    Normally, you will want to start the database server when the
 253    computer boots.<indexterm><primary>booting</><secondary>starting
 254    the server during</></> Autostart scripts are operating-system-specific.
 255    There are a few distributed with
 256    <productname>PostgreSQL</productname> in the
 257    <filename>contrib/start-scripts</> directory. Installing one will require
 258    root privileges.
 259   </para>
 260
 261   <para>
 262    Different systems have different conventions for starting up daemons
 263    at boot time. Many systems have a file
 264    <filename>/etc/rc.local</filename> or
 265    <filename>/etc/rc.d/rc.local</filename>. Others use
 266    <filename>rc.d</> directories. Whatever you do, the server must be
 267    run by the <productname>PostgreSQL</productname> user account
 268    <emphasis>and not by root</emphasis> or any other user. Therefore you
 269    probably should form your commands using <literal>su -c '...'
 270    postgres</literal>.  For example:
 271 <programlisting>
 272 su -c 'pg_ctl start -D /usr/local/pgsql/data -l serverlog' postgres
 273 </programlisting>
 274   </para>
 275
 276   <para>
 277    Here are a few more operating-system-specific suggestions. (In each
 278    case be sure to use the proper installation directory and user
 279    name where we show generic values.)
 280
 281    <itemizedlist>
 282     <listitem>
 283      <para>
 284       For <productname>FreeBSD</productname>, look at the file
 285       <filename>contrib/start-scripts/freebsd</filename> in the
 286       <productname>PostgreSQL</productname> source distribution.
 287       <indexterm><primary>FreeBSD</><secondary>start script</secondary></>
 288      </para>
 289     </listitem>
 290
 291     <listitem>
 292      <para>
 293       On <productname>OpenBSD</productname>, add the following lines
 294       to the file <filename>/etc/rc.local</filename>:
 295       <indexterm><primary>OpenBSD</><secondary>start script</secondary></>
 296 <programlisting>
 297 if [ -x /usr/local/pgsql/bin/pg_ctl -a -x /usr/local/pgsql/bin/postgres ]; then
 298     su - -c '/usr/local/pgsql/bin/pg_ctl start -l /var/postgresql/log -s' postgres
 299     echo -n ' postgresql'
 300 fi
 301 </programlisting>
 302      </para>
 303     </listitem>
 304
 305     <listitem>
 306      <para>
 307       On <productname>Linux</productname> systems either add
 308       <indexterm><primary>Linux</><secondary>start script</secondary></>
 309 <programlisting>
 310 /usr/local/pgsql/bin/pg_ctl start -l logfile -D /usr/local/pgsql/data
 311 </programlisting>
 312       to <filename>/etc/rc.d/rc.local</filename> or look at the file
 313       <filename>contrib/start-scripts/linux</filename> in the
 314       <productname>PostgreSQL</productname> source distribution.
 315      </para>
 316     </listitem>
 317
 318     <listitem>
 319      <para>
 320       On <productname>NetBSD</productname>, either use the
 321       <productname>FreeBSD</productname> or
 322       <productname>Linux</productname> start scripts, depending on
 323       preference. <indexterm><primary>NetBSD</><secondary>start script</secondary></>
 324      </para>
 325     </listitem>
 326
 327     <listitem>
 328      <para>
 329       On <productname>Solaris</productname>, create a file called
 330       <filename>/etc/init.d/postgresql</filename> that contains
 331       the following line:
 332       <indexterm><primary>Solaris</><secondary>start script</secondary></>
 333 <programlisting>
 334 su - postgres -c "/usr/local/pgsql/bin/pg_ctl start -l logfile -D /usr/local/pgsql/data"
 335 </programlisting>
 336       Then, create a symbolic link to it in <filename>/etc/rc3.d</> as
 337       <filename>S99postgresql</>.
 338      </para>
 339     </listitem>
 340    </itemizedlist>
 341
 342   </para>
 343
 344    <para>
 345     While the server is running, its
 346     <acronym>PID</acronym> is stored in the file
 347     <filename>postmaster.pid</filename> in the data directory. This is
 348     used to prevent multiple server instances from
 349     running in the same data directory and can also be used for
 350     shutting down the server.
 351    </para>
 352
 353    <sect2 id="server-start-failures">
 354     <title>Server Start-up Failures</title>
 355
 356     <para>
 357      There are several common reasons the server might fail to
 358      start. Check the server's log file, or start it by hand (without
 359      redirecting standard output or standard error) and see what error
 360      messages appear. Below we explain some of the most common error
 361      messages in more detail.
 362     </para>
 363
 364     <para>
 365 <screen>
 366 LOG:  could not bind IPv4 socket: Address already in use
 367 HINT:  Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
 368 FATAL:  could not create TCP/IP listen socket
 369 </screen>
 370      This usually means just what it suggests: you tried to start
 371      another server on the same port where one is already running.
 372      However, if the kernel error message is not <computeroutput>Address
 373      already in use</computeroutput> or some variant of that, there might
 374      be a different problem. For example, trying to start a server
 375      on a reserved port number might draw something like:
 376 <screen>
 377 $ <userinput>postgres -p 666</userinput>
 378 LOG:  could not bind IPv4 socket: Permission denied
 379 HINT:  Is another postmaster already running on port 666? If not, wait a few seconds and retry.
 380 FATAL:  could not create TCP/IP listen socket
 381 </screen>
 382     </para>
 383
 384     <para>
 385      A message like:
 386 <screen>
 387 FATAL:  could not create shared memory segment: Invalid argument
 388 DETAIL:  Failed system call was shmget(key=5440001, size=4011376640, 03600).
 389 </screen>
 390      probably means your kernel's limit on the size of shared memory is
 391      smaller than the work area <productname>PostgreSQL</productname>
 392      is trying to create (4011376640 bytes in this example). Or it could
 393      mean that you do not have System-V-style shared memory support
 394      configured into your kernel at all. As a temporary workaround, you
 395      can try starting the server with a smaller-than-normal number of
 396      buffers (<xref linkend="guc-shared-buffers">). You will eventually want
 397      to reconfigure your kernel to increase the allowed shared memory
 398      size. You might also see this message when trying to start multiple
 399      servers on the same machine, if their total space requested
 400      exceeds the kernel limit.
 401     </para>
 402
 403     <para>
 404      An error like:
 405 <screen>
 406 FATAL:  could not create semaphores: No space left on device
 407 DETAIL:  Failed system call was semget(5440126, 17, 03600).
 408 </screen>
 409      does <emphasis>not</emphasis> mean you've run out of disk
 410      space. It means your kernel's limit on the number of <systemitem
 411      class="osname">System V</> semaphores is smaller than the number
 412      <productname>PostgreSQL</productname> wants to create. As above,
 413      you might be able to work around the problem by starting the
 414      server with a reduced number of allowed connections
 415      (<xref linkend="guc-max-connections">), but you'll eventually want to
 416      increase the kernel limit.
 417     </para>
 418
 419     <para>
 420      If you get an <quote>illegal system call</> error, it is likely that
 421      shared memory or semaphores are not supported in your kernel at
 422      all. In that case your only option is to reconfigure the kernel to
 423      enable these features.
 424     </para>
 425
 426     <para>
 427      Details about configuring <systemitem class="osname">System V</>
 428      <acronym>IPC</> facilities are given in <xref linkend="sysvipc">.
 429     </para>
 430    </sect2>
 431
 432    <sect2 id="client-connection-problems">
 433     <title>Client Connection Problems</title>
 434
 435     <para>
 436      Although the error conditions possible on the client side are quite
 437      varied and application-dependent, a few of them might be directly
 438      related to how the server was started up. Conditions other than
 439      those shown below should be documented with the respective client
 440      application.
 441     </para>
 442
 443     <para>
 444 <screen>
 445 psql: could not connect to server: Connection refused
 446         Is the server running on host "server.joe.com" and accepting
 447         TCP/IP connections on port 5432?
 448 </screen>
 449      This is the generic <quote>I couldn't find a server to talk
 450      to</quote> failure. It looks like the above when TCP/IP
 451      communication is attempted. A common mistake is to forget to
 452      configure the server to allow TCP/IP connections.
 453     </para>
 454
 455     <para>
 456      Alternatively, you'll get this when attempting Unix-domain socket
 457      communication to a local server:
 458 <screen>
 459 psql: could not connect to server: No such file or directory
 460         Is the server running locally and accepting
 461         connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
 462 </screen>
 463     </para>
 464
 465     <para>
 466      The last line is useful in verifying that the client is trying to
 467      connect to the right place. If there is in fact no server
 468      running there, the kernel error message will typically be either
 469      <computeroutput>Connection refused</computeroutput> or
 470      <computeroutput>No such file or directory</computeroutput>, as
 471      illustrated. (It is important to realize that
 472      <computeroutput>Connection refused</computeroutput> in this context
 473      does <emphasis>not</emphasis> mean that the server got your
 474      connection request and rejected it. That case will produce a
 475      different message, as shown in <xref
 476      linkend="client-authentication-problems">.) Other error messages
 477      such as <computeroutput>Connection timed out</computeroutput> might
 478      indicate more fundamental problems, like lack of network
 479      connectivity.
 480     </para>
 481    </sect2>
 482   </sect1>
 483
 484  <sect1 id="kernel-resources">
 485   <title>Managing Kernel Resources</title>
 486
 487   <para>
 488    A large <productname>PostgreSQL</> installation can quickly exhaust
 489    various operating system resource limits. (On some systems, the
 490    factory defaults are so low that you don't even need a really
 491    <quote>large</> installation.) If you have encountered this kind of
 492    problem, keep reading.
 493   </para>
 494
 495   <sect2 id="sysvipc">
 496    <title>Shared Memory and Semaphores</title>
 497
 498    <indexterm zone="sysvipc">
 499     <primary>shared memory</primary>
 500    </indexterm>
 501
 502    <indexterm zone="sysvipc">
 503     <primary>semaphores</primary>
 504    </indexterm>
 505
 506    <para>
 507     Shared memory and semaphores are collectively referred to as
 508     <quote><systemitem class="osname">System V</>
 509     <acronym>IPC</></quote> (together with message queues, which are not
 510     relevant for <productname>PostgreSQL</>). Almost all modern
 511     operating systems provide these features, but not all of them have
 512     them turned on or sufficiently sized by default, especially systems
 513     with BSD heritage. (On <systemitem class="osname">Windows</>,
 514     <productname>PostgreSQL</> provides its own replacement
 515     implementation of these facilities, and so most of this section
 516     can be disregarded.)
 517    </para>
 518
 519    <para>
 520     The complete lack of these facilities is usually manifested by an
 521     <errorname>Illegal system call</> error upon server start. In
 522     that case there's nothing left to do but to reconfigure your
 523     kernel.  <productname>PostgreSQL</> won't work without them.
 524    </para>
 525
 526    <para>
 527     When <productname>PostgreSQL</> exceeds one of the various hard
 528     <acronym>IPC</> limits, the server will refuse to start and
 529     should leave an instructive error message describing the problem
 530     encountered and what to do about it. (See also <xref
 531     linkend="server-start-failures">.) The relevant kernel
 532     parameters are named consistently across different systems; <xref
 533     linkend="sysvipc-parameters"> gives an overview. The methods to set
 534     them, however, vary. Suggestions for some platforms are given below.
 535     Be warned that it is often necessary to reboot your machine, and
 536     possibly even recompile the kernel, to change these settings.
 537    </para>
 538
 539
 540    <table id="sysvipc-parameters">
 541     <title><systemitem class="osname">System V</> <acronym>IPC</> parameters</>
 542
 543     <tgroup cols="3">
 544      <thead>
 545       <row>
 546        <entry>Name</>
 547        <entry>Description</>
 548        <entry>Reasonable values</>
 549       </row>
 550      </thead>
 551
 552      <tbody>
 553       <row>
 554        <entry><varname>SHMMAX</></>
 555        <entry>Maximum size of shared memory segment (bytes)</>
 556        <entry>at least several megabytes (see text)</entry>
 557       </row>
 558
 559       <row>
 560        <entry><varname>SHMMIN</></>
 561        <entry>Minimum size of shared memory segment (bytes)</>
 562        <entry>1</>
 563       </row>
 564
 565       <row>
 566        <entry><varname>SHMALL</></>
 567        <entry>Total amount of shared memory available (bytes or pages)</>
 568        <entry>if bytes, same as <varname>SHMMAX</varname>; if pages, <literal>ceil(SHMMAX/PAGE_SIZE)</literal></>
 569       </row>
 570
 571       <row>
 572        <entry><varname>SHMSEG</></>
 573        <entry>Maximum number of shared memory segments per process</>
 574        <entry>only 1 segment is needed, but the default is much higher</>
 575       </row>
 576
 577        <row>
 578         <entry><varname>SHMMNI</></>
 579         <entry>Maximum number of shared memory segments system-wide</>
 580         <entry>like <varname>SHMSEG</> plus room for other applications</>
 581        </row>
 582
 583        <row>
 584         <entry><varname>SEMMNI</></>
 585         <entry>Maximum number of semaphore identifiers (i.e., sets)</>
 586         <entry>at least <literal>ceil((max_connections + autovacuum_max_workers) / 16)</literal></>
 587        </row>
 588
 589        <row>
 590         <entry><varname>SEMMNS</></>
 591         <entry>Maximum number of semaphores system-wide</>
 592         <entry><literal>ceil((max_connections + autovacuum_max_workers) / 16) * 17</literal> plus room for other applications</>
 593        </row>
 594
 595        <row>
 596         <entry><varname>SEMMSL</></>
 597         <entry>Maximum number of semaphores per set</>
 598         <entry>at least 17</>
 599        </row>
 600
 601        <row>
 602         <entry><varname>SEMMAP</></>
 603         <entry>Number of entries in semaphore map</>
 604         <entry>see text</>
 605        </row>
 606
 607        <row>
 608         <entry><varname>SEMVMX</></>
 609         <entry>Maximum value of semaphore</>
 610         <entry>at least 1000 (The default is often 32767, don't change unless forced to)</>
 611        </row>
 612
 613      </tbody>
 614     </tgroup>
 615    </table>
 616
 617
 618    <para>
 619     <indexterm><primary>SHMMAX</primary></indexterm> The most important
 620     shared memory parameter is <varname>SHMMAX</>, the maximum size, in
 621     bytes, of a shared memory segment. If you get an error message from
 622     <function>shmget</> like <errorname>Invalid argument</>, it is
 623     likely that this limit has been exceeded.  The size of the required
 624     shared memory segment varies depending on several
 625     <productname>PostgreSQL</> configuration parameters, as shown in
 626     <xref linkend="shared-memory-parameters">.  (Any error message you might
 627     get will include the exact size of the failed allocation request.)
 628     You can, as a temporary solution, lower some of those settings to
 629     avoid the failure.  While it is possible to get
 630     <productname>PostgreSQL</> to run with <varname>SHMMAX</> as small as
 631     2 MB, you need considerably more for acceptable performance.  Desirable
 632     settings are in the tens to hundreds of megabytes.
 633    </para>
 634
 635    <para>
 636     Some systems also have a limit on the total amount of shared memory in
 637     the system (<varname>SHMALL</>).  Make sure this is large enough
 638     for <productname>PostgreSQL</> plus any other applications that
 639     are using shared memory segments.  (Caution: <varname>SHMALL</>
 640     is measured in pages rather than bytes on many systems.)
 641    </para>
 642
 643    <para>
 644     Less likely to cause problems is the minimum size for shared
 645     memory segments (<varname>SHMMIN</>), which should be at most
 646     approximately 500 kB for <productname>PostgreSQL</> (it is
 647     usually just 1). The maximum number of segments system-wide
 648     (<varname>SHMMNI</>) or per-process (<varname>SHMSEG</>) are unlikely
 649     to cause a problem unless your system has them set to zero.
 650    </para>
 651
 652    <para>
 653     <productname>PostgreSQL</> uses one semaphore per allowed connection
 654     (<xref linkend="guc-max-connections">) and allowed autovacuum worker
 655     process (<xref linkend="guc-autovacuum-max-workers">), in sets of 16.
 656     Each such set will
 657     also contain a 17th semaphore which contains a <quote>magic
 658     number</quote>, to detect collision with semaphore sets used by
 659     other applications. The maximum number of semaphores in the system
 660     is set by <varname>SEMMNS</>, which consequently must be at least
 661     as high as <varname>max_connections</> plus
 662     <varname>autovacuum_max_workers</>, plus one extra for each 16
 663     allowed connections plus workers (see the formula in <xref
 664     linkend="sysvipc-parameters">).  The parameter <varname>SEMMNI</>
 665     determines the limit on the number of semaphore sets that can
 666     exist on the system at one time.  Hence this parameter must be at
 667     least <literal>ceil((max_connections + autovacuum_max_workers) / 16)</>.
 668     Lowering the number
 669     of allowed connections is a temporary workaround for failures,
 670     which are usually confusingly worded <errorname>No space
 671     left on device</>, from the function <function>semget</>.
 672    </para>
 673
 674    <para>
 675     In some cases it might also be necessary to increase
 676     <varname>SEMMAP</> to be at least on the order of
 677     <varname>SEMMNS</>. This parameter defines the size of the semaphore
 678     resource map, in which each contiguous block of available semaphores
 679     needs an entry. When a semaphore set is freed it is either added to
 680     an existing entry that is adjacent to the freed block or it is
 681     registered under a new map entry. If the map is full, the freed
 682     semaphores get lost (until reboot). Fragmentation of the semaphore
 683     space could over time lead to fewer available semaphores than there
 684     should be.
 685    </para>
 686
 687    <para>
 688     The <varname>SEMMSL</> parameter, which determines how many
 689     semaphores can be in a set, must be at least 17 for
 690     <productname>PostgreSQL</>.
 691    </para>
 692
 693    <para>
 694     Various other settings related to <quote>semaphore undo</>, such as
 695     <varname>SEMMNU</> and <varname>SEMUME</>, are not of concern
 696     for <productname>PostgreSQL</>.
 697    </para>
 698
 699
 700     <variablelist>
 701      <varlistentry>
 702       <term><systemitem class="osname">AIX</></term>
 703       <indexterm><primary>AIX</><secondary>IPC configuration</></>
 704       <listitem>
 705        <para>
 706         At least as of version 5.1, it should not be necessary to do
 707         any special configuration for such parameters as
 708         <varname>SHMMAX</varname>, as it appears this is configured to
 709         allow all memory to be used as shared memory.  That is the
 710         sort of configuration commonly used for other databases such
 711         as <application>DB/2</application>.</para>
 712
 713        <para> It might, however, be necessary to modify the global
 714        <command>ulimit</command> information in
 715        <filename>/etc/security/limits</filename>, as the default hard
 716        limits for file sizes (<varname>fsize</varname>) and numbers of
 717        files (<varname>nofiles</varname>) might be too low.
 718        </para>
 719       </listitem>
 720      </varlistentry>
 721
 722      <varlistentry>
 723       <term><systemitem class="osname">BSD/OS</></term>
 724       <indexterm><primary>BSD/OS</><secondary>IPC configuration</></>
 725       <listitem>
 726        <formalpara>
 727         <title>Shared Memory</>
 728         <para>
 729          By default, only 4 MB of shared memory is supported. Keep in
 730          mind that shared memory is not pageable; it is locked in RAM.
 731          To increase the amount of shared memory supported by your
 732          system, add something like the following to your kernel configuration
 733          file:
 734 <programlisting>
 735 options "SHMALL=8192"
 736 options "SHMMAX=\(SHMALL*PAGE_SIZE\)"
 737 </programlisting>
 738          <varname>SHMALL</> is measured in 4 kB pages, so a value of
 739          1024 represents 4 MB of shared memory. Therefore the above increases
 740          the maximum shared memory area to 32 MB.
 741          For those running 4.3 or later, you will probably also need to increase
 742          <varname>KERNEL_VIRTUAL_MB</> above the default <literal>248</>.
 743          Once all changes have been made, recompile the kernel, and reboot.
 744         </para>
 745        </formalpara>
 746
 747        <para>
 748         For those running 4.0 and earlier releases, use <command>bpatch</>
 749         to find the <varname>sysptsize</> value in the current
 750         kernel. This is computed dynamically at boot time.
 751 <screen>
 752 $ <userinput>bpatch -r sysptsize</>
 753 <computeroutput>0x9 = 9</>
 754 </screen>
 755         Next, add <varname>SYSPTSIZE</> as a hard-coded value in the
 756         kernel configuration file. Increase the value you found using
 757         <command>bpatch</>. Add 1 for every additional 4 MB of
 758         shared memory you desire.
 759 <programlisting>
 760 options "SYSPTSIZE=16"
 761 </programlisting>
 762         <varname>sysptsize</> cannot be changed by <command>sysctl</command>.
 763        </para>
 764
 765        <formalpara>
 766         <title>Semaphores</>
 767         <para>
 768          You will probably want to increase the number of semaphores
 769          as well; the default system total of 60 will only allow about
 770          50 <productname>PostgreSQL</productname> connections.  Set the
 771          values you want in your kernel configuration file, e.g.:
 772 <programlisting>
 773 options "SEMMNI=40"
 774 options "SEMMNS=240"
 775 </programlisting>
 776         </para>
 777        </formalpara>
 778       </listitem>
 779      </varlistentry>
 780
 781
 782      <varlistentry>
 783       <term><systemitem class="osname">FreeBSD</></term>
 784       <indexterm><primary>FreeBSD</><secondary>IPC configuration</></>
 785       <listitem>
 786        <para>
 787         The default settings are only suitable for small installations
 788         (for example, default <varname>SHMMAX</varname> is 32
 789         MB). Changes can be made via the <command>sysctl</command> or
 790         <command>loader</command> interfaces.  The following
 791         parameters can be set using <command>sysctl</command>:
 792 <screen>
 793 <prompt>$</prompt> <userinput>sysctl -w kern.ipc.shmall=32768</userinput>
 794 <prompt>$</prompt> <userinput>sysctl -w kern.ipc.shmmax=134217728</userinput>
 795 <prompt>$</prompt> <userinput>sysctl -w kern.ipc.semmap=256</userinput>
 796 </screen>
 797         To have these settings persist over reboots, modify
 798         <filename>/etc/sysctl.conf</filename>.
 799        </para>
 800
 801        <para>
 802         The remaining semaphore settings are read-only as far as
 803         <command>sysctl</command> is concerned, but can be changed
 804         before boot using the <command>loader</command> prompt:
 805 <screen>
 806 <prompt>(loader)</prompt> <userinput>set kern.ipc.semmni=256</userinput>
 807 <prompt>(loader)</prompt> <userinput>set kern.ipc.semmns=512</userinput>
 808 <prompt>(loader)</prompt> <userinput>set kern.ipc.semmnu=256</userinput>
 809 </screen>
 810         Similarly these can be saved between reboots in
 811         <filename>/boot/loader.conf</filename>.
 812        </para>
 813
 814        <para>
 815         You might also want to configure your kernel to lock shared
 816         memory into RAM and prevent it from being paged out to swap.
 817         This can be accomplished using the <command>sysctl</command>
 818         setting <literal>kern.ipc.shm_use_phys</literal>.
 819        </para>
 820
 821        <para>
 822         If running in FreeBSD jails by enabling <application>sysctl</>'s
 823         <literal>security.jail.sysvipc_allowed</>, <application>postmaster</>s
 824         running in different jails should be run by different operating system
 825         users.  This improves security because it prevents non-root users
 826         from interfering with shared memory or semaphores in a different jail,
 827         and it allows the PostgreSQL IPC cleanup code to function properly.
 828         (In FreeBSD 6.0 and later the IPC cleanup code doesn't properly detect
 829         processes in other jails, preventing the running of postmasters on the
 830         same port in different jails.)
 831        </para>
 832
 833        <para>
 834         <systemitem class="osname">FreeBSD</> versions before 4.0 work like
 835         <systemitem class="osname">NetBSD</> and <systemitem class="osname">
 836         OpenBSD</> (see below).
 837        </para>
 838       </listitem>
 839      </varlistentry>
 840
 841      <varlistentry>
 842       <term><systemitem class="osname">NetBSD</></term>
 843       <term><systemitem class="osname">OpenBSD</></term>
 844       <indexterm><primary>NetBSD</><secondary>IPC configuration</></>
 845       <indexterm><primary>OpenBSD</><secondary>IPC configuration</></>
 846       <listitem>
 847        <para>
 848         The options <varname>SYSVSHM</> and <varname>SYSVSEM</> need
 849         to be enabled when the kernel is compiled. (They are by
 850         default.) The maximum size of shared memory is determined by
 851         the option <varname>SHMMAXPGS</> (in pages). The following
 852         shows an example of how to set the various parameters
 853         (<systemitem class="osname">OpenBSD</> uses <literal>option</> instead):
 854 <programlisting>
 855 options        SYSVSHM
 856 options        SHMMAXPGS=4096
 857 options        SHMSEG=256
 858
 859 options        SYSVSEM
 860 options        SEMMNI=256
 861 options        SEMMNS=512
 862 options        SEMMNU=256
 863 options        SEMMAP=256
 864 </programlisting>
 865        </para>
 866
 867        <para>
 868         You might also want to configure your kernel to lock shared
 869         memory into RAM and prevent it from being paged out to swap.
 870         This can be accomplished using the <command>sysctl</command>
 871         setting <literal>kern.ipc.shm_use_phys</literal>.
 872        </para>
 873       </listitem>
 874      </varlistentry>
 875
 876
 877      <varlistentry>
 878       <term><systemitem class="osname">HP-UX</></term>
 879       <indexterm><primary>HP-UX</><secondary>IPC configuration</></>
 880       <listitem>
 881        <para>
 882         The default settings tend to suffice for normal installations.
 883         On <productname>HP-UX</> 10, the factory default for
 884         <varname>SEMMNS</> is 128, which might be too low for larger
 885         database sites.
 886        </para>
 887        <para>
 888         <acronym>IPC</> parameters can be set in the <application>System
 889         Administration Manager</> (<acronym>SAM</>) under
 890         <menuchoice><guimenu>Kernel
 891         Configuration</><guimenuitem>Configurable Parameters</></>. Hit
 892         <guibutton>Create A New Kernel</> when you're done.
 893        </para>
 894       </listitem>
 895      </varlistentry>
 896
 897
 898      <varlistentry>
 899       <term><systemitem class="osname">Linux</></term>
 900       <indexterm><primary>Linux</><secondary>IPC configuration</></>
 901       <listitem>
 902        <para>
 903         The default maximum segment size is 32 MB, which is only adequate
 904         for small <productname>PostgreSQL</productname> installations.
 905         However, the remaining
 906         defaults are quite generously sized, and usually do not require
 907         changes. The maximum shared memory segment size can be changed via the
 908         <command>sysctl</command> interface.  For example, to allow 128 MB,
 909         and explicitly set the maximum total shared memory size to 2097152
 910         pages (the default):
 911 <screen>
 912 <prompt>$</prompt> <userinput>sysctl -w kernel.shmmax=134217728</userinput>
 913 <prompt>$</prompt> <userinput>sysctl -w kernel.shmall=2097152</userinput>
 914 </screen>
 915         In addition these settings can be saved between reboots in
 916         <filename>/etc/sysctl.conf</filename>.
 917        </para>
 918
 919        <para>
 920         Older distributions might not have the <command>sysctl</command> program,
 921         but equivalent changes can be made by manipulating the
 922         <filename>/proc</filename> file system:
 923 <screen>
 924 <prompt>$</prompt> <userinput>echo 134217728 &gt;/proc/sys/kernel/shmmax</userinput>
 925 <prompt>$</prompt> <userinput>echo 2097152 &gt;/proc/sys/kernel/shmall</userinput>
 926 </screen>
 927        </para>
 928       </listitem>
 929      </varlistentry>
 930
 931
 932      <varlistentry>
 933       <term><systemitem class="osname">MacOS X</></term>
 934       <indexterm><primary>MacOS X</><secondary>IPC configuration</></>
 935       <listitem>
 936        <para>
 937         In OS X 10.2 and earlier, edit the file
 938         <filename>/System/Library/StartupItems/SystemTuning/SystemTuning</>
 939         and change the values in the following commands:
 940 <programlisting>
 941 sysctl -w kern.sysv.shmmax
 942 sysctl -w kern.sysv.shmmin
 943 sysctl -w kern.sysv.shmmni
 944 sysctl -w kern.sysv.shmseg
 945 sysctl -w kern.sysv.shmall
 946 </programlisting>
 947        </para>
 948
 949        <para>
 950         In OS X 10.3 and later, these commands have been moved to
 951         <filename>/etc/rc</> and must be edited there.  Note that
 952         <filename>/etc/rc</> is usually overwritten by OS X updates (such as
 953         10.3.6 to 10.3.7) so you should expect to have to redo your editing
 954         after each update.
 955        </para>
 956
 957        <para>
 958         In OS X 10.3.9 and later, instead of editing <filename>/etc/rc</>
 959         you can create a file named <filename>/etc/sysctl.conf</>,
 960         containing variable assignments such as:
 961 <programlisting>
 962 kern.sysv.shmmax=4194304
 963 kern.sysv.shmmin=1
 964 kern.sysv.shmmni=32
 965 kern.sysv.shmseg=8
 966 kern.sysv.shmall=1024
 967 </programlisting>
 968         This method is better than editing <filename>/etc/rc</> because
 969         your changes will be preserved across system updates.  Note that
 970         <emphasis>all five</> shared-memory parameters must be set in
 971         <filename>/etc/sysctl.conf</>, else the values will be ignored.
 972        </para>
 973
 974        <para>
 975         Beware that recent releases of OS X ignore attempts to set
 976         <varname>SHMMAX</> to a value that isn't an exact multiple of 4096.
 977        </para>
 978
 979        <para>
 980         <varname>SHMALL</> is measured in 4 kB pages on this platform.
 981        </para>
 982
 983        <para>
 984         In all OS X versions, you'll need to reboot to make changes in the
 985         shared memory parameters take effect.
 986        </para>
 987       </listitem>
 988      </varlistentry>
 989
 990
 991      <varlistentry>
 992       <term><systemitem class="osname">SCO OpenServer</></term>
 993       <indexterm><primary>SCO OpenServer</><secondary>IPC configuration</></>
 994       <listitem>
 995        <para>
 996         In the default configuration, only 512 kB of shared memory per
 997         segment is allowed. To increase the setting, first change to the
 998         directory <filename>/etc/conf/cf.d</>. To display the current value of
 999         <varname>SHMMAX</>, run:
1000 <programlisting>
1001 ./configure -y SHMMAX
1002 </programlisting>
1003         To set a new value for <varname>SHMMAX</>, run:
1004 <programlisting>
1005 ./configure SHMMAX=<replaceable>value</>
1006 </programlisting>
1007         where <replaceable>value</> is the new value you want to use
1008         (in bytes). After setting <varname>SHMMAX</>, rebuild the kernel:
1009 <programlisting>
1010 ./link_unix
1011 </programlisting>
1012         and reboot.
1013        </para>
1014       </listitem>
1015      </varlistentry>
1016
1017
1018      <varlistentry>
1019       <term><systemitem class="osname">Solaris</></term>
1020       <indexterm><primary>Solaris</><secondary>IPC configuration</></>
1021       <listitem>
1022        <para>
1023         At least in version 2.6, the default maximum size of a shared
1024         memory segment is too low for <productname>PostgreSQL</>. The
1025         relevant settings can be changed in <filename>/etc/system</>,
1026         for example:
1027 <programlisting>
1028 set shmsys:shminfo_shmmax=0x2000000
1029 set shmsys:shminfo_shmmin=1
1030 set shmsys:shminfo_shmmni=256
1031 set shmsys:shminfo_shmseg=256
1032
1033 set semsys:seminfo_semmap=256
1034 set semsys:seminfo_semmni=512
1035 set semsys:seminfo_semmns=512
1036 set semsys:seminfo_semmsl=32
1037 </programlisting>
1038         You need to reboot for the changes to take effect.
1039        </para>
1040
1041        <para>
1042         See also <ulink
1043         url="http://sunsite.uakom.sk/sunworldonline/swol-09-1997/swol-09-insidesolaris.html"></>
1044         for information on shared memory under
1045         <productname>Solaris</>.
1046        </para>
1047       </listitem>
1048      </varlistentry>
1049
1050
1051      <varlistentry>
1052       <term><systemitem class="osname">UnixWare</></term>
1053       <indexterm><primary>UnixWare</><secondary>IPC configuration</></>
1054       <listitem>
1055        <para>
1056         On <productname>UnixWare</> 7, the maximum size for shared
1057         memory segments is only 512 kB in the default configuration.
1058         To display the current value of <varname>SHMMAX</>, run:
1059 <programlisting>
1060 /etc/conf/bin/idtune -g SHMMAX
1061 </programlisting>
1062         which displays the current, default, minimum, and maximum
1063         values. To set a new value for <varname>SHMMAX</>,
1064         run:
1065 <programlisting>
1066 /etc/conf/bin/idtune SHMMAX <replaceable>value</>
1067 </programlisting>
1068         where <replaceable>value</> is the new value you want to use
1069         (in bytes). After setting <varname>SHMMAX</>, rebuild the
1070         kernel:
1071 <programlisting>
1072 /etc/conf/bin/idbuild -B
1073 </programlisting>
1074         and reboot.
1075        </para>
1076       </listitem>
1077      </varlistentry>
1078
1079     </variablelist>
1080
1081
1082    <table id="shared-memory-parameters">
1083     <title><productname>PostgreSQL</productname> shared memory usage</>
1084
1085     <tgroup cols="2">
1086      <thead>
1087       <row>
1088        <entry>Usage</>
1089        <entry>Approximate shared memory bytes required (as of 8.3)</>
1090       </row>
1091      </thead>
1092
1093      <tbody>
1094       <row>
1095        <entry>Connections</>
1096        <entry>(1800 + 270 * <xref
1097        linkend="guc-max-locks-per-transaction">) * <xref
1098        linkend="guc-max-connections"></entry>
1099       </row>
1100
1101       <row>
1102        <entry>Autovacuum workers</>
1103        <entry>(1800 + 270 * <xref
1104        linkend="guc-max-locks-per-transaction">) * <xref
1105        linkend="guc-autovacuum-max-workers"></entry>
1106       </row>
1107
1108       <row>
1109        <entry>Prepared transactions</>
1110        <entry>(770 + 270 * <xref
1111        linkend="guc-max-locks-per-transaction">) * <xref linkend="guc-max-prepared-transactions"></entry>
1112       </row>
1113
1114       <row>
1115        <entry>Shared disk buffers</>
1116        <entry>(<xref linkend="guc-block-size"> + 208) * <xref linkend="guc-shared-buffers"></entry>
1117       </row>
1118
1119       <row>
1120        <entry>WAL buffers</>
1121        <entry>(<xref linkend="guc-wal-block-size"> + 8) * <xref linkend="guc-wal-buffers"></entry>
1122       </row>
1123
1124       <row>
1125        <entry>Fixed space requirements</>
1126        <entry>770 kB</entry>
1127       </row>
1128      </tbody>
1129     </tgroup>
1130    </table>
1131
1132   </sect2>
1133
1134   <sect2>
1135    <title>Resource Limits</title>
1136
1137    <para>
1138     Unix-like operating systems enforce various kinds of resource limits
1139     that might interfere with the operation of your
1140     <productname>PostgreSQL</productname> server. Of particular
1141     importance are limits on the number of processes per user, the
1142     number of open files per process, and the amount of memory available
1143     to each process. Each of these have a <quote>hard</quote> and a
1144     <quote>soft</quote> limit. The soft limit is what actually counts
1145     but it can be changed by the user up to the hard limit. The hard
1146     limit can only be changed by the root user. The system call
1147     <function>setrlimit</function> is responsible for setting these
1148     parameters. The shell's built-in command <command>ulimit</command>
1149     (Bourne shells) or <command>limit</command> (<application>csh</>) is
1150     used to control the resource limits from the command line. On
1151     BSD-derived systems the file <filename>/etc/login.conf</filename>
1152     controls the various resource limits set during login. See the
1153     operating system documentation for details. The relevant
1154     parameters are <varname>maxproc</varname>,
1155     <varname>openfiles</varname>, and <varname>datasize</varname>. For
1156     example:
1157 <programlisting>
1158 default:\
1159 ...
1160         :datasize-cur=256M:\
1161         :maxproc-cur=256:\
1162         :openfiles-cur=256:\
1163 ...
1164 </programlisting>
1165     (<literal>-cur</literal> is the soft limit.  Append
1166     <literal>-max</literal> to set the hard limit.)
1167    </para>
1168
1169    <para>
1170     Kernels can also have system-wide limits on some resources.
1171     <itemizedlist>
1172      <listitem>
1173       <para>
1174       On <productname>Linux</productname>
1175       <filename>/proc/sys/fs/file-max</filename> determines the
1176       maximum number of open files that the kernel will support.  It can
1177       be changed by writing a different number into the file or by
1178       adding an assignment in <filename>/etc/sysctl.conf</filename>.
1179       The maximum limit of files per process is fixed at the time the
1180       kernel is compiled; see
1181       <filename>/usr/src/linux/Documentation/proc.txt</filename> for
1182       more information.
1183       </para>
1184      </listitem>
1185     </itemizedlist>
1186    </para>
1187
1188    <para>
1189     The <productname>PostgreSQL</productname> server uses one process
1190     per connection so you should provide for at least as many processes
1191     as allowed connections, in addition to what you need for the rest
1192     of your system.  This is usually not a problem but if you run
1193     several servers on one machine things might get tight.
1194    </para>
1195
1196    <para>
1197     The factory default limit on open files is often set to
1198     <quote>socially friendly</quote> values that allow many users to
1199     coexist on a machine without using an inappropriate fraction of
1200     the system resources.  If you run many servers on a machine this
1201     is perhaps what you want, but on dedicated servers you might want to
1202     raise this limit.
1203    </para>
1204
1205    <para>
1206     On the other side of the coin, some systems allow individual
1207     processes to open large numbers of files; if more than a few
1208     processes do so then the system-wide limit can easily be exceeded.
1209     If you find this happening, and you do not want to alter the
1210     system-wide limit, you can set <productname>PostgreSQL</>'s <xref
1211     linkend="guc-max-files-per-process"> configuration parameter to
1212     limit the consumption of open files.
1213    </para>
1214   </sect2>
1215
1216   <sect2>
1217    <title>Linux Memory Overcommit</title>
1218
1219    <para>
1220     In Linux 2.4 and later, the default virtual memory behavior is not
1221     optimal for <productname>PostgreSQL</productname>. Because of the
1222     way that the kernel implements memory overcommit, the kernel might
1223     terminate the <productname>PostgreSQL</productname> server (the
1224     master server process) if the memory demands of
1225     another process cause the system to run out of virtual memory.
1226    </para>
1227
1228    <para>
1229     If this happens, you will see a kernel message that looks like
1230     this (consult your system documentation and configuration on where
1231     to look for such a message):
1232 <programlisting>
1233 Out of Memory: Killed process 12345 (postgres).
1234 </programlisting>
1235     This indicates that the <filename>postgres</filename> process
1236     has been terminated due to memory pressure.
1237     Although existing database connections will continue to function
1238     normally, no new connections will be accepted.  To recover,
1239     <productname>PostgreSQL</productname> will need to be restarted.
1240    </para>
1241
1242    <para>
1243     One way to avoid this problem is to run
1244     <productname>PostgreSQL</productname> on a machine where you can
1245     be sure that other processes will not run the machine out of
1246     memory.  If memory is tight, increasing the swap space of the
1247     operating system can help avoiding the problem, because the
1248     out-of-memory (OOM) killer is invoked whenever physical memory and
1249     swap space are exhausted.
1250    </para>
1251
1252    <para>
1253     On Linux 2.6 and later, an additional measure is to modify the
1254     kernel's behavior so that it will not <quote>overcommit</> memory.
1255     Although this setting will not prevent the <ulink
1256     url="http://lwn.net/Articles/104179/">OOM killer</> from being invoked
1257     altogether, it will lower the chances significantly and will therefore
1258     lead to more robust system behavior.  This is done by selecting strict
1259     overcommit mode via <command>sysctl</command>:
1260 <programlisting>
1261 sysctl -w vm.overcommit_memory=2
1262 </programlisting>
1263     or placing an equivalent entry in <filename>/etc/sysctl.conf</>.
1264     You might also wish to modify the related setting
1265     <literal>vm.overcommit_ratio</>.  For details see the kernel documentation
1266     file <filename>Documentation/vm/overcommit-accounting</>.
1267    </para>
1268
1269    <para>
1270     Some vendors' Linux 2.4 kernels are reported to have early versions
1271     of the 2.6 overcommit <command>sysctl</command> parameter.  However, setting
1272     <literal>vm.overcommit_memory</> to 2
1273     on a kernel that does not have the relevant code will make
1274     things worse not better.  It is recommended that you inspect
1275     the actual kernel source code (see the function
1276     <function>vm_enough_memory</> in the file <filename>mm/mmap.c</>)
1277     to verify what is supported in your copy before you try this in a 2.4
1278     installation.  The presence of the <filename>overcommit-accounting</>
1279     documentation file should <emphasis>not</> be taken as evidence that the
1280     feature is there.  If in any doubt, consult a kernel expert or your
1281     kernel vendor.
1282    </para>
1283   </sect2>
1284  </sect1>
1285
1286
1287  <sect1 id="server-shutdown">
1288   <title>Shutting Down the Server</title>
1289
1290   <indexterm zone="server-shutdown">
1291    <primary>shutdown</>
1292   </indexterm>
1293
1294   <para>
1295    There are several ways to shut down the database server. You control
1296    the type of shutdown by sending different signals to the master
1297    <command>postgres</command> process.
1298
1299    <variablelist>
1300     <varlistentry>
1301      <term><systemitem>SIGTERM</systemitem><indexterm><primary>SIGTERM</></></term>
1302      <listitem>
1303       <para>
1304        This is the <firstterm>Smart Shutdown</firstterm> mode.
1305        After receiving <systemitem>SIGTERM</systemitem>, the server
1306        disallows new connections, but lets existing sessions end their
1307        work normally. It shuts down only after all of the sessions terminate.
1308        If the server is in online backup mode, it additionally waits
1309        until online backup mode is no longer active.  While backup mode is
1310        active, new connections will still be allowed, but only to superusers
1311        (this exception allows a superuser to connect to terminate
1312        online backup mode).
1313       </para>
1314      </listitem>
1315     </varlistentry>
1316
1317     <varlistentry>
1318      <term><systemitem>SIGINT</systemitem><indexterm><primary>SIGINT</></></term>
1319      <listitem>
1320       <para>
1321        This is the <firstterm>Fast Shutdown</firstterm> mode.
1322        The server disallows new connections and sends all existing
1323        server processes <systemitem>SIGTERM</systemitem>, which will cause them
1324        to abort their current transactions and exit promptly. It then
1325        waits for the server processes to exit and finally shuts down.
1326        If the server is in online backup mode, backup mode will be
1327        terminated, rendering the backup useless.
1328       </para>
1329      </listitem>
1330     </varlistentry>
1331
1332     <varlistentry>
1333      <term><systemitem>SIGQUIT</systemitem><indexterm><primary>SIGQUIT</></></term>
1334      <listitem>
1335       <para>
1336       This is the <firstterm>Immediate Shutdown</firstterm> mode.
1337       The master <command>postgres</command> process will send a
1338       <systemitem>SIGQUIT</systemitem> to all child processes and exit
1339       immediately, without properly shutting itself down. The child processes
1340       likewise exit immediately upon receiving
1341       <systemitem>SIGQUIT</systemitem>. This will lead to recovery (by
1342       replaying the WAL log) upon next start-up. This is recommended
1343       only in emergencies.
1344       </para>
1345      </listitem>
1346     </varlistentry>
1347    </variablelist>
1348   </para>
1349
1350   <para>
1351    The <xref linkend="app-pg-ctl"> program provides a convenient
1352    interface for sending these signals to shut down the server.
1353    Alternatively, you can send the signal directly using <command>kill</>
1354    on non-Windows systems.
1355    The <acronym>PID</> of the <command>postgres</command> process can be
1356    found using the <command>ps</command> program, or from the file
1357    <filename>postmaster.pid</filename> in the data directory. For
1358    example, to do a fast shutdown:
1359 <screen>
1360 $ <userinput>kill -INT `head -1 /usr/local/pgsql/data/postmaster.pid`</userinput>
1361 </screen>
1362   </para>
1363
1364   <important>
1365    <para>
1366     It is best not to use <systemitem>SIGKILL</systemitem> to shut down
1367     the server.  Doing so will prevent the server from releasing
1368     shared memory and semaphores, which might then have to be done
1369     manually before a new server can be started.  Furthermore,
1370     <systemitem>SIGKILL</systemitem> kills the <command>postgres</command>
1371     process without letting it relay the signal to its subprocesses,
1372     so it will be necessary to kill the individual subprocesses by hand as
1373     well.
1374    </para>
1375   </important>
1376
1377   <para>
1378    To terminate an individual session while allowing other sessions to
1379    continue, use <function>pg_terminate_backend()</> (see <xref
1380    linkend="functions-admin-signal-table">) or send a
1381    <systemitem>SIGTERM</> signal to the child process associated with
1382    the session.
1383   </para>
1384  </sect1>
1385
1386  <sect1 id="preventing-server-spoofing">
1387   <title>Preventing Server Spoofing</title>
1388
1389   <indexterm zone="preventing-server-spoofing">
1390    <primary>server spoofing</primary>
1391   </indexterm>
1392
1393   <para>
1394    While the server is running, it is not possible for a malicious user
1395    to take the place of the normal database server.  However, when the
1396    server is down it is possible for a local user to spoof the normal
1397    server by starting their own server.  The spoof server could read
1398    passwords and queries sent by clients, but could not return any data
1399    because the <varname>PGDATA</> directory would still be secure because
1400    of directory permissions. Spoofing is possible because any user can
1401    start a database server; a client cannot identify an invalid server
1402    unless it is specially configured.
1403   </para>
1404
1405   <para>
1406    The simplest way to prevent invalid servers for <literal>local</>
1407    connections is to use a Unix domain socket directory (<xref
1408    linkend="guc-unix-socket-directory">) that has write permission only
1409    for a trusted local user.  This prevents a malicious user from creating
1410    their own socket file in that directory.  If you are concerned that
1411    some applications might still reference <filename>/tmp</> for the
1412    socket file and hence be vulnerable to spoofing, during operating system
1413    startup create symbolic link <filename>/tmp/.s.PGSQL.5432</> that points
1414    to the relocated socket file.  You also might need to modify your
1415    <filename>/tmp</> cleanup script to preserve the symbolic link.
1416   </para>
1417
1418   <para>
1419    For TCP connections the server
1420    must accept only <literal>hostssl</> connections (<xref
1421    linkend="auth-pg-hba-conf">) and have SSL
1422    <filename>server.key</filename> (key) and
1423    <filename>server.crt</filename> (certificate) files (<xref
1424    linkend="ssl-tcp">). The TCP client must connect using
1425    <literal>sslmode='verify-ca'</> or
1426    <literal>'verify-full'</> and have the required certificate
1427    files present (<xref linkend="libpq-connect">).
1428   </para>
1429  </sect1>
1430
1431  <sect1 id="encryption-options">
1432   <title>Encryption Options</title>
1433
1434   <indexterm zone="encryption-options">
1435    <primary>encryption</primary>
1436   </indexterm>
1437
1438   <para>
1439    <productname>PostgreSQL</productname> offers encryption at several
1440    levels, and provides flexibility in protecting data from disclosure
1441    due to database server theft, unscrupulous administrators, and
1442    insecure networks. Encryption might also be required to secure
1443    sensitive data such as medical records or financial transactions.
1444   </para>
1445
1446   <variablelist>
1447
1448   <varlistentry>
1449    <term>Password Storage Encryption</term>
1450    <listitem>
1451
1452     <para>
1453      By default, database user passwords are stored as MD5 hashes, so
1454      the administrator cannot determine the actual password assigned
1455      to the user. If MD5 encryption is used for client authentication,
1456      the unencrypted password is never even temporarily present on the
1457      server because the client MD5 encrypts it before being sent
1458      across the network.
1459     </para>
1460    </listitem>
1461   </varlistentry>
1462
1463   <varlistentry>
1464    <term>Encryption For Specific Columns</term>
1465
1466    <listitem>
1467     <para>
1468      The <filename>contrib</> function library
1469      <function>pgcrypto</function> allows certain fields to be stored
1470      encrypted. This is useful if only some of the data is sensitive.
1471      The client supplies the decryption key and the data is decrypted
1472      on the server and then sent to the client.
1473     </para>
1474
1475     <para>
1476      The decrypted data and the decryption key are present on the
1477      server for a brief time while it is being decrypted and
1478      communicated between the client and server. This presents a brief
1479      moment where the data and keys can be intercepted by someone with
1480      complete access to the database server, such as the system
1481      administrator.
1482     </para>
1483    </listitem>
1484   </varlistentry>
1485
1486   <varlistentry>
1487    <term>Data Partition Encryption</term>
1488
1489    <listitem>
1490     <para>
1491      On Linux, encryption can be layered on top of a file system mount
1492      using a <quote>loopback device</quote>. This allows an entire
1493      file system partition be encrypted on disk, and decrypted by the
1494      operating system. On FreeBSD, the equivalent facility is called
1495      GEOM Based Disk Encryption, or <acronym>gbde</acronym>.
1496     </para>
1497
1498     <para>
1499      This mechanism prevents unencrypted data from being read from the
1500      drives if the drives or the entire computer is stolen. This does
1501      not protect against attacks while the file system is mounted,
1502      because when mounted, the operating system provides an unencrypted
1503      view of the data. However, to mount the file system, you need some
1504      way for the encryption key to be passed to the operating system,
1505      and sometimes the key is stored somewhere on the host that mounts
1506      the disk.
1507     </para>
1508    </listitem>
1509   </varlistentry>
1510
1511   <varlistentry>
1512    <term>Encrypting Passwords Across A Network</term>
1513
1514    <listitem>
1515      <para>
1516       The <literal>MD5</> authentication method double-encrypts the
1517       password on the client before sending it to the server. It first
1518       MD5 encrypts it based on the user name, and then encrypts it
1519       based on a random salt sent by the server when the database
1520       connection was made. It is this double-encrypted value that is
1521       sent over the network to the server. Double-encryption not only
1522       prevents the password from being discovered, it also prevents
1523       another connection from using the same encrypted password to
1524       connect to the database server at a later time.
1525      </para>
1526     </listitem>
1527   </varlistentry>
1528
1529   <varlistentry>
1530    <term>Encrypting Data Across A Network</term>
1531
1532     <listitem>
1533      <para>
1534       SSL connections encrypt all data sent across the network: the
1535       password, the queries, and the data returned. The
1536       <filename>pg_hba.conf</> file allows administrators to specify
1537       which hosts can use non-encrypted connections (<literal>host</>)
1538       and which require SSL-encrypted connections
1539       (<literal>hostssl</>). Also, clients can specify that they
1540       connect to servers only via SSL. <application>Stunnel</> or
1541       <application>SSH</> can also be used to encrypt transmissions.
1542      </para>
1543     </listitem>
1544   </varlistentry>
1545
1546   <varlistentry>
1547    <term>SSL Host Authentication</term>
1548
1549    <listitem>
1550     <para>
1551      It is possible for both the client and server to provide SSL
1552      certificates to each other. It takes some extra configuration
1553      on each side, but this provides stronger verification of identity
1554      than the mere use of passwords. It prevents a computer from
1555      pretending to be the server just long enough to read the password
1556      send by the client. It also helps prevent "man in the middle"
1557      attacks where a computer between the client and server pretends to
1558      be the server and reads and passes all data between the client and
1559      server.
1560     </para>
1561    </listitem>
1562   </varlistentry>
1563
1564   <varlistentry>
1565    <term>Client-Side Encryption</term>
1566
1567    <listitem>
1568     <para>
1569      If the system administrator cannot be trusted, it is necessary
1570      for the client to encrypt the data; this way, unencrypted data
1571      never appears on the database server. Data is encrypted on the
1572      client before being sent to the server, and database results have
1573      to be decrypted on the client before being used.
1574     </para>
1575    </listitem>
1576   </varlistentry>
1577
1578   </variablelist>
1579
1580  </sect1>
1581
1582  <sect1 id="ssl-tcp">
1583   <title>Secure TCP/IP Connections with SSL</title>
1584
1585   <indexterm zone="ssl-tcp">
1586    <primary>SSL</primary>
1587   </indexterm>
1588
1589   <para>
1590    <productname>PostgreSQL</> has native support for using
1591    <acronym>SSL</> connections to encrypt client/server communications
1592    for increased security. This requires that
1593    <productname>OpenSSL</productname> is installed on both client and
1594    server systems and that support in <productname>PostgreSQL</> is
1595    enabled at build time (see <xref linkend="installation">).
1596   </para>
1597
1598   <para>
1599    With <acronym>SSL</> support compiled in, the
1600    <productname>PostgreSQL</> server can be started with
1601    <acronym>SSL</> enabled by setting the parameter
1602    <xref linkend="guc-ssl"> to <literal>on</> in
1603    <filename>postgresql.conf</>.  The server will listen for both standard
1604    and <acronym>SSL</> connections on the same TCP port, and will negotiate
1605    with any connecting client on whether to use <acronym>SSL</>.  By
1606    default, this is at the client's option; see <xref
1607    linkend="auth-pg-hba-conf"> about how to set up the server to require
1608    use of <acronym>SSL</> for some or all connections.
1609   </para>
1610
1611   <para>
1612    <productname>PostgreSQL</productname> reads the system-wide
1613    <productname>OpenSSL</productname> configuration file. By default, this
1614    file is named <filename>openssl.cnf</filename> and is located in the
1615    directory reported by <literal>openssl version -d</>.
1616    This default can be overridden by setting environment variable
1617    <envar>OPENSSL_CONF</envar> to the name of the desired configuration file.
1618   </para>
1619
1620   <para>
1621    <productname>OpenSSL</productname> supports a wide range of ciphers
1622    and authentication algorithms, of varying strength.  While a list of
1623    ciphers can be specified in the <productname>OpenSSL</productname>
1624    configuration file, you can specify ciphers specifically for use by
1625    the database server by modifying <xref linkend="guc-ssl-ciphers"> in
1626    <filename>postgresql.conf</>.
1627   </para>
1628
1629   <note>
1630    <para>
1631     It is possible to have authentication without encryption overhead by
1632     using <literal>NULL-SHA</> or <literal>NULL-MD5</> ciphers.  However,
1633     a man-in-the-middle could read and pass communications between client
1634     and server.  Also, encryption overhead is minimal compared to the
1635     overhead of authentication.  For these reasons NULL ciphers are not
1636     recommended.
1637    </para>
1638   </note>
1639
1640   <para>
1641    To start in <acronym>SSL</> mode, the files <filename>server.crt</>
1642    and <filename>server.key</> must exist in the server's data directory.
1643    These files should contain the server certificate and private key,
1644    respectively.
1645    On Unix systems, the permissions on <filename>server.key</filename> must
1646    disallow any access to world or group; achieve this by the command
1647    <command>chmod 0600 server.key</command>.
1648    If the private key is protected with a passphrase, the
1649    server will prompt for the passphrase and will not start until it has
1650    been entered.
1651   </para>
1652
1653   <sect2 id="ssl-client-certificates">
1654    <title>Using client certificates</title>
1655    <para>
1656    To require the client to supply a trusted certificate, place
1657    certificates of the certificate authorities (<acronym>CA</acronym>)
1658    you trust in the file <filename>root.crt</filename> in the data
1659    directory, and set the <literal>clientcert</literal> parameter
1660    to <literal>1</literal> on the appropriate line(s) in pg_hba.conf.
1661    A certificate will then be requested from the client during
1662    SSL connection startup.  (See <xref linkend="libpq-ssl"> for a
1663    description of how to set up certificates on the client.)  The server will
1664    verify that the client's certificate is signed by one of the trusted
1665    certificate authorities.  Certificate Revocation List (CRL) entries
1666    are also checked if the file <filename>root.crl</filename> exists.
1667    <!-- If this URL changes replace it with a URL to www.archive.org. -->
1668    (See <ulink
1669    url="http://h71000.www7.hp.com/DOC/83final/BA554_90007/ch04s02.html"></>
1670    for diagrams showing SSL certificate usage.)
1671   </para>
1672
1673   <para>
1674    The <literal>clientcert</literal> option in <filename>pg_hba.conf</>
1675    is available for all authentication methods, but only for rows
1676    specified as <literal>hostssl</>. Unless specified, the default is
1677    not to verify the client certificate.
1678   </para>
1679
1680   <para>
1681    You can use the authentication method <literal>cert</> to use the
1682    client certificate for authenticating users. See
1683    <xref linkend="auth-cert"> for details.
1684   </para>
1685   </sect2>
1686
1687   <sect2 id="ssl-server-files">
1688    <title>SSL Server File Usage</title>
1689   <para>
1690    The files <filename>server.key</>, <filename>server.crt</>,
1691    <filename>root.crt</filename>, and <filename>root.crl</filename>
1692    are only examined during server start; so you must restart
1693    the server for changes in them to take effect.
1694   </para>
1695
1696   <table id="ssl-file-usage">
1697    <title>SSL Server File Usage</title>
1698    <tgroup cols="3">
1699     <thead>
1700      <row>
1701       <entry>File</entry>
1702       <entry>Contents</entry>
1703       <entry>Effect</entry>
1704      </row>
1705     </thead>
1706
1707     <tbody>
1708
1709      <row>
1710       <entry><filename>server.crt</></entry>
1711       <entry>server certificate</entry>
1712       <entry>requested by client</entry>
1713      </row>
1714
1715      <row>
1716       <entry><filename>server.key</></entry>
1717       <entry>server private key</entry>
1718       <entry>proves server certificate sent by owner; does not indicate
1719       certificate owner is trustworthy</entry>
1720      </row>
1721
1722      <row>
1723       <entry><filename>root.crt</></entry>
1724       <entry>trusted certificate authorities</entry>
1725       <entry>checks that client certificate is
1726       signed by a trusted certificate authority</entry>
1727      </row>
1728
1729      <row>
1730       <entry><filename>root.crl</></entry>
1731       <entry>certificates revoked by certificate authorities</entry>
1732       <entry>client certificate must not be on this list</entry>
1733      </row>
1734
1735     </tbody>
1736    </tgroup>
1737   </table>
1738   </sect2>
1739
1740   <sect2 id="ssl-certificate-creation">
1741    <title>Creating a Self-Signed Certificate</title>
1742
1743    <para>
1744     To create a quick self-signed certificate for the server, use the
1745     following <productname>OpenSSL</productname> command:
1746 <programlisting>
1747 openssl req -new -text -out server.req
1748 </programlisting>
1749     Fill out the information that <application>openssl</> asks for. Make sure
1750     you enter the local host name as <quote>Common Name</>; the challenge
1751     password can be left blank. The program will generate a key that is
1752     passphrase protected; it will not accept a passphrase that is less
1753     than four characters long. To remove the passphrase (as you must if
1754     you want automatic start-up of the server), run the commands:
1755 <programlisting>
1756 openssl rsa -in privkey.pem -out server.key
1757 rm privkey.pem
1758 </programlisting>
1759     Enter the old passphrase to unlock the existing key. Now do:
1760 <programlisting>
1761 openssl req -x509 -in server.req -text -key server.key -out server.crt
1762 </programlisting>
1763     to turn the certificate into a self-signed certificate and to copy
1764     the key and certificate to where the server will look for them.
1765     Finally do:
1766 <programlisting>
1767 chmod og-rwx server.key
1768 </programlisting>
1769     because the server will reject the file if its permissions are more
1770     liberal than this.
1771     For more details on how to create your server private key and
1772     certificate, refer to the <productname>OpenSSL</> documentation.
1773    </para>
1774
1775    <para>
1776     A self-signed certificate can be used for testing, but a certificate
1777     signed by a certificate authority (<acronym>CA</>) (either one of the
1778     global <acronym>CAs</> or a local one) should be used in production
1779     so the client can verify the server's identity. If all the clients
1780     are local to the organization, using a local <acronym>CA</> is
1781     recommended.
1782    </para>
1783
1784   </sect2>
1785
1786  </sect1>
1787
1788  <sect1 id="ssh-tunnels">
1789   <title>Secure TCP/IP Connections with <application>SSH</application> Tunnels</title>
1790
1791   <indexterm zone="ssh-tunnels">
1792    <primary>ssh</primary>
1793   </indexterm>
1794
1795   <para>
1796    One can use <application>SSH</application> to encrypt the network
1797    connection between clients and a
1798    <productname>PostgreSQL</productname> server. Done properly, this
1799    provides an adequately secure network connection, even for non-SSL-capable
1800    clients.
1801   </para>
1802
1803   <para>
1804    First make sure that an <application>SSH</application> server is
1805    running properly on the same machine as the
1806    <productname>PostgreSQL</productname> server and that you can log in using
1807    <command>ssh</command> as some user. Then you can establish a secure
1808    tunnel with a command like this from the client machine:
1809 <programlisting>
1810 ssh -L 63333:localhost:5432 joe@foo.com
1811 </programlisting>
1812    The first number in the <option>-L</option> argument, 63333, is the
1813    port number of your end of the tunnel; it can be chosen freely.
1814    (IANA reserves ports 49152 through 65535 for private use.)  The
1815    second number, 5432, is the remote end of the tunnel: the port
1816    number your server is using. The name or IP address between the
1817    port numbers is the host with the database server you are going to
1818    connect to, as seen from the host you are logging in to, which
1819    is <literal>foo.com</literal> in this example. In order to connect
1820    to the database server using this tunnel, you connect to port 63333
1821    on the local machine:
1822 <programlisting>
1823 psql -h localhost -p 63333 postgres
1824 </programlisting>
1825    To the database server it will then look as though you are really
1826    user <literal>joe</literal> on host <literal>foo.com</literal>
1827    connecting to <literal>localhost</literal> in that context, and it
1828    will use whatever authentication procedure was configured for
1829    connections from this user and host.  Note that the server will not
1830    think the connection is SSL-encrypted, since in fact it is not
1831    encrypted between the
1832    <application>SSH</application> server and the
1833    <productname>PostgreSQL</productname> server.  This should not pose any
1834    extra security risk as long as they are on the same machine.
1835   </para>
1836
1837   <para>
1838    In order for the
1839    tunnel setup to succeed you must be allowed to connect via
1840    <command>ssh</command> as <literal>joe@foo.com</literal>, just
1841    as if you had attempted to use <command>ssh</command> to set up a
1842    terminal session.
1843   </para>
1844
1845   <para>
1846    You could also have set up the port forwarding as
1847 <programlisting>
1848 ssh -L 63333:foo.com:5432 joe@foo.com
1849 </programlisting>
1850    but then the database server will see the connection as coming in
1851    on its <literal>foo.com</literal> interface, which is not opened by
1852    the default setting <literal>listen_addresses =
1853    'localhost'</literal>.  This is usually not what you want.
1854   </para>
1855
1856   <para>
1857    If you have to <quote>hop</quote> to the database server via some
1858    login host, one possible setup could look like this:
1859 <programlisting>
1860 ssh -L 63333:db.foo.com:5432 joe@shell.foo.com
1861 </programlisting>
1862    Note that this way the connection
1863    from <literal>shell.foo.com</literal>
1864    to <literal>db.foo.com</literal> will not be encrypted by the SSH
1865    tunnel.
1866    SSH offers quite a few configuration possibilities when the network
1867    is restricted in various ways.  Please refer to the SSH
1868    documentation for details.
1869   </para>
1870
1871   <tip>
1872    <para>
1873     Several other applications exist that can provide secure tunnels using
1874     a procedure similar in concept to the one just described.
1875    </para>
1876   </tip>
1877
1878  </sect1>
1879
1880 </chapter>