external/ibm-public/postfix/dist/proto/DATABASE_README.html

   1 <!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN"
   2         "http://www.w3.org/TR/html4/loose.dtd">
   3
   4 <html>
   5
   6 <head>
   7
   8 <title>Postfix Lookup Table Overview</title>
   9
  10 <meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
  11
  12 </head>
  13
  14 <body>
  15
  16 <h1><img src="postfix-logo.jpg" width="203" height="98" ALT="">Postfix
  17 Lookup Table Overview</h1>
  18
  19 <hr>
  20
  21 <h2>Overview </h2>
  22
  23 This document covers the following topics:
  24
  25 <ul>
  26
  27 <li><a href="#intro">The Postfix lookup table model</a>
  28
  29 <li><a href="#lists">Postfix lists versus tables </a>
  30
  31 <li><a href="#preparing">Preparing Postfix for LDAP or SQL lookups</a>
  32
  33 <li><a href="#detect">Maintaining Postfix lookup table files</a>
  34
  35 <li><a href="#safe_db">Updating Berkeley DB files safely</a>
  36
  37 <li><a href="#types">Postfix lookup table types</a>
  38
  39 </ul>
  40
  41 <h2><a name="intro">The Postfix lookup table model</a></h2>
  42
  43 <p> Postfix uses lookup tables to store and look up information
  44 for access control, address rewriting and even for content filtering.
  45 All Postfix lookup tables are specified as "type:table", where
  46 "type" is one of the database types described under "<a
  47 href="#types">Postfix lookup table types</a>" at the end of this
  48 document, and where "table" is the lookup table name. The Postfix
  49 documentation uses the terms "database" and "lookup table" for the
  50 same thing.  </p>
  51
  52 <p> Examples of lookup tables that appear often in the Postfix
  53 documentation: </p>
  54
  55 <blockquote>
  56 <pre>
  57 /etc/postfix/main.cf:
  58     alias_maps = hash:/etc/postfix/aliases            (local aliasing)
  59     header_checks = regexp:/etc/postfix/header_checks (content filtering)
  60     transport_maps = hash:/etc/postfix/transport      (routing table)
  61     virtual_alias_maps = hash:/etc/postfix/virtual    (address rewriting)
  62 </pre>
  63 </blockquote>
  64
  65 <p> All Postfix lookup tables store information as (key, value)
  66 pairs.  This interface may seem simplistic at first, but it turns
  67 out to be very powerful. The (key, value) query interface completely
  68 hides the complexities of LDAP or SQL from Postfix. This is a good
  69 example of connecting complex systems with simple interfaces.  </p>
  70
  71 <p> Benefits of the Postfix (key, value) query interface:</p>
  72
  73 <ul>
  74
  75 <li>  You can implement Postfix lookup tables first with local
  76 Berkeley DB files and then switch to LDAP or MySQL without any
  77 impact on the Postfix configuration itself, as described under "<a
  78 href="#preparing">Preparing Postfix for LDAP or SQL lookups</a>"
  79 below.
  80
  81 <li> You can use Berkeley DB files with fixed lookup strings for
  82 simple address rewriting operations and you can use regular expression
  83 tables for the more complicated work. In other words, you don't
  84 have to put everything into the same table.
  85
  86 </ul>
  87
  88 <h2><a name="lists">Postfix lists versus tables </a></h2>
  89
  90 <p> Most Postfix lookup tables are used to look up information.
  91 Examples are address rewriting (the lookup string is the old address,
  92 and the result is the new address) or access control (the lookup
  93 string is the client, sender or recipient, and the result is an
  94 action such as "reject"). </p>
  95
  96 <p> With some tables, however, Postfix needs to know only if the
  97 lookup key exists.  The lookup result itself is not used. Examples
  98 are the local_recipient_maps that determine what local recipients
  99 Postfix accepts in mail from the network, the mydestination parameter
 100 that specifies what domains Postfix delivers locally, or the
 101 mynetworks parameter that specifies the IP addresses of trusted
 102 clients or client networks.  Technically, these are lists, not
 103 tables. Despite the difference, Postfix lists are described here
 104 because they use the same underlying infrastructure as Postfix
 105 lookup tables.  </p>
 106
 107 <h2><a name="preparing">Preparing Postfix for LDAP or SQL lookups</a>
 108 </h2>
 109
 110 <p> LDAP and SQL are complex systems. Trying to set up both Postfix
 111 and LDAP or SQL at the same time is definitely not a good idea.
 112 You can save yourself a lot of time by implementing Postfix first
 113 with local files such as Berkeley DB.  Local files have few surprises,
 114 and are easy to debug with the postmap(1) command:  </p>
 115
 116 <blockquote>
 117 <pre>
 118 % <b>postmap -q info@example.com hash:/etc/postfix/virtual </b>
 119 </pre>
 120 </blockquote>
 121
 122 <p> Once you have local files working properly you can follow the
 123 instructions in ldap_table(5), mysql_table(5) or pgsql_table(5)
 124 and replace local file lookups with LDAP or SQL lookups.  When you
 125 do this, you should use the postmap(1) command again, to verify
 126 that database lookups still produce the exact same results as local
 127 file lookup:  </p>
 128
 129 <blockquote>
 130 <pre>
 131 % <b>postmap -q info@example.com ldap:/etc/postfix/virtual.cf </b>
 132 </pre>
 133 </blockquote>
 134
 135 <p> Be sure to exercise all the partial address or parent domain
 136 queries that are documented under "table search order" in the
 137 relevant manual page:  access(5), canonical(5), virtual(5),
 138 transport(5), or under the relevant configuration parameter:
 139 mynetworks, relay_domains, parent_domain_matches_subdomains. </p>
 140
 141 <h2><a name="detect">Maintaining Postfix lookup table files</a></h2>
 142
 143 <p> When you make changes to a database while the mail system is
 144 running, it would be desirable if Postfix avoids reading information
 145 while that information is being changed. It would also be nice if
 146 you can change a database without having to execute "postfix reload",
 147 in order to force Postfix to use the new information. Each time
 148 you do "postfix reload" Postfix loses a lot of performance.
 149 </p>
 150
 151 <ul>
 152
 153 <li> <p> If you change a network database such as LDAP, NIS or
 154 SQL, there is no need to execute "postfix reload".  The LDAP, NIS
 155 or SQL server takes care of read/write access conflicts and gives
 156 the new data to Postfix once that data is available.  </p>
 157
 158 <li> <p> If you change a regexp: or pcre: file then Postfix may or
 159 may not pick up the file changes immediately. This is because a
 160 Postfix process reads the entire file into memory once and never
 161 examines the file again.  </p>
 162
 163 <ul>
 164
 165 <li> <p> If the file is used by a short-running process such as
 166 smtpd(8), cleanup(8) or local(8), there is no need to execute
 167 "postfix reload" after making a change. </p>
 168
 169 <li> <p> If the file is being used by a long-running process such
 170 as trivial-rewrite(8) on a busy server it may be necessary to
 171 execute "postfix reload". </p>
 172
 173 </ul>
 174
 175 <li> <p> If you change a local file based database such as DBM or
 176 Berkeley DB, there is no need to execute "postfix reload".  Postfix
 177 uses file locking to avoid read/write access conflicts, and whenever
 178 a Postfix daemon process notices that a file has changed it will
 179 terminate before handling the next client request, so that a new
 180 process can initialize with the new database.  </p>
 181
 182 </ul>
 183
 184 <h2><a name="safe_db">Updating Berkeley DB files safely</a></h2>
 185
 186 <p> Although Postfix uses file locking to avoid access conflicts
 187 while updating Berkeley DB or other local database files, you still
 188 have a problem when the update fails because the disk is full or
 189 because something else happens.  This is because commands such as
 190 postmap(1) or postalias(1) overwrite existing files. If the update
 191 fails in the middle then you have no usable database, and Postfix
 192 will stop working. This is not an issue with the CDB database type
 193 available with Postfix 2.2 and later: <a href="CDB_README.html">CDB</a>
 194 creates a new file, and renames the file upon successful completion.
 195 </p>
 196
 197 <p> With multi-file databases such as DBM, there is no simple
 198 solution. With Berkeley DB and other "one file" databases, it is
 199 possible to add some extra robustness by using "mv" to REPLACE an
 200 existing database file instead of overwriting it:  </p>
 201
 202 <blockquote>
 203 <pre>
 204 # <b>postmap access.in &amp;&amp; mv access.in.db access.db</b>
 205 </pre>
 206 </blockquote>
 207
 208 <p> This converts the input file "access.in" into the output file
 209 "access.in.db", and replaces the file "access.db" only when the
 210 postmap(1) command was successful. Of course typing such commands
 211 becomes boring quickly, and this is why people use "make" instead,
 212 as shown below.  User input is shown in bold font. </p>
 213
 214 <blockquote>
 215 <pre>
 216 # <b>cat Makefile</b>
 217 all: aliases.db access.db virtual.db ...etcetera...
 218
 219 # Note 1: commands are specified after a TAB character.
 220 # Note 2: use postalias(1) for local aliases, postmap(1) for the rest.
 221 aliases.db: aliases.in
 222         postalias aliases.in
 223         mv aliases.in.db aliases.db
 224
 225 access.db: access.in
 226         postmap access.in
 227         mv access.in.db access.db
 228
 229 virtual.db: virtual.in
 230         postmap virtual.in
 231         mv virtual.in.db virtual.db
 232
 233 ...etcetera...
 234 # <b>vi access.in</b>
 235 ...editing session not shown...
 236 # <b>make</b>
 237 postmap access.in
 238 mv access.in.db access.db
 239 #
 240 </pre>
 241 </blockquote>
 242
 243 <p> The "make" command updates only the files that have changed.
 244 In case of error, the "make" command will stop and will not invoke
 245 the "mv" command, so that Postfix will keep using the existing
 246 database file as if nothing happened. </p>
 247
 248 <h2><a name="types">Postfix lookup table types</a> </h2>
 249
 250 <p> To find out what database types your Postfix system supports,
 251 use the "<b>postconf -m</b>" command.  Here is a list of database types
 252 that are often supported: </p>
 253
 254 <blockquote>
 255
 256 <dl>
 257
 258 <dt> <b>btree</b> </dt>
 259
 260 <dd> A sorted, balanced tree structure.  This is available only on
 261 systems with support for Berkeley DB databases. Database files are
 262 created with the postmap(1) or postalias(1) command. The lookup
 263 table name as used in "btree:table" is the database file name
 264 without the ".db" suffix.  </dd>
 265
 266 <dt> <b>cdb</b> </dt>
 267
 268 <dd> A read-optimized structure with no support for incremental updates.
 269 Database files are created with the postmap(1) or postalias(1) command.
 270 The lookup table name as used in "cdb:table" is the database file name
 271 without the ".cdb" suffix.  This feature is available with Postfix 2.2
 272 and later. </dd>
 273
 274 <dt> <b>cidr</b> </dt>
 275
 276 <dd> A table that associates values with Classless Inter-Domain
 277 Routing (CIDR) patterns. The table format is described in cidr_table(5).
 278 </dd>
 279
 280 <dt> <b>dbm</b> </dt>
 281
 282 <dd> An indexed file type based on hashing.  This is available only
 283 on systems with support for DBM databases. Database files are
 284 created with the postmap(1) or postalias(1) command. The lookup
 285 table name as used in "dbm:table" is the database file name without
 286 the ".dir" or ".pag" suffix.  </dd>
 287
 288 <dt> <b>environ</b> </dt>
 289
 290 <dd> The UNIX process environment array. The lookup key is the
 291 variable name. The lookup table name in "environ:table" is ignored.
 292 </dd>
 293
 294 <dt> <b>hash</b> </dt>
 295
 296 <dd> An indexed file type based on hashing.  This is available only
 297 on systems with support for Berkeley DB databases. Database files are
 298 created with the postmap(1) or postalias(1) command. The database
 299 name as used in "hash:table" is the database file name without the
 300 ".db" suffix. </dd>
 301
 302 <dt> <b>ldap</b> (read-only) </dt>
 303
 304 <dd> Perform lookups using the LDAP protocol. Configuration details
 305 are given in the ldap_table(5). </dd>
 306
 307 <dt> <b>mysql</b> (read-only) </dt>
 308
 309 <dd> Perform MySQL database lookups. Configuration details are given
 310 in mysql_table(5). </dd>
 311
 312 <dt> <b>netinfo</b> (read-only) </dt>
 313
 314 <dd> Perform Netinfo database lookups. </dd>
 315
 316 <dt> <b>nis</b> (read-only) </dt>
 317
 318 <dd> Perform NIS database lookups. </dd>
 319
 320 <dt> <b>nisplus</b> (read-only) </dt>
 321
 322 <dd> Perform NIS+ database lookups. Configuration details are given
 323 in nisplus_table(5). </dd>
 324
 325 <dt> <b>pcre</b> (read-only) </dt>
 326
 327 <dd> A lookup table based on Perl Compatible Regular Expressions.
 328 The file format is described in pcre_table(5). The lookup table
 329 name as used in "pcre:table" is the name of the regular expression
 330 file.  </dd>
 331
 332 <dt> <b>pgsql</b> (read-only) </dt>
 333
 334 <dd> Perform PostgreSQL database lookups.  Configuration details
 335 are given in pgsql_table(5). </dd>
 336
 337 <dt> <b>proxy</b> (read-only) </dt>
 338
 339 <dd> Access information via the Postfix proxymap(8) service. The
 340 lookup table name syntax is "proxy:type:table". </dd>
 341
 342 <dt> <b>regexp</b> (read-only) </dt>
 343
 344 <dd> A lookup table based on regular expressions. The file format
 345 is described in regexp_table(5). The lookup table name as used in
 346 "regexp:table" is the name of the regular expression file. </dd>
 347
 348 <dt> <b>sdbm</b> </dt>
 349
 350 <dd> An indexed file type based on hashing.  This is available only
 351 on systems with support for SDBM databases. Database files are
 352 created with the postmap(1) or postalias(1) command. The lookup
 353 table name as used in "sdbm:table" is the database file name without
 354 the ".dir" or ".pag" suffix.  </dd>
 355
 356 <dt> <b>static</b> (read-only) </dt>
 357
 358 <dd> Always returns its lookup table name as lookup result.  For
 359 example, the lookup table "static:foobar" always returns the string
 360 "foobar" as lookup result. </dd>
 361
 362 <dt> <b>tcp</b> </dt>
 363
 364 <dd> Access information through a TCP/IP server. The protocol is
 365 described in tcp_table(5). The lookup table name is "tcp:host:port"
 366 where "host" specifies a symbolic hostname or a numeric IP address,
 367 and "port" specifies a symbolic service name or a numeric port
 368 number. This protocol is not available in the stable Postfix release.
 369 </dd>
 370
 371 <dt> <b>unix</b> (read-only) </dt>
 372
 373 <dd> A limited way to query the UNIX authentication database. The
 374 following tables are implemented:
 375
 376 <dl>
 377
 378 <dt> <b>unix:passwd.byname</b> </dt>
 379
 380 <dd>The table is the UNIX password database. The key is a login
 381 name.  The result is a password file entry in passwd(5) format.
 382 </dd>
 383
 384 <dt> <b>unix:group.byname</b> </dt>
 385
 386 <dd> The table is the UNIX group database. The key is a group name.
 387 The result is a group file entry in group(5) format. </dd>
 388
 389 </dl> </dd>
 390
 391 </dl>
 392
 393 </blockquote>
 394
 395 <p> Other lookup table types may be available depending on how
 396 Postfix was built. With some Postfix distributions the list is
 397 dynamically extensible as support for lookup tables is dynamically
 398 linked into Postfix.  </p>
 399
 400 </body>
 401
 402 </html>