postfix-master/FILTER_README.html

   1 <!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN"
   2         "http://www.w3.org/TR/html4/loose.dtd">
   3
   4 <html>
   5
   6 <head>
   7
   8 <title>Postfix After-Queue Content Filter </title>
   9
  10 <meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
  11
  12 </head>
  13
  14 <body>
  15
  16 <h1><img src="postfix-logo.jpg" width="203" height="98" ALT="">Postfix After-Queue Content Filter </h1>
  17
  18 <hr>
  19
  20 <h2>Introduction</h2>
  21
  22 <p> This document requires Postfix version 2.1 or later. </p>
  23
  24 <p> Normally, Postfix receives mail, stores it in the mail queue
  25 and then delivers it. With the external content filter described
  26 here, mail is filtered AFTER it is queued. This approach decouples
  27 mail receiving processes from mail filtering processes, and gives
  28 you maximal control over how many filtering processes you are
  29 willing to run in parallel.  </p>
  30
  31 <p> The after-queue content filter is meant to be used as follows: </p>
  32
  33 <blockquote>
  34
  35 <table>
  36
  37 <tr>
  38
  39         <td bgcolor="#f0f0ff" align="center" valign="middle">
  40         Network or<br> local users </td>
  41
  42     <td align="center" valign="middle"> <tt> -&gt; </tt> </td>
  43
  44         <td bgcolor="#f0f0ff" align="center" valign="middle">
  45         Postfix<br> queue </td>
  46
  47     <td align="center" valign="middle"> <tt> -&gt; </tt> </td>
  48
  49         <td bgcolor="#f0f0ff" align="center" valign="middle">
  50         <b>Content<br> filter</b> </td>
  51
  52     <td align="center" valign="middle"> <tt> -&gt; </tt> </td>
  53
  54         <td bgcolor="#f0f0ff" align="center" valign="middle">
  55         Postfix<br> queue </td>
  56
  57     <td align="center" valign="middle"> <tt> -&gt; </tt> </td>
  58
  59         <td bgcolor="#f0f0ff" align="center" valign="middle">
  60         Network or<br> local mailbox </td>
  61
  62 </tr>
  63
  64 </table>
  65
  66 </blockquote>
  67
  68 <p> This document describes implementations that use a single
  69 Postfix instance for everything: receiving, filtering and delivering
  70 mail.  Applications that use two separate Postfix instances will
  71 be covered by a later version of this document.  </p>
  72
  73 <p> The after-queue content filter is not to be confused with the
  74 approaches described in the <a href="SMTPD_PROXY_README.html">SMTPD_PROXY_README</a> or <a href="MILTER_README.html">MILTER_README</a>
  75 documents,
  76 where incoming SMTP mail is filtered BEFORE it is stored into the
  77 Postfix queue.  </p>
  78
  79 <p> This document describes two approaches to content filter
  80 all email, as well as several options to filter mail selectively: </p>
  81
  82 <ul>
  83
  84 <li><a href="#principles">Principles of operation</a>
  85
  86 <li>Simple content filter
  87
  88 <ul>
  89
  90 <li><a href="#simple_filter">Simple content filter example</a>
  91
  92 <li><a href="#simple_performance">Simple content filter performance</a>
  93
  94 <li><a href="#simple_limitations">Simple content filter limitations</a>
  95
  96 <li><a href="#simple_turnoff">Turning off the simple content filter</a>
  97
  98 </ul>
  99
 100 <li>Advanced content filter
 101
 102 <ul>
 103
 104 <li><a href="#advanced_filter">Advanced content filter example</a>
 105
 106 <li><a href="#performance">Advanced content filter performance</a>
 107
 108 <li><a href="#advanced_turnoff">Turning off the advanced content filter</a>
 109
 110 </ul>
 111
 112 <li>Selective content filtering
 113
 114 <ul>
 115
 116 <li><a href="#remote_only">Filtering mail from outside users only</a>
 117
 118 <li><a href="#domain_dependent">Different filters for different domains</a>
 119
 120 <li><a href="#dynamic_filter">FILTER actions in access or header/body tables</a>
 121
 122 </ul>
 123
 124 </ul>
 125
 126
 127 <h2><a name="principles">Principles of operation</a> </h2>
 128
 129 <p> An after-queue content filter receives unfiltered mail from Postfix
 130 (as described further below) and can do one of the following: </p>
 131
 132 <ol>
 133
 134 <li> <p> Re-inject the mail back into Postfix, perhaps after changing
 135     content and/or destination. </p>
 136
 137 <li> <p> Discard or quarantine the mail. </p>
 138
 139 <li> <p> Reject the mail (by sending a suitable status code back to
 140     Postfix). Postfix will send the mail back to the sender address. </p>
 141
 142 </ol>
 143
 144 <p> NOTE: in this time of mail worms and forged spam, it is a VERY
 145 BAD IDEA to send viruses back to the sender address, because the
 146 sender address is almost certainly not the originator. It is better
 147 to discard known viruses, and to quarantine material that is
 148 suspect so that a human can decide what to do with it. </p>
 149
 150 <h2><a name="simple_filter">Simple content filter example</a></h2>
 151
 152 <p> The first example is simple to set up, but has major limitations
 153 that will be addressed in a second example.  Postfix receives
 154 unfiltered mail from the network with the <a href="smtpd.8.html">smtpd(8)</a> server, and
 155 delivers unfiltered mail to a content filter with the Postfix
 156 <a href="pipe.8.html">pipe(8)</a> delivery agent.  The content filter injects filtered mail
 157 back into Postfix with the Postfix <a href="sendmail.1.html">sendmail(1)</a> command, so that
 158 Postfix can deliver it to the final destination. </p>
 159
 160 <p> This means that mail submitted via the Postfix <a href="sendmail.1.html">sendmail(1)</a>
 161 command cannot be content filtered. </p>
 162
 163 <p> In the figure below, names followed by a number represent
 164 Postfix commands or daemon programs. See the <a href="OVERVIEW.html">OVERVIEW</a>
 165 document for an introduction to the Postfix architecture. </p>
 166
 167 <blockquote>
 168
 169 <table>
 170
 171 <tr>
 172
 173         <td align="center" valign="top"> Unfiltered<br> <br> </td>
 174
 175     <td align="center" valign="top"> <tt> -&gt;</tt><br> <br> </td>
 176
 177        <td bgcolor="#f0f0ff" align="center" valign="middle">
 178        <a href="smtpd.8.html">smtpd(8)</a><br> <br> <a href="pickup.8.html">pickup(8)</a> </td>
 179
 180    <td align="center" valign="middle"> <tt> &gt;- </tt> </td>
 181
 182        <td bgcolor="#f0f0ff" align="center" valign="middle">
 183        <a href="cleanup.8.html">cleanup(8)</a> </td>
 184
 185    <td align="center" valign="middle"> <tt> -&gt; </tt> </td>
 186
 187        <td bgcolor="#f0f0ff" align="center" valign="middle">
 188        <a href="qmgr.8.html">qmgr(8)</a><br> Postfix <br> queue </td>
 189
 190    <td align="center" valign="middle"> <tt> -&lt; </tt> </td>
 191
 192        <td bgcolor="#f0f0ff" align="center" valign="middle">
 193        <a href="local.8.html">local(8)</a><br> <a href="smtp.8.html">smtp(8)</a><br> <a href="pipe.8.html">pipe(8)</a> </td>
 194
 195    <td align="center" valign="top"> <tt> -&gt;</tt><br> <tt>
 196    -&gt;</tt><br> </td>
 197
 198        <td align="center" valign="top"> Filtered<br> Filtered<br>
 199        </td>
 200
 201 </tr>
 202
 203 <tr>
 204
 205    <td colspan="2"> </td>
 206
 207        <td align="center" valign="middle"> ^<br> <tt> | </tt> </td>
 208
 209    <td colspan="5"> </td>
 210
 211        <td align="center" valign="middle"> <tt> |<br> v </tt> </td>
 212
 213    <td colspan="2"> </td>
 214
 215 </tr>
 216
 217 <tr>
 218
 219    <td colspan="2"> </td>
 220
 221         <td bgcolor="#f0f0ff" align="center" valign="middle">
 222         <a href="QSHAPE_README.html#maildrop_queue"> maildrop <br>
 223         queue </a> </td>
 224
 225    <td align="center" valign="middle"> <tt> &lt;- </tt> </td>
 226
 227         <td bgcolor="#f0f0ff" align="center" valign="middle">Postfix<br>
 228         <a href="postdrop.1.html">postdrop(1)</a> </td>
 229
 230    <td align="center" valign="middle"> <tt> &lt;- </tt> </td>
 231
 232         <td bgcolor="#f0f0ff" align="center" valign="middle">Postfix<br>
 233         <a href="sendmail.1.html">sendmail(1)</a> </td>
 234
 235    <td align="center" valign="middle"> <tt> &lt;- </tt> </td>
 236
 237         <td bgcolor="#f0f0ff" align="center" valign="middle">Content
 238         <br> filter </td>
 239
 240    <td colspan="2"> </td>
 241
 242 </tr>
 243
 244 </table>
 245
 246 </blockquote>
 247
 248 <p> The content filter can be a simple shell script like this: </p>
 249
 250 <blockquote>
 251 <pre>
 252  1 #!/bin/sh
 253  2
 254  3 # Simple shell-based filter. It is meant to be invoked as follows:
 255  4 #       /path/to/script -f sender recipients...
 256  5
 257  6 # Localize these. The -G option does nothing before Postfix 2.3.
 258  7 INSPECT_DIR=/var/spool/filter
 259  8 SENDMAIL="/usr/sbin/sendmail -G -i" # NEVER NEVER NEVER use "-t" here.
 260  9
 261 10 # Exit codes from &lt;sysexits.h&gt;
 262 11 EX_TEMPFAIL=75
 263 12 EX_UNAVAILABLE=69
 264 13
 265 14 # Clean up when done or when aborting.
 266 15 trap "rm -f in.$$" 0 1 2 3 15
 267 16
 268 17 # Start processing.
 269 18 cd $INSPECT_DIR || {
 270 19     echo $INSPECT_DIR does not exist; exit $EX_TEMPFAIL; }
 271 20
 272 21 cat &gt;in.$$ || {
 273 22     echo Cannot save mail to file; exit $EX_TEMPFAIL; }
 274 23
 275 24 # Specify your content filter here.
 276 25 # filter &lt;in.$$ || {
 277 26 #   echo Message content rejected; exit $EX_UNAVAILABLE; }
 278 27
 279 28 $SENDMAIL "$@" &lt;in.$$
 280 29
 281 30 exit $?
 282 </pre>
 283 </blockquote>
 284
 285 <p> Notes: </p>
 286
 287 <ul>
 288
 289 <li> <p> Line 8: The -G option says the filter output is not a local
 290 mail submission: don't do silly things like appending the local
 291 domain name to addresses in message headers.  This option does
 292 nothing before Postfix version 2.3.  </p>
 293
 294 <li> <p> Line 8: The -i option says don't stop reading input when
 295 a line contains "." only.  </p>
 296
 297 <li> <p> Line 8: NEVER NEVER NEVER use the "-t" command-line option
 298 here. It will mis-deliver mail, like sending messages from a mailing
 299 list back to the mailing list. </p>
 300
 301 <li> <p> Line 21: The idea is to first capture the message to
 302 file and then run the content through a third-party content filter
 303 program. </p>
 304
 305 <li> <p> Line 22: If the message cannot be captured to file, mail
 306 delivery is deferred by terminating with exit status 75 (EX_TEMPFAIL).
 307 Postfix places the message in the deferred mail queue and tries
 308 again later.  </p>
 309
 310 <li> <p> Line 25: You will need to specify a real content filter
 311 program here that receives the content on standard input. </p>
 312
 313 <li> <p> Line 26: If the content filter program finds a problem,
 314 the mail is bounced by terminating with exit status 69 (EX_UNAVAILABLE).
 315 Postfix will send the message back to the sender as undeliverable
 316 mail.
 317 </p>
 318
 319 <li> <p> NOTE: in this time of mail worms and spam, it is a BAD
 320 IDEA to send known viruses or spam back to the sender, because that
 321 address is likely to be forged. It is safer to discard known viruses
 322 and to quarantine suspicious content so that it can
 323 be inspected by a human being. </p>
 324
 325 <li> <p> Line 28: If the content is OK, it is given as input to
 326 the Postfix sendmail command, and the exit status of the filter
 327 command is whatever exit status the Postfix sendmail command
 328 produces. Postfix will deliver the message as usual. </p>
 329
 330 <li> <p> Line 30: Postfix returns the exit status of the Postfix
 331 sendmail command. </p>
 332
 333 </ul>
 334
 335 <p> I suggest that you first run this script by hand until you are
 336 satisfied with the results. Run it with a real message (headers+body)
 337 as input: </p>
 338
 339 <blockquote>
 340 <pre>
 341 % /path/to/script -f sender -- recipient... &lt;message-file
 342 </pre>
 343 </blockquote>
 344
 345 <p> Once you're satisfied with the content filtering script: </p>
 346
 347 <ul>
 348
 349 <li> <p> Create a dedicated local user account called "filter".  This
 350 user handles all potentially dangerous mail content - that is
 351 why it should be a separate account. Do not use "nobody", and
 352 most certainly do not use "root" or "postfix". </p>
 353
 354 <li> <p> Create a directory /var/spool/filter that is accessible only
 355 to the "filter" user. This is where the content filtering script
 356 is supposed to store its temporary files. </p>
 357
 358 <li> <p> Configure Postfix to deliver mail to the content filter
 359 with the <a href="pipe.8.html">pipe(8)</a> delivery agent (see the <a href="pipe.8.html">pipe(8)</a> manpage for a
 360 description of the command syntax below). </p>
 361
 362 <pre>
 363 /etc/postfix/<a href="master.5.html">master.cf</a>:
 364   # =============================================================
 365   # service type  private unpriv  chroot  wakeup  maxproc command
 366   #               (yes)   (yes)   (yes)   (never) (100)
 367   # =============================================================
 368   filter    unix  -       n       n       -       10      pipe
 369     flags=Rq user=filter null_sender=
 370     argv=/path/to/script -f ${sender} -- ${recipient}
 371 </pre>
 372
 373 <p> This runs up to 10 content filters in parallel. Instead of a
 374 limit of 10 concurrent processes, use whatever process limit is
 375 feasible for your machine.  Content inspection software can gobble
 376 up a lot of system resources, so you don't want to have too much
 377 of it running at the same time. The empty null_sender setting is
 378 required with Postfix 2.3 and later. </p>
 379
 380 <li> <p> To turn on content filtering for mail arriving via SMTP
 381 only, append "-o <a href="postconf.5.html#content_filter">content_filter</a>=filter:dummy" to the <a href="master.5.html">master.cf</a>
 382 entry that defines the Postfix SMTP server: </p>
 383
 384 <pre>
 385 /etc/postfix/<a href="master.5.html">master.cf</a>:
 386   # =============================================================
 387   # service type  private unpriv  chroot  wakeup  maxproc command
 388   #               (yes)   (yes)   (yes)   (never) (100)
 389   # =============================================================
 390   smtp      inet  ...other stuff here, do not change...   smtpd
 391         -o <a href="postconf.5.html#content_filter">content_filter</a>=filter:dummy
 392 </pre>
 393
 394 <p> The "-o <a href="postconf.5.html#content_filter">content_filter</a>" line causes Postfix to add one content
 395 filter request record to each incoming mail message, with content
 396 "filter:dummy". This record overrides the normal mail routing
 397 and causes mail to be given to the content filter instead. </p>
 398
 399 <p> The <a href="postconf.5.html#content_filter">content_filter</a> configuration parameter expects a value of
 400 the form <i>transport:destination</i>. The <i>transport</i> name
 401 specifies the first field of a mail delivery agent definition in
 402 <a href="master.5.html">master.cf</a>; the syntax of the next-hop <i>destination</i> is described
 403 in the manual page of the corresponding delivery agent. </p>
 404
 405 <p> The meaning of an empty next-hop filter <i>destination</i> is
 406 version dependent.  Postfix 2.7 and later will use the recipient
 407 domain; earlier versions will use $<a href="postconf.5.html#myhostname">myhostname</a>.  Specify
 408 "<a href="postconf.5.html#default_filter_nexthop">default_filter_nexthop</a> = $<a href="postconf.5.html#myhostname">myhostname</a>" for compatibility with Postfix
 409 2.6 or earlier, or specify a non-empty next-hop filter <i>destination</i>.
 410 </p>
 411
 412 <p> The <a href="postconf.5.html#content_filter">content_filter</a> setting has lower precedence than a FILTER
 413 action that is specified in an <a href="access.5.html">access(5)</a>, <a href="header_checks.5.html">header_checks(5)</a> or
 414 <a href="header_checks.5.html">body_checks(5)</a> table. </p>
 415
 416 <li> <p> Execute "<b>postfix reload</b>" to complete the change.
 417 </p>
 418
 419 </ul>
 420
 421 <h2> <a name="simple_performance">Simple content filter performance</a> </h2>
 422
 423 <p> With the shell script as shown above you will lose a factor of
 424 four in Postfix performance for transit mail that arrives and leaves
 425 via SMTP. You will lose another factor in transit performance for
 426 each additional temporary file that is created and deleted in the
 427 process of content filtering.  The performance impact is less for
 428 mail that is submitted or delivered locally, because such deliveries
 429 are already slower than SMTP transit mail. </p>
 430
 431 <h2><a name="simple_limitations">Simple content filter limitations</a></h2>
 432
 433 <p> The problem with content filters like the one above is that
 434 they are not very robust. The reason is that the software does not
 435 talk a well-defined protocol with Postfix. If the filter shell
 436 script aborts because the shell runs into some memory allocation
 437 problem, the script will not produce a nice exit status as defined
 438 in the file /usr/include/sysexits.h.  Instead of going to the
 439 <a href="QSHAPE_README.html#deferred_queue">deferred queue</a>, mail will bounce.  The same lack of robustness can
 440 happen when the content filtering software itself runs into a
 441 resource problem. </p>
 442
 443 <p> The simple content filter method is not suitable for content
 444 filter actions that are invoked via <a href="postconf.5.html#header_checks">header_checks</a> or <a href="postconf.5.html#body_checks">body_checks</a>
 445 patterns.  These patterns will be applied again after mail is
 446 re-injected with the Postfix sendmail command, resulting in a mail
 447 filtering loop.  The advanced content filtering method (see below)
 448 makes it possible to turn off <a href="postconf.5.html#header_checks">header_checks</a> or <a href="postconf.5.html#body_checks">body_checks</a> patterns
 449 for filtered mail. </p>
 450
 451 <h2><a name="simple_turnoff">Turning off the simple content filter</a> </h2>
 452
 453 <p> To turn off "simple" content filtering: </p>
 454
 455 <ul> <li> <p> Edit the <a href="master.5.html">master.cf</a> file, remove the "-o
 456 <a href="postconf.5.html#content_filter">content_filter</a>=filter:dummy" text from the entry that defines the
 457 Postfix SMTP server. </p>
 458
 459 <li> <p> Execute "<b>postsuper -r ALL</b>" to remove content
 460 filter request records from existing queue files. </p>
 461
 462 <li> <p> Execute another "<b>postfix reload</b>". </p>
 463
 464 </ul>
 465
 466 <h2><a name="advanced_filter">Advanced content filter example</a></h2>
 467
 468 <p> The second example is more complex, but can give better
 469 performance, and is less likely to bounce mail when the machine
 470 runs into some resource problem.  This content filter receives
 471 unfiltered mail with SMTP on localhost port 10025, and sends filtered
 472 mail back into Postfix with SMTP on localhost port 10026. </p>
 473
 474 <p> For non-SMTP capable content filtering software, Bennett Todd's
 475 SMTP proxy implements a nice PERL/SMTP content filtering framework.
 476 See: <a href="http://bent.latency.net/smtpprox/">http://bent.latency.net/smtpprox/</a>. </p>
 477
 478 <p> In the figure below, names followed by a number represent
 479 Postfix commands or daemon programs. See the <a href="OVERVIEW.html">OVERVIEW</a>
 480 document for an introduction to the Postfix architecture. </p>
 481
 482 <blockquote>
 483
 484 <table>
 485
 486 <tr>
 487
 488        <td align="center" valign="middle"> Unfiltered<br> <br>
 489        Unfiltered</td>
 490
 491     <td align="center" valign="middle"> <tt> -&gt;</tt><br> <br>
 492     <tt> -&gt; </tt> </td>
 493
 494        <td bgcolor="#f0f0ff" align="center" valign="middle">
 495        <a href="smtpd.8.html">smtpd(8)</a><br> <br> <a href="pickup.8.html">pickup(8)</a> </td>
 496
 497    <td align="center" valign="middle"> <tt> &gt;- </tt> </td>
 498
 499        <td bgcolor="#f0f0ff" align="center" valign="middle">
 500        <a href="cleanup.8.html">cleanup(8)</a> </td>
 501
 502    <td align="center" valign="middle"> <tt> -&gt; </tt> </td>
 503
 504        <td bgcolor="#f0f0ff" align="center" valign="middle">
 505        <a href="qmgr.8.html">qmgr(8)</a><br> Postfix <br> queue </td>
 506
 507    <td align="center" valign="middle"> <tt> -&lt; </tt> </td>
 508
 509        <td bgcolor="#f0f0ff" align="center" valign="middle">
 510        <a href="smtp.8.html">smtp(8)</a><br> <br> <a href="local.8.html">local(8)</a> </td>
 511
 512    <td align="center" valign="middle"> <tt> -&gt;</tt><br> <br>
 513    <tt> -&gt; </tt></td>
 514
 515        <td align="center" valign="middle"> Filtered<br> <br>
 516        Filtered</td>
 517
 518 </tr>
 519
 520 <tr>
 521
 522    <td colspan="4"> </td>
 523
 524        <td align="center" valign="middle"> ^<br> <tt> | </tt> </td>
 525
 526    <td> </td>
 527
 528        <td align="center" valign="middle"> <tt> |<br> v </tt> </td>
 529
 530    <td colspan="4"> </td>
 531
 532 </tr>
 533
 534 <tr>
 535
 536    <td colspan="4"> </td>
 537
 538        <td bgcolor="#f0f0ff" align="center" valign="middle">
 539        <a href="smtpd.8.html">smtpd(8)</a><br> 10026 </td>
 540
 541    <td> </td>
 542
 543        <td bgcolor="#f0f0ff" align="center" valign="middle">
 544        <a href="smtp.8.html">smtp(8)</a><br> </td>
 545
 546    <td colspan="4"> </td>
 547
 548 </tr>
 549
 550 <tr>
 551
 552    <td colspan="4"> </td>
 553
 554        <td align="center" valign="middle"> ^<br> <tt> | </tt> </td>
 555
 556    <td> </td>
 557
 558        <td align="center" valign="middle"> <tt> |<br> v </tt> </td>
 559
 560    <td colspan="4"> </td>
 561
 562 </tr>
 563
 564 <tr>
 565
 566    <td colspan="4"> </td>
 567
 568         <td colspan="3" bgcolor="#f0f0ff" align="center"
 569         valign="middle">content filter 10025</td>
 570
 571    <td colspan="4"> </td>
 572
 573 </tr>
 574
 575 </table>
 576
 577 </blockquote>
 578
 579 <p> The example given here filters all mail, including mail that
 580 arrives via SMTP and mail that is locally submitted via the Postfix
 581 sendmail command (local submissions enter Postfix via the <a href="pickup.8.html">pickup(8)</a>
 582 server; to keep the figure simple we omit local submission details).
 583 See examples near the end of this document for
 584 how to exclude local users from filtering, or how to configure a
 585 destination dependent content filter. </p>
 586
 587 <p> You can expect to lose about a factor of two in Postfix
 588 performance for mail that arrives and leaves via SMTP, provided
 589 that the content filter creates no temporary files. Each temporary
 590 file created by the content filter adds another factor to the
 591 performance loss. </p>
 592
 593 <h3>Advanced content filter: requesting that all mail is filtered</h3>
 594
 595 <p> To enable the advanced content filter method for all mail,
 596 specify in <a href="postconf.5.html">main.cf</a>:  </p>
 597
 598 <blockquote>
 599 <pre>
 600 /etc/postfix/<a href="postconf.5.html">main.cf</a>:
 601     <a href="postconf.5.html#content_filter">content_filter</a> = scan:localhost:10025
 602     <a href="postconf.5.html#receive_override_options">receive_override_options</a> = <a href="postconf.5.html#no_address_mappings">no_address_mappings</a>
 603 </pre>
 604 </blockquote>
 605
 606 <ul>
 607
 608 <li> <p> The "<a href="postconf.5.html#receive_override_options">receive_override_options</a>" line disables address
 609 manipulation before the content filter, so that the content filter
 610 sees the original mail addresses instead of the result of virtual
 611 alias expansion, canonical mapping, automatic bcc, address
 612 masquerading, etc. </p>
 613
 614 <li> <p> The "<a href="postconf.5.html#content_filter">content_filter</a>" line causes Postfix to add one content
 615 filter request record to each incoming mail message, with content
 616 "scan:localhost:10025".  The content filter request records are
 617 added by the <a href="smtpd.8.html">smtpd(8)</a> and <a href="pickup.8.html">pickup(8)</a> servers (and <a href="qmqpd.8.html">qmqpd(8)</a> if you
 618 decide to enable this service). </p>
 619
 620 <li> <p> Content filter requests are stored in queue files; this
 621 is how Postfix keeps track of what mail needs filtering.  When a
 622 queue file contains a content filter request, the queue manager
 623 will deliver the mail to the specified content filter regardless
 624 of its final destination. </p>
 625
 626 <li> <p> The <a href="postconf.5.html#content_filter">content_filter</a> configuration parameter expects a value
 627 of the form <i>transport:destination</i>. The <i>transport</i> name
 628 specifies the first field of a mail delivery agent definition in
 629 <a href="master.5.html">master.cf</a>; the syntax of the next-hop <i>destination</i> is described
 630 in the manual page of the corresponding delivery agent. </p>
 631
 632 <li> <p> The meaning of an empty next-hop filter <i>destination</i>
 633 is version dependent.  Postfix 2.7 and later will use the recipient
 634 domain; earlier versions will use $<a href="postconf.5.html#myhostname">myhostname</a>.  Specify
 635 "<a href="postconf.5.html#default_filter_nexthop">default_filter_nexthop</a> = $<a href="postconf.5.html#myhostname">myhostname</a>" for compatibility with Postfix
 636 2.6 or earlier, or specify a non-empty next-hop filter <i>destination</i>.
 637
 638 <li> <p> The <a href="postconf.5.html#content_filter">content_filter</a> setting has lower precedence than a
 639 FILTER action that is specified in an <a href="access.5.html">access(5)</a>, <a href="header_checks.5.html">header_checks(5)</a>
 640 or <a href="header_checks.5.html">body_checks(5)</a> table. </p>
 641
 642 </ul>
 643
 644 <h3> Advanced content filter: sending unfiltered mail to the content
 645 filter</h3>
 646
 647 <p> In this example, "scan" is an instance of the Postfix SMTP
 648 client with slightly different configuration parameters. This is
 649 how one would set up the service in the Postfix <a href="master.5.html">master.cf</a> file:
 650 </p>
 651
 652 <blockquote>
 653 <pre>
 654 /etc/postfix/<a href="master.5.html">master.cf</a>:
 655     # =============================================================
 656     # service type  private unpriv  chroot  wakeup  maxproc command
 657     #               (yes)   (yes)   (yes)   (never) (100)
 658     # =============================================================
 659     scan      unix  -       -       n       -       10      smtp
 660         -o <a href="postconf.5.html#smtp_send_xforward_command">smtp_send_xforward_command</a>=yes
 661         -o <a href="postconf.5.html#disable_mime_output_conversion">disable_mime_output_conversion</a>=yes
 662         -o <a href="postconf.5.html#smtp_generic_maps">smtp_generic_maps</a>=
 663 </pre>
 664 </blockquote>
 665
 666 <ul>
 667
 668 <li> <p> This runs up to 10 content filters in parallel. Instead
 669 of a limit of 10 concurrent processes, use whatever process limit
 670 is feasible for your machine.  Content inspection software can
 671 gobble up a lot of system resources, so you don't want to have too
 672 much of it running at the same time. </p>
 673
 674 <li> <p> With "-o <a href="postconf.5.html#smtp_send_xforward_command">smtp_send_xforward_command</a>=yes", the scan transport
 675 will try to forward the original client name and IP address
 676 through the content filter to the
 677 after-filter smtpd process, so that filtered mail is logged with
 678 the real client name IP address. See <a href="smtp.8.html">smtp(8)</a> and <a href="XFORWARD_README.html">XFORWARD_README</a>
 679 for more information. </p>
 680
 681 <li> <p> The "-o <a href="postconf.5.html#disable_mime_output_conversion">disable_mime_output_conversion</a>=yes" is a workaround
 682 that prevents the breaking of domainkeys and other digital signatures.
 683 This is needed because some SMTP-based content filters don't announce
 684 8BITMIME support, even though they can handle 8-bit mail. </p>
 685
 686 <li> <p> The "-o <a href="postconf.5.html#smtp_generic_maps">smtp_generic_maps</a>=" is a workaround that prevents
 687 local address rewriting with <a href="generic.5.html">generic(5)</a> maps. Such rewriting should
 688 happen only when mail is sent out to the Internet.  </p>
 689
 690 </ul>
 691
 692 <h3>Advanced content filter: running the content filter</h3>
 693
 694 <p> The content filter can be set up with the Postfix spawn service,
 695 which is the Postfix equivalent of inetd. For example, to instantiate
 696 up to 10 content filtering processes on localhost port 10025: </p>
 697
 698 <blockquote>
 699 <pre>
 700 /etc/postfix/<a href="master.5.html">master.cf</a>:
 701     # ===================================================================
 702     # service       type  private unpriv  chroot  wakeup  maxproc command
 703     #                     (yes)   (yes)   (yes)   (never) (100)
 704     # ===================================================================
 705     localhost:10025 inet  n       n       n       -       10      spawn
 706         user=filter argv=/path/to/filter localhost 10026
 707 </pre>
 708 </blockquote>
 709
 710 <ul>
 711
 712 <li> <p> "filter" is a dedicated local user account.  The user will
 713 never log in, and can be given a "*" password and non-existent
 714 shell and home directory.  This user handles all potentially
 715 dangerous mail content - that is why it should be a separate account.
 716 </p>
 717
 718 <li> <p> By default, Postfix will terminate a command that runs
 719 longer than <a href="postconf.5.html#command_time_limit">command_time_limit</a> seconds (default: 1000s). This is a
 720 safety measure that prevents filters from running forever. </p>
 721
 722 </ul>
 723
 724 <p> If you want to have your filter listening on port localhost:10025
 725 instead of Postfix, then you must run your filter as a stand-alone
 726 program, and must not use the Postfix spawn service.  </p>
 727
 728 <h3>Advanced filter: injecting mail back into Postfix</h3>
 729
 730 <p> The job of the content filter is to either bounce mail with a
 731 suitable diagnostic, or to feed the mail back into Postfix through
 732 a dedicated listener on port localhost 10026. </p>
 733
 734 <p> The simplest content filter just copies SMTP commands and data
 735 between its inputs and outputs. If it has a problem, all it has to
 736 do is to reply to an input of `.' from Postfix with `550 content
 737 rejected', and to disconnect without sending `.' on the connection
 738 that injects mail back into Postfix. </p>
 739
 740 <blockquote>
 741 <pre>
 742 /etc/postfix/<a href="master.5.html">master.cf</a>:
 743     # ===================================================================
 744     # service       type  private unpriv  chroot  wakeup  maxproc command
 745     #                     (yes)   (yes)   (yes)   (never) (100)
 746     # ===================================================================
 747     localhost:10026 inet  n       -       n       -       10      smtpd
 748         -o <a href="postconf.5.html#content_filter">content_filter</a>=
 749         -o <a href="postconf.5.html#receive_override_options">receive_override_options</a>=<a href="postconf.5.html#no_unknown_recipient_checks">no_unknown_recipient_checks</a>,<a href="postconf.5.html#no_header_body_checks">no_header_body_checks</a>,<a href="postconf.5.html#no_milters">no_milters</a>
 750         -o <a href="postconf.5.html#smtpd_helo_restrictions">smtpd_helo_restrictions</a>=
 751         -o <a href="postconf.5.html#smtpd_client_restrictions">smtpd_client_restrictions</a>=
 752         -o <a href="postconf.5.html#smtpd_sender_restrictions">smtpd_sender_restrictions</a>=
 753         -o <a href="postconf.5.html#smtpd_recipient_restrictions">smtpd_recipient_restrictions</a>=<a href="postconf.5.html#permit_mynetworks">permit_mynetworks</a>,reject
 754         -o <a href="postconf.5.html#mynetworks">mynetworks</a>=127.0.0.0/8
 755         -o <a href="postconf.5.html#smtpd_authorized_xforward_hosts">smtpd_authorized_xforward_hosts</a>=127.0.0.0/8
 756 </pre>
 757 </blockquote>
 758
 759 <ul>
 760
 761 <li> <p> NOTE: do not use spaces around the "=" or "," characters. </p>
 762
 763 <li> <p> NOTE: the SMTP server must not have a smaller process
 764 limit than the "filter" <a href="master.5.html">master.cf</a> entry. </p>
 765
 766 <li> <p> The "-o <a href="postconf.5.html#content_filter">content_filter</a>=" overrides <a href="postconf.5.html">main.cf</a> settings, and
 767 requests no content filtering for mail from the content filter.
 768 This is required or else mail will loop. </p>
 769
 770 <li> <p> The "-o <a href="postconf.5.html#receive_override_options">receive_override_options</a>" overrides <a href="postconf.5.html">main.cf</a> settings
 771 to avoid duplicating work that was already done before the content
 772 filter. These options are complementary to the options that are
 773 specified in <a href="postconf.5.html">main.cf</a>: </p>
 774
 775 <ul>
 776
 777     <li> <p> We specify "<a href="postconf.5.html#no_unknown_recipient_checks">no_unknown_recipient_checks</a>" to disable
 778     attempts to find out if a recipient is unknown. </p>
 779
 780     <li> <p> We specify "<a href="postconf.5.html#no_header_body_checks">no_header_body_checks</a>" to disable header/body
 781     checks.  </p>
 782
 783     <li> <p> We specify "<a href="postconf.5.html#no_milters">no_milters</a>" to disable Milter applications
 784     (this option is available only in Postfix 2.3 and later).  </p>
 785
 786     <li> <p> We don't specify "<a href="postconf.5.html#no_address_mappings">no_address_mappings</a>" here.  This
 787     enables virtual alias expansion, canonical mappings, address
 788     masquerading, and other address mappings after the content
 789     filter. The <a href="postconf.5.html">main.cf</a> setting of "<a href="postconf.5.html#receive_override_options">receive_override_options</a>"
 790     disables these mappings before the content filter.  </p>
 791
 792 </ul>
 793
 794   <p> These receive override options are either implemented by the
 795   SMTP server itself, or they are passed on to the cleanup server.
 796   </p>
 797
 798 <li> <p> The "-o smtpd_xxx_restrictions" and "-o <a href="postconf.5.html#mynetworks">mynetworks</a>=127.0.0.0/8"
 799 override <a href="postconf.5.html">main.cf</a> settings. They turn off junk mail controls that
 800 would only waste time here.
 801
 802 <li> <p> With "-o <a href="postconf.5.html#smtpd_authorized_xforward_hosts">smtpd_authorized_xforward_hosts</a>=127.0.0.0/8",
 803 the scan transport will try to forward the original client name
 804 and IP address to the after-filter smtpd process, so that filtered
 805 mail is logged with the real client name and IP address.  See
 806 <a href="XFORWARD_README.html">XFORWARD_README</a> and <a href="smtpd.8.html">smtpd(8)</a>. </p>
 807
 808 </ul>
 809
 810 <h2><a name="performance">Advanced content filter performance</a></h2>
 811
 812 <p> With the "sandwich" approach to content filtering described
 813 here, it is important to match the filter concurrency to the
 814 available CPU, memory and I/O resources.  Too few content filter
 815 processes and mail accumulates in the <a href="QSHAPE_README.html#active_queue">active queue</a> even with low
 816 traffic volume; too much concurrency and Postfix ends up deferring
 817 mail destined for the content filter because processes fail due to
 818 insufficient resources. </p>
 819
 820 <p> Currently, content filter performance tuning is a process of
 821 trial and error; analysis is handicapped because filtered and
 822 unfiltered messages share the same queue.  As mentioned in the
 823 introduction of this document, content filtering with multiple
 824 Postfix instances will be covered in a future version.  </p>
 825
 826 <h2><a name="advanced_turnoff">Turning off the advanced content filter</a> </h2>
 827
 828 <p> To turn off "advanced" content filtering: </p>
 829
 830 <ul> <li> <p> Delete or comment out the two following <a href="postconf.5.html">main.cf</a> lines.
 831 The other changes made for advanced content filtering have no effect
 832 when content filtering is turned off. </p>
 833
 834 <blockquote>
 835 <pre>
 836 /etc/postfix/<a href="postconf.5.html">main.cf</a>:
 837     <a href="postconf.5.html#content_filter">content_filter</a> = scan:localhost:10025
 838     <a href="postconf.5.html#receive_override_options">receive_override_options</a> = <a href="postconf.5.html#no_address_mappings">no_address_mappings</a>
 839 </pre>
 840 </blockquote>
 841
 842 <li> <p> Execute "<b>postsuper -r ALL</b>" to remove content
 843 filter request records from existing queue files. </p>
 844
 845 <li> <p> Execute another "<b>postfix reload</b>". </p>
 846
 847 </ul>
 848
 849 <h2><a name="remote_only">Filtering mail from outside users only</a></h2>
 850
 851 <p> The easiest approach is to configure ONE Postfix instance with
 852 multiple SMTP server IP addresses in <a href="master.5.html">master.cf</a>: </p>
 853
 854 <ul>
 855
 856 <li> <p> Two SMTP server IP addresses for mail from inside users only,
 857 with content filtering turned off. </p>
 858
 859 <pre>
 860 /etc/postfix.<a href="master.5.html">master.cf</a>:
 861     # ==================================================================
 862     # service      type  private unpriv  chroot  wakeup  maxproc command
 863     #                    (yes)   (yes)   (yes)   (never) (100)
 864     # ==================================================================
 865     1.2.3.4:smtp   inet  n       -       n       -       -       smtpd
 866         -o <a href="postconf.5.html#smtpd_client_restrictions">smtpd_client_restrictions</a>=<a href="postconf.5.html#permit_mynetworks">permit_mynetworks</a>,reject
 867     127.0.0.1:smtp inet  n       -       n       -       -       smtpd
 868         -o <a href="postconf.5.html#smtpd_client_restrictions">smtpd_client_restrictions</a>=<a href="postconf.5.html#permit_mynetworks">permit_mynetworks</a>,reject
 869 </pre>
 870
 871 <li> <p> One SMTP server address for mail from outside users with
 872 content filtering turned on. </p>
 873
 874 <pre>
 875 /etc/postfix.<a href="master.5.html">master.cf</a>:
 876     # =================================================================
 877     # service     type  private unpriv  chroot  wakeup  maxproc command
 878     #                   (yes)   (yes)   (yes)   (never) (100)
 879     # =================================================================
 880     1.2.3.5:smtp  inet  n       -       n       -       -       smtpd
 881         -o <a href="postconf.5.html#content_filter">content_filter</a>=filter-service:filter-destination
 882         -o <a href="postconf.5.html#receive_override_options">receive_override_options</a>=<a href="postconf.5.html#no_address_mappings">no_address_mappings</a>
 883 </pre>
 884
 885 </ul>
 886
 887 <p> After this, you can follow the same procedure as outlined in
 888 the "advanced" or "simple" content filtering examples above, except
 889 that you must not specify "<a href="postconf.5.html#content_filter">content_filter</a>" or "<a href="postconf.5.html#receive_override_options">receive_override_options</a>"
 890 in the <a href="postconf.5.html">main.cf</a> file. </p>
 891
 892 <h2><a name="domain_dependent">Different filters for different
 893 domains</a></h2>
 894
 895 <p> If you are an MX service provider and want to apply different
 896 content filters for different domains, you can configure ONE Postfix
 897 instance with multiple SMTP server IP addresses in <a href="master.5.html">master.cf</a>. Each
 898 address provides a different content filter service. </p>
 899
 900 <blockquote>
 901 <pre>
 902 /etc/postfix.<a href="master.5.html">master.cf</a>:
 903     # =================================================================
 904     # service     type  private unpriv  chroot  wakeup  maxproc command
 905     #                   (yes)   (yes)   (yes)   (never) (100)
 906     # =================================================================
 907     # SMTP service for domains that are filtered with service1:dest1
 908     1.2.3.4:smtp  inet  n       -       n       -       -       smtpd
 909         -o <a href="postconf.5.html#content_filter">content_filter</a>=service1:dest1
 910         -o <a href="postconf.5.html#receive_override_options">receive_override_options</a>=<a href="postconf.5.html#no_address_mappings">no_address_mappings</a>
 911
 912     # SMTP service for domains that are filtered with service2:dest2
 913     1.2.3.5:smtp  inet  n       -       n       -       -       smtpd
 914         -o <a href="postconf.5.html#content_filter">content_filter</a>=service2:dest2
 915         -o <a href="postconf.5.html#receive_override_options">receive_override_options</a>=<a href="postconf.5.html#no_address_mappings">no_address_mappings</a>
 916 </pre>
 917 </blockquote>
 918
 919 <p> After this, you can follow the same procedure as outlined in
 920 the "advanced" or "simple" content filtering examples above, except
 921 that you must not specify "<a href="postconf.5.html#content_filter">content_filter</a>" or "<a href="postconf.5.html#receive_override_options">receive_override_options</a>"
 922 in the <a href="postconf.5.html">main.cf</a> file. </p>
 923
 924 <p> Set up MX records in the DNS that route each domain to the
 925 proper SMTP server instance. </p>
 926
 927 <h2><a name="dynamic_filter">FILTER actions in access or header/body
 928 tables</a></h2>
 929
 930 <p> The above filtering configurations are static. Mail that follows
 931 a given path is either always filtered or it is never filtered. As
 932 of Postfix 2.0 you can also turn on content filtering on the fly.
 933 </p>
 934
 935 <p> To turn on content filtering with an <a href="access.5.html">access(5)</a> table rule: </p>
 936
 937 <blockquote>
 938 <pre>
 939 /etc/postfix/access:
 940     <i>whatever</i>       FILTER foo:bar
 941 </pre>
 942 </blockquote>
 943
 944 <p> To turn on content filtering with a <a href="header_checks.5.html">header_checks(5)</a> or
 945 <a href="header_checks.5.html">body_checks(5)</a> table pattern: </p>
 946
 947 <blockquote>
 948 <pre>
 949 /etc/postfix/header_checks:
 950     /<i>whatever</i>/     FILTER foo:bar
 951 </pre>
 952 </blockquote>
 953
 954 <p> You can do this in smtpd access maps as well as the cleanup
 955 server's header/body_checks.  This feature must be used with great
 956 care:  you must disable all the UCE features in the after-filter
 957 smtpd and cleanup daemons or else you will have a content filtering
 958 loop. </p>
 959
 960 <p> Limitations: </p>
 961
 962 <ul>
 963
 964 <li> <p> FILTER actions from smtpd access maps and header/body_checks
 965 take precedence over filters specified with the <a href="postconf.5.html">main.cf</a> <a href="postconf.5.html#content_filter">content_filter</a>
 966 parameter. </p>
 967
 968 <li> <p> If a message triggers more than one filter action, only
 969 the last one takes effect. </p>
 970
 971 <li> <p> The same content filter is applied to all the recipients
 972 of a given message. </p>
 973
 974 </ul>
 975
 976 </body>
 977
 978 </html>