mk/make-config.sh: more human friendly $OBJDIR/mk-config.env layout

[s-mailx.git] / INSTALL

I n s t a l l i n g  S - n a i l / S - m a i l x
================================================

1.0 Compilation
1.1 Configuration failed!
1.2 Building failed!
1.3 Tests failed!
1.4 How can i enable debugging?
2.0 Notes on building the latest release

1.0 Compilation
---------------

System specific notes can be found in the next section.
All (optional) features are documented (and adjustable) in make.rc.
Adjustments may also take place, and are usually done, from the command
line, overriding those made in make.rc (if any).
The set of builtin MIME types could be adjusted in ./mime.types.
Experts could find it valuable to adjust C code in ./mx-config.h.

Without any adjustments all non-experimental features will be enabled,
except some which provide redundant functionality (e.g., OPT_SPAM_SPAMC
is disabled because the generic OPT_SPAM_FILTER can do the same).
None of the features are "require"d by default, so that configuration
will not fail shall any of them not be available or usable.
The generated configuration is tracked: changes will be recognized
and cause automatic cleanups and rebuilds as necessary.

  $ make -j2 tangerine  # equals "$ make config build test install"
  $ make citron         # equals "$ make config build install"
  $ make distclean      # *Completely* cleanup working directory

With adjustments:

  $ make OPT_POP3=no OPT_SMTP=require OBJDIR=/tmpfs/mx tangerine
  $ make OPT_CROSS_BUILD=y VAL_PREFIX=/some/nasty/prefix citron

With utility program and feature adjustments:

  $ make awk=/usr/bin/nawk VERBOSE=yes OPT_NET=no tangerine

If OPT_DOTLOCK has been enabled then the minimal privilege-separated
SETUID (to VAL_DOTLOCK_USER, default "root") helper program will be build
and installed, and therefore the installation process needs to have the
appropriate privileges.  In this case it may be useful to separate the
configuration / building and the installation tasks and give the last
step higher privileges via super(1), sudo(1), su(1) or a similar
mechanism, e.g.:

  $ make VAL_PREFIX=/usr config && make -j4 &&
    super make DESTDIR=./xy install

would create a "s-nail" binary and install a "s-nail" manual etc.  under
the prefix "/usr" but rooted under "[./]xy", i.e., the binary would be
installed as "[./]xy/usr/bin/s-nail".

Out-of-tree compilation is supported; to use it, create the target
directory of desire, change into it and run the make-emerge.sh script
shipped with S-nail, then proceed as with normal in-tree building.
Generated configuration files, objects and test output will be located
in the directory .obj/ (or what $OBJDIR defines), in- or off-tree.

The following make(1) targets exist, the default being `build':

- tangerine   Shorthand for "$ make config build test install": create
              or check and update configuration, build, test and install.
              The variable $DESTDIR will be honoured (see make.rc),
              but not be tracked in the configuration.
- citron      Shorthand for "$ make config build install".
- all         Shorthand for "$ make config build".

- config      Only create or check and update the configuration.
- build       Only build (using the existing configuration).
- install     Only install using the built files of the existing
              configuration.
              The variable $DESTDIR will be honoured (see make.rc),
              but not be tracked in the configuration.

              S-nail will create an uninstall shell script
              (${VAL_SID}${VAL_MAILX}-uninstall.sh), but which will
              *not* be installed if $DESTDIR is set non-empty -- within
              $DESTDIR, that is.  Copy or move it manually from the S-nail
              $OBJDIR into $DESTDIR as necessary in that case.

- clean       Remove anything which can be rebuild.
- distclean   Remove anything which can be rebuild or reconfigured.

- test        Run mx-test.sh in --check-only mode on the built binary.
- testnj      Likewise, but add --no-jobs to avoid spawning multiple
              tests in parallel.

Setting the make(1) variable $VERBOSE to an arbitrary value during
`config' time, as in "$ make VERBOSE=xy tangerine", will change the
output of the `all', `install' etc. targets to a more verbose one.

There are also some predefined configuration sets available.
It is possible to apply manual adjustments on top of those.

- CONFIG=NULL, CONFIG=NULLI
  Anything that can be turned off is off.  MIME cannot.
  The latter adds and "require"s iconv(3) and adds the user interface
  strings.

- CONFIG=MINIMAL
  Possibly what people want who need nothing but a MIME-capable mailx(1)
  and do not regret improved usability for the rare interactive use
  occasions.  Adds documentation strings, the built-in line editor (MLE)
  with history support and key bindings, error tracking, basic colour
  support and IDNA addresses, as well as generic spam filter support.

  "Require"s iconv(3), regex(3) and the dotlock helper.

- CONFIG=NETSEND
  Sending messages directly to the mail provider via the SMTP protocol,
  instead of requiring a local mail-transfer-agent (MTA) who does.

  Adds SSL/TLS, SMTP, GSSAPI and .netrc file parsing on top of MINIMAL.

  "Require"s iconv(3), SSL/TLS, SMTP (sockets) and the dotlock helper.

- CONFIG=MAXIMAL
  Like MINIMAL, but turns on all (other) options, also obsolete or
  redundant ones, but none of them required.

If during configuration or building some libraries seem to be missing
that you know are installed on your system, or if other errors occur
due to missing files but which you know exist, please ensure that the
environment variable $C_INCLUDE_PATH includes the necessary "include/"
paths and the environment variable $LD_LIBRARY_PATH includes the
necessary "lib/"rary paths.

The build system will inspect these two environment variables and
automatically convert them to cc(1) (c99(1)) -I and -L options (since
these environment variables are, different to the command line options,
not part of the POSIX standard).
To set these environment variables, the following can be done in
a Bourne / Korn / POSIX compatible shell:

  $ C_INCLUDE_PATH="${C_INCLUDE_PATH}:/usr/local/include"
  $ LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/usr/local/lib"
  $ export C_INCLUDE_PATH LD_LIBRARY_PATH
  $ make tangerine CONFIG=MINIMAL

Other than the standard paths /usr/{bin,include,lib} and /usr/local/[.]
should possibly be placed first instead, presuming that they are meant
to override things which usually exist in standard locations.
If all else fails you can also forcefully pass in include directives and
library paths via prefilled $INCS and $LIBS variables:

  $ make INCS=-I/mypath/include LIBS="-L/mypath/lib -lmylib" tangerine

1.1 Configuration failed!
-------------------------

Please see the section "Notes on building the latest release" below
first, maybe it has an item for the failing platform, or even gives an
answer on how to proceed nonetheless.

The configuration process creates a directory ./.obj (or $OBJDIR, if
set) to store some files of interest (it is also here where the build
and the tests take place):

- mk-config.log  Any output generated during the configuration process.
- mk-config.h    C program header representing the configuration.
- mk-config.env  sh(1) environment representing the configuration.

Of special interest is mk-config.log since the error usually manifests
here in textual output.  Maybe that makes it obvious what can be done
(header files could not be found because of missing entries in
$C_INCLUDE_PATH, libraries could not be linked because of incomplete
$LD_LIBRARY_PATH, etc.).

If things went wrong so completely that even "$ make distclean" is not
possible, simply removing .obj ($OBJDIR) will do: "$ rm -rf ./.obj".
In any case it would be appreciated if you could report the problem,
so that problems can be fixed!

1.2 Building failed!
--------------------

Terrible!  This should not happen if configuration succeeded!
Again, it would be appreciated if you could report the problem, so that
problems can be fixed!

1.3 Tests failed!
-----------------

Please see the section "Notes on building the latest release" below
first: sometimes it is known that certain tests fail on a specific
target platform, and there is no applicable fix on application level.
For example, it is not uncommon that Unicode tests fail because the
platform uses old or incomplete character information.

It could also be a miscompilation, caused by excessive optimization
and/or a compiler bug.  If OPT_AUTOCC had been used in conjunction with
a non-debug target (the defaults), an additional cc_maxopt=1 (or an
EXTRA_CFLAGS=-O1 etc.) on the command line and a successive
reconfiguration and -compilation could turn the green testing light on.

In order to inspect the error more thoroughly, the mx-test.sh script
can be used in --run-test mode to invoke the failing tests (testing
echoes the actual test names in brackets), for example

  $ ./mx-test.sh --run-test ./s-nail localopts mbox

would run the tests "localopts" and "mbox".  Test output files look
like t.TESTNAME-XY and reside  in the ./.obj ($OBJDIR) directory.
(If the build and test is done in a repository checkout, and if the
branch "test-out" exists, then in addition to the failing test output
the expected as well as -- optionally -- the file difference of these
two is generated automatically, too.  And these outputs are also
generated for normal "$ make test" runs, then.)

It might make sense to send the output files of the failing tests to
the mailing-list.  Because of ML message size restrictions, creating
a compressed tar(1) archive of the files would be appreciated:

  $ rm -f .obj/t.*.old .obj/t.*.diff  # In repository checkout only
  $ tar -cf - .obj/t.* | xz > tlogs.tar.xz
or
  $ tar -cf - .obj/t.* | gzip > tlogs.tar.gz
or
  $ tar -cf - .obj/t.* | zstd -T0 -19 > tlogs.tar.zst
etc.

Thank you!

1.4 How can i enable debugging?
-------------------------------

Enable OPT_DEBUG=yes during compilation

  $ make CONFIG=MAXIMAL OPT_DEBUG=yes

to gain compiler debug flags, memory bound canaries and Not-Yet-Dead
function graph listings in case of crashes etc.
For sanitizers plus special options exist, please read make.rc for more.

In case of errors using the -d (*debug*) and -v (*verbose*; may be set
multiply for more verbosity) options (variables) is always good idea:

  $ s-nail -dvv
or
  $ s-nail -Sdebug -Sverbose -Sverbose

Should you really discover any problems with S-nail it would be very
useful for development if you would contact the mailing-list!
Thank you!

2.0 Notes on building the latest release
----------------------------------------

- All systems:
  * Running the tests on a NFS filesystem may cause false negatives.
    (Seen on the SunOS 5.11 OpenCSW machines, in [pipe_handlers]
    failure due to hard link count mismatch since NFS links the file
    to some other name, too, and that lingers too long, reproducibly).

  * Various warnings for diverse compiler versions etc.  This is a race
    which cannot be won, so i make the software compile and test under
    each compiler version i get my fingers at, and of course i fix bugs
    when i see them.
    The warnings on unused returns from some I/O functions will vanish
    after the large v15 I/O and MIME rewrite.

- All 32-bit systems:
  * There _may_ be warnings about format strings, like, e.g.,
      auxlily.c:1610:10: warning: format '%lu' expects type 'long
      unsigned int', but argument 3 has type 'size_t'
    The codebase is ISO C89 which has no %z printf(3) format.
    However we try hard to detect the real type size and define the
    "PRI[du]Z" macros which end up with the correct size, which is
    also compile-time asserted (via "MCTA(sizeof(size_t) == XZ)").

    By forcing ISO C99 mode when compiling these warnings vanish, e.g.,
    with gcc(1) and clang(1): with OPT_AUTOCC pass
    "EXTRA_CFLAGS=-std=c99", otherwise ensure -std=c99 in $CFLAGS.

Development:

. CRUX Linux <https://www.crux.nu/> (3.5/x86-64).

Frequently:

. AlpineLinux <https://www.alpinelinux.org> (x86-64: edge).

. ArchLinux <https://www.archlinux.org/> (<2019>,current).

. FreeBSD <https://www.freebsd.org/> (x86: 11.{2,3-RC2},12.1).
  - v11. series test errors: see DragonFly.

. OpenBSD <https://www.openbsd.org/> (x86: 6.[456]).

Pre-release, and other landmarks:

. DragonFly BSD <https://www.dragonflybsd.org/> (5.{6.1,8.0}).
  - The tests [iconv_mbyte_base64]-{4,8} fail because the C function
    iswprint(3) falsely claims that U+FF08 and U+FF09 (FULLWIDTH
    LEFT and RIGHT PARENTHESIS, respectively) are not printable,
    causing replacements.  This is a known bug, see
      <https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225692>

. NetBSD <https://www.netbsd.org/> (amd64: 9.0).

. OSUKISS Linux <https://osukiss.safe.ca/> (boing-boom-tschak)
  @ Thanks to Jean-Marc Pigeon for generously providing access to
    a VZGOT on the most "beefy" machine i have ever worked on! ^_^

. Solaris <http://opencsw.org/>
  @ First of all: thanks to OpenCSW.org for offering SSH access to
    their Solaris build cluster!
    (Sparc and x86 machines with SunOS 5.9, 5.10 and 5.11.)

  * The original Sun roff (aka AT&T ditroff) is not capable to display
    the manual.  We offer preformatted manual pages, please see the
    actual release announcement for details.

  * Some notes collected on earlier trials:
    + We forcefully disable stack protectors on SunOS/gcc because of
      linking errors seen in earlier tests.
    + If you get the compiler / system header installation error
        Undefined                       first referenced
         symbol                             in file
        __builtin_stdarg_start              auxlily.o
      you have to overwrite the symbol with __builtin_va_start,
      e.g., in conjunction with OPT_AUTOCC add this:
       EXTRA_CFLAGS='-D__builtin_stdarg_start=__builtin_va_start'

  - You may see failures in Unicode-related tests due to outdated or
    false Unicode tables on elder releases (as in NetBSD).

. Void Linux <https://www.voidlinux.org/> (x86)
  - Still not have setup all my environment after having lost all,
    so not tested recently.

# s-ts-mode