join,sort,uniq: with -z, treat '\n' as a field separator

[coreutils.git] / README

These are the GNU core utilities.  This package is the union of
the GNU fileutils, sh-utils, and textutils packages.

Most of these programs have significant advantages over their Unix
counterparts, such as greater speed, additional options, and fewer
arbitrary limits.

The programs that can be built with this package are:

  [ arch base32 base64 basename cat chcon chgrp chmod chown chroot cksum comm
  coreutils cp csplit cut date dd df dir dircolors dirname du echo env
  expand expr factor false fmt fold groups head hostid hostname id install
  join kill link ln logname ls md5sum mkdir mkfifo mknod mktemp mv nice nl
  nohup nproc numfmt od paste pathchk pinky pr printenv printf ptx pwd
  readlink realpath rm rmdir runcon seq sha1sum sha224sum sha256sum sha384sum
  sha512sum shred shuf sleep sort split stat stdbuf stty sum sync tac tail
  tee test timeout touch tr true truncate tsort tty uname unexpand uniq
  unlink uptime users vdir wc who whoami yes

See the file NEWS for a list of major changes in the current release.

If you obtained this file as part of a "git clone", then see the
README-hacking file.  If this file came to you as part of a tar archive,
then see the file INSTALL for compilation and installation instructions.

These programs are intended to conform to POSIX (with BSD and other
extensions), like the rest of the GNU system.  By default they conform
to older POSIX (1003.2-1992), and therefore support obsolete usages
like "head -10" and "chown owner.group file".  This default is
overridden at build-time by the value of <unistd.h>'s _POSIX2_VERSION
macro, and this in turn can be overridden at runtime as described in
the documentation under "Standards conformance".

The ls, dir, and vdir commands are all separate executables instead of
one program that checks argv[0] because people often rename these
programs to things like gls, gnuls, l, etc.  Renaming a program
file shouldn't affect how it operates, so that people can get the
behavior they want with whatever name they want.

Special thanks to Paul Eggert, Brian Matthews, Bruce Evans, Karl Berry,
Kaveh Ghazi, and François Pinard for help with debugging and porting
these programs.  Many thanks to all of the people who have taken the
time to submit problem reports and fixes.  All contributed changes are
attributed in the commit logs.

And thanks to the following people who have provided accounts for
portability testing on many different types of systems: Bob Proulx,
Christian Robert, François Pinard, Greg McGary, Harlan Stenn,
Joel N. Weber, Mark D. Roth, Matt Schalit, Nelson H. F. Beebe,
Réjean Payette, Sam Tardieu.

Thanks to Michael Stone for inflicting test releases of this package
on Debian's unstable distribution, and to all the kind folks who used
that distribution and found and reported bugs.

Note that each man page is now automatically generated from a template
and from the corresponding --help usage message.  Patches to the template
files (man/*.x) are welcome.  However, the authoritative documentation
is in texinfo form in the doc directory.


*********************************************
On Mac OS X 10.5.1 (Darwin 9.1), test failure
---------------------------------------------

Mac OS X 10.5.1 (Darwin 9.1) provides only partial (and incompatible)
ACL support, so although "./configure && make" succeeds, "make check"
exposes numerous failures.  The solution is to turn off ACL support
manually via "./configure --disable-acl".  For details, see
<http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/12292/focus=12318>.


*****************************************
Test failure with NLS and gettext <= 0.17
-----------------------------------------

Due to a conflict between libintl.h and gnulib's new xprintf module,
when you configure with NLS support, and with a gettext installation
older than 0.17.1 (not yet released, at the time of this writing),
then some tests fail, at least on NetBSD 1.6.  To work around it in
the mean time, you can configure with --disable-nls.  For details,
see <http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/12015/>.


*********************
Pre-C99 build failure
---------------------

In 2009 we added this requirement:
To build the coreutils from source, you must have a C99-conforming
compiler, due to the use of declarations after non-declaration statements
in several files in src/.  There is code in configure to find and, if
possible, enable an appropriate compiler.  However, if configure doesn't
find a C99 compiler, it continues nonetheless, and your build will fail.
There used to be a "c99-to-c89.diff" patch you could apply to convert
to code that even an old pre-c99 compiler can handle, but it was too
tedious to maintain, so has been removed.


***********************
HPUX 11.x build failure
-----------------------

A known problem exists when compiling on HPUX on both hppa and ia64
in 64-bit mode (i.e., +DD64) on HP-UX 11.0, 11.11, and 11.23.  This
is not due to a bug in the package but instead due to a bug in the
system header file which breaks things in 64-bit mode.  The default
compilation mode is 32-bit and the software compiles fine using the
default mode.  To build this software in 64-bit mode you will need
to fix the system /usr/include/inttypes.h header file.  After
correcting that file the software also compiles fine in 64-bit mode.
Here is one possible patch to correct the problem:

--- /usr/include/inttypes.h.orig        Thu May 30 01:00:00 1996
+++ /usr/include/inttypes.h     Sun Mar 23 00:20:36 2003
@@ -489 +489 @@
-#ifndef __STDC_32_MODE__
+#ifndef __LP64__


************************
OSF/1 4.0d build failure
------------------------

If you use /usr/bin/make on an OSF/1 4.0d system, it will fail due
to the presence of the "[" target.  That version of make appears to
treat "[" as some syntax relating to locks.  To work around that,
the best solution is to use GNU make.  Otherwise, simply remove
all mention of "[$(EXEEXT)" from src/Makefile.


*************************************************
"make check" failure on IRIX 6.5 and Solaris <= 9
-------------------------------------------------

Using the vendor make program to run "make check" fails on these two systems.
If you want to run all of the tests there, use GNU make.



**********************
Running tests as root:
----------------------

If you run the tests as root, note that a few of them create files
and/or run programs as a non-root user, 'nobody' by default.
If you want to use some other non-root username, specify it via
the NON_ROOT_USERNAME environment variable.  Depending on the
permissions with which the working directories have been created,
using 'nobody' may fail, because that user won't have the required
read and write access to the build and test directories.
I find that it is best to unpack and build as a non-privileged
user, and then to run the following command as that user in order
to run the privilege-requiring tests:

  sudo env PATH="$PATH" NON_ROOT_USERNAME=$USER make -k check-root

If you can run the tests as root, please do so and report any
problems.  We get much less test coverage in that mode, and it's
arguably more important that these tools work well when run by
root than when run by less privileged users.


***************
Reporting bugs:
---------------

Send bug reports, questions, comments, etc. to bug-coreutils@gnu.org.
To suggest a patch, see the files README-hacking and HACKING for tips.

If you have a problem with 'sort', try running 'sort --debug', as it
can can often help find and fix problems without having to wait for an
answer to a bug report.  If the debug output does not suffice to fix
the problem on your own, please compress and attach it to the rest of
your bug report.

IMPORTANT: if you take the time to report a test failure,
please be sure to include the output of running 'make check'
in verbose mode for each failing test.  For example,
if the test that fails is tests/df/df-P.sh, then you would
run this command:

  make check TESTS=tests/df/df-P.sh VERBOSE=yes SUBDIRS=. >> log 2>&1

For some tests, you can get even more detail by adding DEBUG=yes.
Then include the contents of the file 'log' in your bug report.


***************************************

There are many tests, but nowhere near as many as we need.
Additions and corrections are very welcome.

If you see a problem that you've already reported, feel free to re-report
it -- it won't bother me to get a reminder.  Besides, the more messages I
get regarding a particular problem the sooner it'll be fixed -- usually.
If you sent a complete patch and, after a couple weeks you haven't
received any acknowledgement, please ping us.  A complete patch includes
a well-written ChangeLog entry, unified (diff -u format) diffs relative
to the most recent test release (or, better, relative to the latest
sources in the public repository), an explanation for why the patch is
necessary or useful, and if at all possible, enough information to
reproduce whatever problem prompted it.  Plus, you'll earn lots of
karma if you include a test case to exercise any bug(s) you fix.
Here are instructions for checking out the latest development sources:

  http://savannah.gnu.org/git/?group=coreutils

If your patch adds a new feature, please try to get some sort of consensus
that it is a worthwhile change.  One way to do that is to send mail to
coreutils@gnu.org including as much description and justification
as you can.  Based on the feedback that generates, you may be able to
convince us that it's worth adding.  Please also consult the list of
previously discussed but ultimately rejected feature requests at:
http://www.gnu.org/software/coreutils/rejected_requests.html


WARNING:  Now that we use the ./bootstrap script, you should not run
autoreconf manually.  Doing that will overwrite essential source files
with older versions, which may make the package unbuildable or introduce
subtle bugs.


WARNING:  If you modify files like configure.in, m4/*.m4, aclocal.m4,
or any Makefile.am, then don't be surprised if what gets regenerated no
longer works.  To make things work, you'll have to be using appropriate
versions of the tools listed in bootstrap.conf's buildreq string.

All of these programs except 'test' recognize the '--version' option.
When reporting bugs, please include in the subject line both the package
name/version and the name of the program for which you found a problem.

For general documentation on the coding and usage standards
this distribution follows, see the GNU Coding Standards,
http://www.gnu.org/prep/standards_toc.html.

For any copyright year range specified as YYYY-ZZZZ in this package
note that the range specifies every single year in that closed interval.

Mail suggestions and bug reports for these programs to
the address on the last line of --help output.


========================================================================

Copyright (C) 1998-2016 Free Software Foundation, Inc.

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with no
Invariant Sections, with no Front-Cover Texts, and with no Back-Cover
Texts.  A copy of the license is included in the "GNU Free
Documentation License" file as part of this distribution.