gzip: test invalid-input bug

[gzip.git] / NEWS

GNU gzip NEWS                                    -*- outline -*-

* Noteworthy changes in release ?.? (????-??-??) [?]

** Bug fixes

  'gzip -d' no longer fails to report invalid compressed data
  that uses a dictionary distance outside the input window.
  [bug present since the beginning]


* Noteworthy changes in release 1.12 (2022-04-07) [stable]

** Changes in behavior

  'gzip -l' no longer misreports file lengths 4 GiB and larger.
  Previously, 'gzip -l' output the 32-bit value stored in the gzip
  header even though that is the uncompressed length modulo 2**32.
  Now, 'gzip -l' calculates the uncompressed length by decompressing
  the data and counting the resulting bytes.  Although this can take
  much more time, nowadays the correctness pros seem to outweigh the
  performance cons.

  'zless' is no longer installed on platforms lacking 'less'.

** Bug fixes

  zgrep applied to a crafted file name with two or more newlines
  can no longer overwrite an arbitrary, attacker-selected file.
  This addresses CVE-2022-1271, ZDI-CAN-16587.
  [bug introduced in gzip-1.3.10]

  zgrep now names input file on error instead of mislabeling it as
  "(standard input)", if grep supports the GNU -H and --label options.

  'zdiff -C 5' no longer misbehaves by treating '5' as a file name.
  [bug present since the beginning]

  Configure-time options like --program-prefix now work.


* Noteworthy changes in release 1.11 (2021-09-03) [stable]

** Performance improvements

  IBM Z platforms now support hardware-accelerated deflation.


* Noteworthy changes in release 1.10 (2018-12-29) [stable]

** Changes in behavior

  Compressed gzip output no longer contains the current time as a
  timestamp when the input is not a regular file.  Instead, the output
  contains a null (zero) timestamp.  This makes gzip's behavior more
  reproducible when used as part of a pipeline.  (As a reminder, even
  regular files will use null timestamps after the year 2106, due to a
  limitation in the gzip format.)

** Bug fixes

  A use of uninitialized memory on some malformed inputs has been fixed.
  [bug present since the beginning]

  A few theoretical race conditions in signal handers have been fixed.
  These bugs most likely do not happen on practical platforms.
  [bugs present since the beginning]


* Noteworthy changes in release 1.9 (2018-01-07) [stable]

** Bug fixes

  gzip -d -S SUFFIX file.SUFFIX would fail for any upper-case byte in SUFFIX.
  E.g., before, this command would fail:
    $ :|gzip > kT && gzip -d -S T kT
    gzip: kT: unknown suffix -- ignored
  [bug present since the beginning]

  When decompressing data in 'pack' format, gzip no longer mishandles
  leading zeros in the end-of-block code.  [bug introduced in gzip-1.6]

  When converting from system-dependent time_t format to the 32-bit
  unsigned MTIME format used in gzip files, if a timestamp does not
  fit gzip now substitutes zero instead of the timestamp's low-order
  32 bits, as per Internet RFC 1952.  When converting from MTIME to
  time_t format, if a timestamp does not fit gzip now warns and
  substitutes the nearest in-range value instead of crashing or
  silently substituting an implementation-defined value (typically,
  the timestamp's low-order bits).  This affects timestamps before
  1970 and after 2106, and timestamps after 2038 on platforms with
  32-bit signed time_t.  [bug present since the beginning]

  Commands implemented via shell scripts are now more consistent about
  failure status.  For example, 'gunzip --help >/dev/full' now
  consistently exits with status 1 (error), instead of with status 2
  (warning) on some platforms.  [bug present since the beginning]

  Support for VMS and Amiga has been removed.  It was not working anyway,
  and it reportedly caused file name glitches on MS-Windowsish platforms.


* Noteworthy changes in release 1.8 (2016-04-26) [stable]

** Bug fixes

  gzip -l no longer falsely reports a write error when writing to a pipe.
  [bug introduced in gzip-1.7]

  Port to Oracle Solaris Studio 12 on x86-64.
  [bug present since at least gzip-1.2.4]

  When configuring gzip, ./configure DEFS='...-DNO_ASM...' now
  suppresses assembler again.  [bug introduced in gzip-1.3.5]


* Noteworthy changes in release 1.7 (2016-03-27) [stable]

** Changes in behavior

  The GZIP environment variable is now obsolescent; gzip now warns if
  it is used, and rejects attempts to use dangerous options or operands.
  You can use an alias or script instead.

  Installed programs like 'zgrep' now use the PATH environment variable
  as usual to find subsidiary programs like 'gzip' and 'grep'.
  Previously they prepended the installation directory to the PATH,
  which sometimes caused 'make check' to test the wrong gzip executable.
  [bug introduced in gzip-1.3.13]

** New features

  gzip now accepts the --synchronous option, which causes it to use
  fsync and similar primitives to transfer output data to the output
  file's storage device when the file system supports this.  Although
  this option makes gzip safer in the presence of system crashes, it
  can make gzip considerably slower.

  gzip now accepts the --rsyncable option. This option is accepted in
  all modes, but has effect only when compressing: it makes the resulting
  output more amenable to efficient use of rsync.  For example, when a
  large input file gets a small change, a gzip --rsyncable image of
  that file will remain largely unchanged, too.  Without --rsyncable,
  even a tiny change in the input could result in a totally different
  gzip-compressed output file.

** Bug fixes

  gzip -k -v no longer reports that files are replaced.
  [bug present since the beginning]

  zgrep -f A B C no longer reads A more than once if A is not a regular file.
  This better supports invocations like 'zgrep -f <(COMMAND) B C' in Bash.
  [bug introduced in gzip-1.2]


* Noteworthy changes in release 1.6 (2013-06-09) [stable]

** New features

  gzip now accepts the --keep (-k) option, for consistency with tools
  like xz, lzip and bzip2.  With this option, gzip no longer removes
  named input files when compressing or decompressing.

** Bug fixes

  gzip -d no longer malfunctions with certain invalid data in 'pack' format.
  [bug introduced in gzip-0.8]

  When overwriting, gzip no longer acts as if you typed "y" when you type "n",
  on some platforms when compiled with optimization.
  [bug introduced in gzip-1.3.6]

  zgrep no longer malfunctions with a multi-digit context option like -15.
  Now, it passes that option to grep (equivalent to -C15) just as it does
  for single-digit options. [bug introduced in gzip-1.3.12]

  zmore now acts more like 'more', and is more portable to POSIXish hosts.


* Noteworthy changes in release 1.5 (2012-06-17) [stable]

** Bug fixes

  gzip -d now decodes and checks header CRC16 checksums as specified by
  the FHCRC section of Internet RFC 1952.

  "gzip -d -S '' precious.gz" is now rejected immediately.  Before,
  that command would emulate "rm -i precious.gz", but with an easily-
  misunderstood prompt.  I.e., gzip would ask if it's ok to remove the
  existing file, "precious.gz".  If you made the mistake of saying "yes",
  it would remove that input file before attempting to uncompress it.

  gzip -cdf now properly handles input consisting of gzip'd data followed
  by uncompressed data.  Before it would output raw compressed input, too.
  For example, now "(printf x|gzip; echo y)|gzip -dcf" prints "xy\n",
  while before it would print "x<compressed data>y\n".

  gzip -rf no longer compresses files more than once (e.g., replacing
  FOO with FOO.gz.gz) on file systems such as ZFS where a readdir
  loop that unlinks and creates files can revisit output files.


* Noteworthy changes in release 1.4 (2010-01-20) [stable]

** Bug fixes

  gzip -d could segfault and/or clobber the stack, possibly leading to
  arbitrary code execution.  This affects x86_64 but not 32-bit systems.
  This fixes CVE-2010-0001.
  For more details, see https://bugzilla.redhat.com/554418

  gzip -d would fail with a CRC error for some valid inputs.
  So far, the only valid input known to exhibit this failure was
  compressed "from FAT filesystem (MS-DOS, OS/2, NT)".  In addition,
  to trigger the failure, your memcpy implementation must copy in
  the "reverse" order.


* Noteworthy changes in release 1.3.14 (2009-10-30) [beta]

** Bug fixes

  gzip no longer fails when there is exactly one trailing NUL byte
  gzip has always accepted trailing NUL bytes.  Note the plural.

  zdiff would exit with status 2 (indicating an error) rather than 1 to
  indicate differences when both inputs were compressed and different.

  zdiff would fail to print differences in two compressed inputs

  zgrep -f - didn't work


* Noteworthy changes in release 1.3.13 (2009-09-30) [stable]

** 'gzip -f foo.gz' now creates a file foo.gz.gz instead of complaining.

** Bug fixes

  gzip -d no longer fails with "-" as 2nd or subsequent argument

  gzip no longer ignores a close-induced write failure, e.g., on NFS

  gzip -d no longer segfaults on certain invalid inputs


Major changes in Gzip 1.3.12 (2007-04-13)

* znew now uses $TMPDIR (default /tmp) instead of always using /tmp.

* It is now documented that gzip ignores case when examining file name
  extensions; for example, 'gzip test.Gz' (without -f) fails because
  the file name ends in '.Gz'.

Major changes in Gzip 1.3.11 (2007-02-05)

* As per the GNU coding standards, the behavior of gzip and its
  companion executables no longer depend on the name used to invoke them.
  For example, 'gzip' and 'gunzip' are no longer hard links;
  instead, 'gunzip' is now a small program that invokes 'gzip -d'.

* zdiff now checks for subsidiary gzip failures, and works around
  bugs in IRIX 6 sh, Tru64 4.0F ksh, and Solaris 8 bash.

Major changes in Gzip 1.3.10 (2006-12-30)

* gzip -c and zcat now work on special files, files with special mode bits,
  and files with multiple hard links.
* gzip -q now exits with status 2 (not 1) when SIGPIPE is received.
* zcmp and zdiff did not work in the usual case, due to a typo.
* zgrep has many bugs fixed with argument handling, special characters,
  and exit status.
* zless no longer mishandles $%=~ in file names.

Gzip 1.3.9 (2006-12-15)

* No major changes; only porting fixes.

Major changes in Gzip 1.3.8 (2006-12-08)

* Fix some gzip problems:
  - A security fix from Debian 1.3.5-5 was inadvertently omitted.
  - The assembler is now invoked with --noexecstack if supported,
    so that gzip can better resist stack-smashing attacks.

Major changes in Gzip 1.3.7 (2006-12-06)

* Fix some gzip problems:
  - Refuse to compress setuid or setgid files, or files with the sticky bit.
  - Fix more race conditions in setting file permissions and owner,
    removing output files, following symbolic links, and dealing with
    special files.
  - Remove most of the code working around ENAMETOOLONG deficiencies.
    Systems with those deficiencies are long-dead, and the workarounds
    had race conditions on modern hosts.
  - Catch CPU time and file size limit signals, too.
  - Check for read errors when closing files.
  - Fix a core dump caused by a stray abort mistakenly introduced in 1.3.6.
* Fix some gzexe problems:
  - Improve resistance to denial-of-service attacks.
  - Fix some quoting and escaping bugs.
  - Do not assume /tmp is sticky (though it should be!).
  - Do not assume the working directory can be written.
  - Rely on PATH in the generated executable, as the man page says.
  - Don't assume IFS is sane.
  - Exit with signal's status, if signaled.

Major changes in Gzip 1.3.6 (2006-11-20)

* Fix some race conditions in setting file timestamps, permissions, and owner.
* Fix some race conditions in signal handling.
* When gzip exits due to a signal, it exits with the signal's status, not 1.
* gzip now restores file timestamps to the resolution supported by the
  time-setting primitives of the operating system, typically 1 microsecond.
  Formerly it restored them only to the nearest second.
* gzip -r no longer attempts to reset the last-access times of directories
  it reads, as this messes up when other processes are reading the directories.
* The options --version and --help now work on all gzip-installed executables,
  and now use a format similar to other GNU programs.
* The manual is now distributed under the terms of the GNU Free
  Documentation License without invariant sections or cover texts.
* Port to current versions of Autoconf, Automake, and Gnulib.

Major changes from 1.3.4 to 1.3.5
* gzip now removes any output symlink before writing output file.
* zgrep etc. scripts now port to POSIX 1003.1-2001 hosts.
* zforce no longer assumes 14-byte file name length limit.
* zless is now implemented using less and LESSOPEN, not zmore and PAGER.
* assembly-language speedups reenabled; were mistakenly disabled in 1.3.

Major changes from 1.3.3 to 1.3.4
* Less output is lost when decompressing a truncated file.
* The manual is now distributed under the terms of the GNU Free
  Documentation License.

No major changes in 1.3.2 or 1.3.3 (bug fixes only)

Major changes from 1.3 to 1.3.1
* zgrep now supports --, -H, -h, -L, -l, -C, -d, -m and their long equivalents.

Major changes from 1.2.4 to 1.3
* Add support for large files, e.g. files larger than 2 GB on Solaris 2.6.
* Adjust file size listing format for files larger than 10 GB.
* New command `zless'.
* `zdiff' now reports exit status like `diff' does.
* `zcat' is now always called `zcat', not `gzcat'.
  Similarly for `zdiff', `zgrep', `zmore', `znew', `zforce'.
* Warn about a compressed file's trailing zeros only if verbose,
  for compatibility with recent versions of GNU tar.
* Conform to changes to GNU makefile standards.
* Port to Autoconf 2.13.
* Convert to Automake.
* Fix bugs in the following areas:
  - files larger than 4 GB
  - security hole involving symlinks from /tmp
  - security hole involving long file names
  - permissions bug when compressing a symbolic link to a file
  - core dumps
  - concatenated compressed files on INBUFSIZ boundaries
  - porting bugs on hosts with signed chars
  - porting bug with upper and lower case
  - porting bug for hosts that reserve the names `basename' or `warning'

Major changes from 1.2.3 to 1.2.4
* By default, do not restore file name and timestamp from those saved
  inside the .gz file (behave as 'compress'). Added the --name option
  to force name and timestamp restoration.
* Accept - as synonym for stdin.
* Use manlinks=so or ln to support either hard links or .so in man pages
* Accept foo.gz~ in zdiff.
* Added support for Windows NT
* Handle ENAMETOOLONG for strict Posix systems
* Use --recursive instead of --recurse to comply with Webster and
  the GNU stdandard.
* Allow installation of shell scripts with a g prefix: make G=g install
* Install by default zcat as gzcat if gzcat already exists in path.
* Let zmore behave as more when invoked without parameters (give help)
* Let gzip --list reject files not in gzip format even with --force.
* Don't complain about non gzip files for options -rt or -rl.
* Added advice in INSTALL for several systems.

Major changes from 1.2.2 to 1.2.3
* Don't display the output name when decompressing except with --verbose.
* Remove usage of alloca in getopt.c and all makefiles.
* Added the zfile shell script in subdirectory sample.
* Moved the list of compiler bugs from README to INSTALL.
* Added vms/Readme.vms.

Major changes from 1.2.1 to 1.2.2
* Fix a compilation error on Sun with cc (worked with gcc).

Major changes from 1.2 to 1.2.1
* Let zmore act as more if the data is not gzipped.
* made gzexe more secure (don't rely on PATH).
* By default, display output name only when the name was actually truncated.

Major changes from 1.1.2 to 1.2
* Added the --list option to display the file characteristics.
* Added the --no-name option: do not save or restore original filename
  Save the original name by default.
* Allow gunzip --suffix "" to attempt decompression on any file
  regardless of its extension if an original name is present.
* Add support for the SCO compress -H format.
* gzip --fast now compresses faster (speed close to that of compress)
  with degraded compression ratio (but still better than compress).
  Default level changed to -6 (acts exactly as previous level -5) to
  be a better indication of its placement in the speed/ratio range.
* Use smart name truncation: 123456789012.c -> 123456789.c.gz
   instead of 12345678901.gz
* With --force, let zcat pass non gzip'ed data unchanged (zcat == cat)
* Added the zgrep shell script.
* Made sub.c useful for 16 bit sound, 24 bit images, etc..
* Suppress warnings about suffix for gunzip -r, except with --verbose.
* On MSDOS, use .gz extension when possible (files without extension)
* Moved the sample programs to a subdirectory sample.
* Added a "Special targets" section in INSTALL.

Major changes from 1.1.1 to 1.1.2.
* Fix serious bug for VMS (-gz not removed when decompressing).
* Allow suffix other than .gz in znew.
* Do not display compression ratio when decompressing stdin.
* In zmore.in, work around brain damaged stty -g (Ultrix).
* Display a correct compression ratio for .Z files.
* Added .z to .gz renaming script in INTALL.
* Allow setting CFLAGS in configure.

Major changes from 1.1 to 1.1.1.
* Fix serious bug in vms.c (affects Vax/VMS only).
* Added --ascii option.
* Add workaround in configure.in for Ultrix (quote eval argument)

Major changes from 1.0.7 to 1.1.
* Use .gz suffix by default, add --suffix option.
* Let gunzip accept a "_z" suffix (used by one 'compress' on Vax/VMS).
* Quit when reading garbage from stdin instead of reporting an error.
* Added makefile for VAX/MMS and support for wildcards on VMS.
* Added support for MSC under OS/2.
* Added support for Prime/PRIMOS.
* Display compression ratio also when decompressing (with --verbose).
* Quit after --version (GNU standard)
* Use --force to bypass isatty() check
* Continue processing other files in case of recoverable error.
* Added comparison of zip and gzip in the readme file.
* Added small sample programs (ztouch, sub, add)
* Use less memory when compiled with -DSMALL_MEM (for MSDOS).
* Remove the "off by more than one minute" timestamp kludge

Major changes from 1.0.6 to 1.0.7.
* Allow zmore to read from standard input (like more).
* Support the 68000 (Atari ST) in match.S.
* Retry partial writes (required on Linux when gzip is suspended in a pipe).
* Allow full pathnames and renamings in gzexe.
* Don't let gzexe compress setuid executables or gzip itself.
* Added vms/Makefile.gcc for gcc on the Vax.
* Allow installation of binaries and shell scripts in different dirs.
* Allows complex PAGER variable in zmore (e.g.: PAGER="col -x | more")
* Allow installation of zcat as gzcat.
* Several small changes for portability to old or weird systems.
* Suppress help message and send compressed data to the terminal when
  gzip is invoked without parameters and without redirection.
*  Add compile option GNU_STANDARD to respect the GNU coding standards:
   with -DGNU_STANDARD, behave as gzip even if invoked under the name gunzip.
(I don't like the last two changes, which were requested by the FSF.)

Major changes from 1.0.5 to 1.0.6.
* Let gzexe detect executables that are already gzexe'd.
* Keep file attributes in znew and gzexe if cpmod is available.
* Don't try restoring record format on VMS (1.0.5 did not work correctly)
* Added asm version for 68000 in amiga/match.a.
  Use asm version for Atari TT and NeXT.
* For OS/2, assume HPFS by default, add flag OS2FAT if necessary.
* Fixed some bugs in zdiff and define zcmp as a link to zdiff.


Major changes from 1.0.4 to 1.0.5.
* For VMS, restore the file type for variable record format, otherwise
    extract in fixed length format (not perfect, but better than
    forcing all files to be in stream_LF format).
* For VMS, use "-z" default suffix and accept a version number in file names.
* For Unix, allow compression of files with name ending in 'z'. Use only
  .z, .*-z, .tgz, .taz as valid gzip extensions. In the last two cases,
  extract to .tar by default.
* On some versions of MSDOS, files with a 3 character extension could not
  be compressed.
* Garbage collect files in /tmp created by gzexe.
* Fix the 'OS code' byte in the gzip header.
* For the Amiga, add the missing utime.h and add support for gcc.


Major changes from 1.0.3 to 1.0.4.
* Added optimized asm version for 68020.
* Add support for DJGPP.

* Add support for the Atari ST.
* Added zforce to rename gzip'ed files with truncated names.
* Do not install with name uncompress (some systems rely on the
  absence of any check in the old uncompress).
* Added missing function (fcfree) in msdos/tailor.c
* Let gunzip handle .tgz files, and let gzip skip them.
* Added -d option (decompress) for gzexe and try preserving file permissions.
* Suppress all warnings with -q.
* Use GZIP_OPT for VMS to avoid conflict with program name.
* ... and many other small changes (see ChangeLog)


Major changes from 1.0.2 to 1.0.3
* Added -K option for znew to keep old .Z files if smaller
* Added -q option (quiet) to cancel -v in GZIP env variable.
* Made gzexe safer on systems with filename limitation to 14 chars.
* Fixed bugs in handling of GZIP env variable and incorrect free with Turbo C.


Major changes from 1.0.1 to 1.0.2
* Added env variable GZIP for default options. Example:
   for sh:   GZIP="-8 -v"; export GZIP
   for csh:  setenv GZIP "-8 -v"
* Added support for the Amiga.
* znew now keeps the old .Z if it is smaller than the .z file.
  This can happen for some large and very redundant files.
* Do not complain about trailing garbage for record oriented IO (Vax/VMS).
  This implies however that multi-part gzip files are not supported
  on such systems.
* Added gzexe to compress rarely used executables.
* Reduce memory usage (required for MSDOS and useful on all systems).
* Preserve timestamp in znew -P (pipe option) if touch -r works.


Major changes from 1.0 to 1.0.1
* fix trivial errors in the Borland makefile (msdos/Makefile.bor)


Major changes from 0.8.2 to 1.0
* gzip now runs on Vax/VMS
* gzip will not not overwrite files without -f when using /bin/sh in
  background.
* Support the test option -t for compressed (.Z) files.
  Allow some data recovery for bad .Z files.
* Added makefiles for MSDOS (Only tested for MSC, not Borland).
* still more changes to configure for several systems


Major changes from 0.8.1 to 0.8.2:
* yet more changes to configure for Linux and other systems
* Allow zcat on a file with multiple links.


Major changes from 0.8 to 0.8.1:
* znew has now a pipe option -P to reduce the disk space requirements,
  but this option does not preserve timestamps.
* Fixed some #if directives for compilation with TurboC.


Major changes from 0.7 to 0.8:
* gzip can now extract .z files created by 'pack'.
* configure should no longer believe that every machine is a 386
* Fix the entry for /etc/magic in INSTALL.
* Add patch for GNU tar 1.11.1 and a pointer to crypt++.el
* Uncompress files with multiple links only with -f.
* Fix for uncompress of .Z files on 16-bit machines
* Create a correct output name for file names of exactly N-1 chars when
  the system has a limit of N chars.


Major changes from 0.6 to 0.7:
* Use "make check" instead of "make test".
* Keep timestamp and pass options to gzip in znew.
* Do not create .z.z files with gzip -r.
* Allow again gunzip .zip files (was working in 0.5)
* Allow again compilation with TurboC 2.0 (was working in 0.4)


Major changes form 0.5 to 0.6:
* gunzip reported an error when extracting certain .z files. The .z files
  produced by gzip 0.5 are correct and can be read by gunzip 0.6.
* gunzip now supports multiple compressed members within a single .z file.
* Fix the check for i386 in configure.
* Added "make test" to check for compiler bugs. (gcc -finline-functions
  is broken at least on the NeXT.)
* Use environment variable PAGER in zmore if it is defined.
* Accept gzcat in addition to zcat for people having /usr/bin before
  /usr/local/bin in their path.


========================================================================

Copyright (C) 1999, 2001-2002, 2006-2007, 2009-2022 Free Software Foundation,
Inc.
Copyright (C) 1992, 1993 Jean-loup Gailly

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with no
Invariant Sections, with no Front-Cover Texts, and with no Back-Cover
Texts.  A copy of the license is included in the ``GNU Free
Documentation License'' file as part of this distribution.