HBASE-26353 Support loadable dictionaries in hbase-compression-zstd (#3748)
commitbfa45841251e3224e3fbb07599ae5dd6791b2846
authorAndrew Purtell <apurtell@apache.org>
Tue, 19 Oct 2021 20:37:24 +0000 (19 13:37 -0700)
committerGitHub <noreply@github.com>
Tue, 19 Oct 2021 20:37:24 +0000 (19 13:37 -0700)
tree1c91c8b50e043d063d05f9647e7dd2986d416218
parent8a6fed7b1982aa35cf0290c64a1c37489e03c735
HBASE-26353 Support loadable dictionaries in hbase-compression-zstd (#3748)

ZStandard supports initialization of compressors and decompressors with a
precomputed dictionary, which can dramatically improve and speed up compression
of tables with small values. For more details, please see

  The Case For Small Data Compression
  https://github.com/facebook/zstd#the-case-for-small-data-compression

Signed-off-by: Duo Zhang <zhangduo@apache.org>
hbase-common/src/main/java/org/apache/hadoop/hbase/io/compress/CompressionUtil.java
hbase-common/src/main/java/org/apache/hadoop/hbase/io/compress/DictionaryCache.java [new file with mode: 0644]
hbase-common/src/test/java/org/apache/hadoop/hbase/io/compress/CompressionTestBase.java
hbase-compression/hbase-compression-zstd/src/main/java/org/apache/hadoop/hbase/io/compress/zstd/ZstdCodec.java
hbase-compression/hbase-compression-zstd/src/main/java/org/apache/hadoop/hbase/io/compress/zstd/ZstdCompressor.java
hbase-compression/hbase-compression-zstd/src/main/java/org/apache/hadoop/hbase/io/compress/zstd/ZstdDecompressor.java
hbase-compression/hbase-compression-zstd/src/test/java/org/apache/hadoop/hbase/io/compress/zstd/TestZstdCodec.java
hbase-compression/hbase-compression-zstd/src/test/resources/zstd.test.data [new file with mode: 0644]
hbase-compression/hbase-compression-zstd/src/test/resources/zstd.test.dict [new file with mode: 0644]