HBASE-26353 Support loadable dictionaries in hbase-compression-zstd (#3787)
commit45f76a410434a61e36dc86d83abfcbc3976e32b8
authorAndrew Purtell <apurtell@apache.org>
Wed, 27 Oct 2021 14:39:55 +0000 (27 07:39 -0700)
committerGitHub <noreply@github.com>
Wed, 27 Oct 2021 14:39:55 +0000 (27 07:39 -0700)
tree67ef124bd10166d779e0100382adcca6e8ab3896
parenta5a349f289dd9b579c3026224186b6ea82dbd9b5
HBASE-26353 Support loadable dictionaries in hbase-compression-zstd (#3787)

ZStandard supports initialization of compressors and decompressors with a
precomputed dictionary, which can dramatically improve and speed up compression
of tables with small values. For more details, please see

  The Case For Small Data Compression
  https://github.com/facebook/zstd#the-case-for-small-data-compression

Signed-off-by: Duo Zhang <zhangduo@apache.org>
15 files changed:
hbase-common/src/main/java/org/apache/hadoop/hbase/io/compress/CompressionUtil.java
hbase-common/src/main/java/org/apache/hadoop/hbase/io/compress/DictionaryCache.java [new file with mode: 0644]
hbase-common/src/test/java/org/apache/hadoop/hbase/io/compress/CompressionTestBase.java
hbase-compression/hbase-compression-zstd/src/main/java/org/apache/hadoop/hbase/io/compress/zstd/ZstdCodec.java
hbase-compression/hbase-compression-zstd/src/main/java/org/apache/hadoop/hbase/io/compress/zstd/ZstdCompressor.java
hbase-compression/hbase-compression-zstd/src/main/java/org/apache/hadoop/hbase/io/compress/zstd/ZstdDecompressor.java
hbase-compression/hbase-compression-zstd/src/test/java/org/apache/hadoop/hbase/io/compress/zstd/TestZstdCodec.java
hbase-compression/hbase-compression-zstd/src/test/java/org/apache/hadoop/hbase/io/compress/zstd/TestZstdDictionary.java [new file with mode: 0644]
hbase-compression/hbase-compression-zstd/src/test/java/org/apache/hadoop/hbase/io/compress/zstd/TestZstdDictionarySplitMerge.java [new file with mode: 0644]
hbase-compression/hbase-compression-zstd/src/test/resources/zstd.test.data [new file with mode: 0644]
hbase-compression/hbase-compression-zstd/src/test/resources/zstd.test.dict [new file with mode: 0644]
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/MergeTableRegionsProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/SplitTableRegionProcedure.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileInfo.java