Zlib和Zstd 性能对比评测
Posted 流子
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Zlib和Zstd 性能对比评测相关的知识，希望对你有一定的参考价值。
Zstandard 是facebook厂商提供的一种高压缩率的实时压缩算法，他在压缩比和压缩速度上做了很好的平衡，同时还提供高速的解码器，针对小数据，也提供了一种叫做字典压缩算法，从样本中可以创建出字典集合数据，Zstandard库以BSD协议方式开源。
该算法是由Facebook的Yann Collet开发的一个无损数据压缩算法。Zstandard在设计上与DEFLATE（.zip、gzip）算法有着差不多的压缩比，但有更高的压缩和解压缩速度。
Zstandard使用字典算法（LZ77）结合熵编码法的有限状态熵（tANS）。—–Wikipedia
Zlib 就不介绍了，大家都很熟悉了，一套古老的压缩算法。性能对比，两年前做过简单对比，Zstandard （zstd）压缩算法在JAVA上的使用
这次再做一次深入的评测和对比，其中，实验样本用的是我们游戏协议的文本样本，测试结果见文章结尾：
和zlib对比
1.在小数据量下（256），level=3,相对zlib,!!#ff0000 压缩效率大幅提高!!(94230.272 -> 153682 , 163.09%)，表现更加稳定(± 7958.782 ops/s–>±1801.146 ops/s)，解压缩效率小幅降低（608231.351 -> 565699.876 , -7%），并且表现更加稳定± 312460.294 ops/s -->± 12919.602 ops/s)， 但压缩比会小幅降低（18.0%–>17.97%）
2.在大数据量下（8192），level=3,相对zlib!!#ff0000 ,压缩效率大幅提高!!(3939.309-> 29897.221 ,758.99%)，稳定性小幅降低(45.24/3939.309=1.14%->513.773/ 29897.221 =1.71%)，解压缩效率大幅提高（30828.328 -> 82746.561 , 268.41%），稳定性小幅提高(（4388.578/30828.328=14.23%–>8866.313/ 82746.561=10.71%)， 压缩比基本持平（ 63.5%–>63.48%）
总结 1，从数据上看出，zstandard比zlib,同样的压缩比情况下，速度高于zlib,同样的速度情况下，压缩比要高于zlib
自身不同level 对比
3.小数据量下(256 bytes)，随着level增大，压缩率会小幅提高，提升不明显，在level>=2 下 ，!!#ff0000 压缩效率会大幅降低，!!解压缩效率也会小幅降低。
压缩（level=3）
其中,ratio=1-compress/size
Benchmark (level) (size) Mode Cnt Score Error Units compress ratio
Compress.ZstdLevelBenchMark.zstdCompress 1 256 thrpt 10 158247.321 2539.721 ops/s 219 14.45%
Compress.ZstdLevelBenchMark.zstdCompress 2 256 thrpt 10 160513.122 1055.204 ops/s 217 15.23%
Compress.ZstdLevelBenchMark.zstdCompress 3 256 thrpt 10 153682.210 1801.146 ops/s 210 17.97%
Compress.ZstdLevelBenchMark.zstdCompress 4 256 thrpt 10 133600.466 1974.326 ops/s 206 19.53%
Compress.ZstdLevelBenchMark.zstdCompress 5 256 thrpt 10 124519.999 3317.794 ops/s 204 20.31%
Compress.ZstdLevelBenchMark.zstdCompress 6 256 thrpt 10 122500.287 1030.539 ops/s 204 20.31%
解压（level=3）
Benchmark (level) (size) Mode Cnt Score Error Units
Compress.ZstdLevelBenchMark.zstdUncompress 1 256 thrpt 10 582084.385 7778.949 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 2 256 thrpt 10 580312.371 6809.382 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 3 256 thrpt 10 565699.876 12919.602 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 4 256 thrpt 10 562818.616 5036.532 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 5 256 thrpt 10 501616.311 5596.259 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 6 256 thrpt 10 494291.843 7102.794 ops/s
4.大数据量下(8192 bytes)，随着level增大，压缩率会小幅提高，提升不明显，!!#ff0000 压缩效率会大幅降低!!，解压缩效率会小幅提高
压缩（level=6）
Benchmark (level) (size) Mode Cnt Score Error Units compress ratio
Compress.ZstdLevelBenchMark.zstdCompress 1 8192 thrpt 10 34586.557 302.868 ops/s 3091 62.27%
Compress.ZstdLevelBenchMark.zstdCompress 2 8192 thrpt 10 34113.274 243.294 ops/s 3035 62.95%
Compress.ZstdLevelBenchMark.zstdCompress 3 8192 thrpt 10 29897.221 513.773 ops/s 2992 63.48%
Compress.ZstdLevelBenchMark.zstdCompress 4 8192 thrpt 10 18699.637 176.416 ops/s 2948 64.01%
Compress.ZstdLevelBenchMark.zstdCompress 5 8192 thrpt 10 13744.149 76.323 ops/s 2885 64.78%
Compress.ZstdLevelBenchMark.zstdCompress 6 8192 thrpt 10 9931.221 79.205 ops/s 2886 64.77%
解压（level=6）
Benchmark (level) (size) Mode Cnt Score Error Units
Compress.ZstdLevelBenchMark.zstdUncompress 1 8192 thrpt 10 83840.509 9960.244 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 2 8192 thrpt 10 83274.771 6297.771 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 3 8192 thrpt 10 82746.561 8866.313 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 4 8192 thrpt 10 84652.976 6150.251 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 5 8192 thrpt 10 84665.619 6465.654 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 6 8192 thrpt 10 85139.016 5582.793 ops/s
总结 2，从以上3，4点对比，默认值3 ，或者 2 可以认为是相对较优解。
综上所述，
在协议这块，服务器用压缩API的频率远大于解压缩API频率， 在由zlib替换了level=3的Zstd后，压缩效率会大幅提高，表现也更加稳定，压缩率会有小幅降低。
在level的调整方面，随着level增大，压缩率会有小幅提高，但压缩效率会大幅降低，得不偿失，所以保持默认值3或者2是比较合适的。如果追求压缩/解压缩效率则选择2，追求压缩比，则选择3
##测试结果
zlib
Benchmark (size) Mode Cnt Score Error Units compress ratio
Compress.ZstdLevelBenchMark.zlibCompress 256 thrpt 10 94230.272 ± 7958.782 ops/s 197 18.0%
Compress.ZstdLevelBenchMark.zlibCompress 512 thrpt 10 67053.071 ± 5606.840 ops/s 308 39.3%
Compress.ZstdLevelBenchMark.zlibCompress 1024 thrpt 10 32361.968 ± 1426.980 ops/s 448 58.5%
Compress.ZstdLevelBenchMark.zlibCompress 2048 thrpt 10 15301.493 ± 354.073 ops/s 733 64.3%
Compress.ZstdLevelBenchMark.zlibCompress 4096 thrpt 10 6656.437 ± 96.692 ops/s 1359 68.4%
Compress.ZstdLevelBenchMark.zlibCompress 8192 thrpt 10 3939.309 ± 45.240 ops/s 3126 63.5%
Compress.ZstdLevelBenchMark.zlibUncompress 256 thrpt 10 608231.351 ± 312460.294 ops/s
Compress.ZstdLevelBenchMark.zlibUncompress 512 thrpt 10 460449.016 ± 206413.902 ops/s
Compress.ZstdLevelBenchMark.zlibUncompress 1024 thrpt 10 169738.557 ± 33999.682 ops/s
Compress.ZstdLevelBenchMark.zlibUncompress 2048 thrpt 10 110027.019 ± 17135.330 ops/s
Compress.ZstdLevelBenchMark.zlibUncompress 4096 thrpt 10 65892.491 ± 11272.236 ops/s
Compress.ZstdLevelBenchMark.zlibUncompress 8192 thrpt 10 30828.328 ± 4388.578 ops/s
Zstd
Benchmark (level) (size) Mode Cnt Score Error Units compress ratio
Compress.ZstdLevelBenchMark.zstdCompress 1 256 thrpt 10 158247.321 2539.721 ops/s 219 14.45%
Compress.ZstdLevelBenchMark.zstdCompress 1 512 thrpt 10 138946.669 707.322 ops/s 314 38.67%
Compress.ZstdLevelBenchMark.zstdCompress 1 1024 thrpt 10 116642.039 1013.260 ops/s 443 56.74%
Compress.ZstdLevelBenchMark.zstdCompress 1 2048 thrpt 10 84307.054 1845.538 ops/s 738 63.96%
Compress.ZstdLevelBenchMark.zstdCompress 1 4096 thrpt 10 57226.220 277.580 ops/s 1304 68.16%
Compress.ZstdLevelBenchMark.zstdCompress 1 8192 thrpt 10 34586.557 302.868 ops/s 3091 62.27%
Compress.ZstdLevelBenchMark.zstdCompress 2 256 thrpt 10 160513.122 1055.204 ops/s 217 15.23%
Compress.ZstdLevelBenchMark.zstdCompress 2 512 thrpt 10 139264.767 1479.608 ops/s 311 39.26%
Compress.ZstdLevelBenchMark.zstdCompress 2 1024 thrpt 10 114781.873 1121.017 ops/s 411 59.86%
Compress.ZstdLevelBenchMark.zstdCompress 2 2048 thrpt 10 82449.922 681.039 ops/s 704 65.63%
Compress.ZstdLevelBenchMark.zstdCompress 2 4096 thrpt 10 55644.063 749.733 ops/s 1329 67.55%
Compress.ZstdLevelBenchMark.zstdCompress 2 8192 thrpt 10 34113.274 243.294 ops/s 3035 62.95%
Compress.ZstdLevelBenchMark.zstdCompress 3 256 thrpt 10 153682.210 1801.146 ops/s 210 17.97%
Compress.ZstdLevelBenchMark.zstdCompress 3 512 thrpt 10 132749.271 1685.915 ops/s 311 39.26%
Compress.ZstdLevelBenchMark.zstdCompress 3 1024 thrpt 10 109401.965 1214.010 ops/s 425 58.50%
Compress.ZstdLevelBenchMark.zstdCompress 3 2048 thrpt 10 81395.267 1234.244 ops/s 732 64.26%
Compress.ZstdLevelBenchMark.zstdCompress 3 4096 thrpt 10 52683.335 485.770 ops/s 1295 68.38%
Compress.ZstdLevelBenchMark.zstdCompress 3 8192 thrpt 10 29897.221 513.773 ops/s 2992 63.48%
Compress.ZstdLevelBenchMark.zstdCompress 4 256 thrpt 10 133600.466 1974.326 ops/s 206 19.53%
Compress.ZstdLevelBenchMark.zstdCompress 4 512 thrpt 10 112088.964 4874.136 ops/s 307 40.04%
Compress.ZstdLevelBenchMark.zstdCompress 4 1024 thrpt 10 88978.838 1492.176 ops/s 420 58.98%
Compress.ZstdLevelBenchMark.zstdCompress 4 2048 thrpt 10 62289.020 1805.546 ops/s 724 64.65%
Compress.ZstdLevelBenchMark.zstdCompress 4 4096 thrpt 10 38117.885 496.076 ops/s 1288 68.55%
Compress.ZstdLevelBenchMark.zstdCompress 4 8192 thrpt 10 18699.637 176.416 ops/s 2948 64.01%
Compress.ZstdLevelBenchMark.zstdCompress 5 256 thrpt 10 124519.999 3317.794 ops/s 204 20.31%
Compress.ZstdLevelBenchMark.zstdCompress 5 512 thrpt 10 101343.380 1078.978 ops/s 293 42.77%
Compress.ZstdLevelBenchMark.zstdCompress 5 1024 thrpt 10 75868.554 405.721 ops/s 420 58.98%
Compress.ZstdLevelBenchMark.zstdCompress 5 2048 thrpt 10 50464.417 497.478 ops/s 714 65.14%
Compress.ZstdLevelBenchMark.zstdCompress 5 4096 thrpt 10 29152.233 320.349 ops/s 1243 69.65%
Compress.ZstdLevelBenchMark.zstdCompress 5 8192 thrpt 10 13744.149 76.323 ops/s 2885 64.78%
Compress.ZstdLevelBenchMark.zstdCompress 6 256 thrpt 10 122500.287 1030.539 ops/s 204 20.31%
Compress.ZstdLevelBenchMark.zstdCompress 6 512 thrpt 10 96711.036 965.084 ops/s 293 42.77%
Compress.ZstdLevelBenchMark.zstdCompress 6 1024 thrpt 10 63150.268 2078.721 ops/s 420 58.98%
Compress.ZstdLevelBenchMark.zstdCompress 6 2048 thrpt 10 38628.981 669.073 ops/s 714 65.14%
Compress.ZstdLevelBenchMark.zstdCompress 6 4096 thrpt 10 19968.703 271.908 ops/s 1240 69.73%
Compress.ZstdLevelBenchMark.zstdCompress 6 8192 thrpt 10 9931.221 79.205 ops/s 2886 64.77%
Compress.ZstdLevelBenchMark.zstdUncompress 1 256 thrpt 10 582084.385 7778.949 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 1 512 thrpt 10 480672.486 2697.970 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 1 1024 thrpt 10 361403.668 3443.690 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 1 2048 thrpt 10 189317.885 1558.473 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 1 4096 thrpt 10 134702.624 1711.808 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 1 8192 thrpt 10 83840.509 9960.244 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 2 256 thrpt 10 580312.371 6809.382 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 2 512 thrpt 10 487137.103 3394.255 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 2 1024 thrpt 10 364243.783 3039.647 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 2 2048 thrpt 10 266417.312 11819.619 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 2 4096 thrpt 10 128809.854 5231.345 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 2 8192 thrpt 10 83274.771 6297.771 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 3 256 thrpt 10 565699.876 12919.602 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 3 512 thrpt 10 484201.159 8496.238 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 3 1024 thrpt 10 351868.427 6034.207 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 3 2048 thrpt 10 264361.085 13951.503 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 3 4096 thrpt 10 134640.265 1068.485 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 3 8192 thrpt 10 82746.561 8866.313 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 4 256 thrpt 10 562818.616 5036.532 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 4 512 thrpt 10 461458.897 5798.547 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 4 1024 thrpt 10 349583.096 2818.184 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 4 2048 thrpt 10 265175.187 10967.808 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 4 4096 thrpt 10 132574.256 1409.214 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 4 8192 thrpt 10 84652.976 6150.251 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 5 256 thrpt 10 501616.311 5596.259 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 5 512 thrpt 10 419099.670 1799.619 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 5 1024 thrpt 10 353087.876 6711.421 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 5 2048 thrpt 10 264624.591 2530.357 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 5 4096 thrpt 10 133355.669 1524.036 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 5 8192 thrpt 10 84665.619 6465.654 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 6 256 thrpt 10 494291.843 7102.794 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 6 512 thrpt 10 418350.498 2466.316 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 6 1024 thrpt 10 355302.788 2142.318 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 6 2048 thrpt 10 265958.624 11417.774 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 6 4096 thrpt 10 131764.090 6460.557 ops/s
Compress.ZstdLevelBenchMark.zstdUncompress 6 8192 thrpt 10 85139.016 5582.793 ops/s
以上是关于Zlib和Zstd 性能对比评测的主要内容，如果未能解决你的问题，请参考以下文章
快如闪电：Linux多线程压缩软件pigz和压缩神器zstd