Skip to content

Commit f5c9ba6

Browse files
Aleksey Vaneevinikep
authored andcommitted
Update lzav to v5.7
1 parent d1f02de commit f5c9ba6

File tree

4 files changed

+104
-81
lines changed

4 files changed

+104
-81
lines changed

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ Supported compressors
6666
- [libdeflate v1.24](https://github.com/ebiggers/libdeflate)
6767
- [lizard v2.1](https://github.com/inikep/lizard)
6868
- [lz4/lz4hc v1.10.0](https://github.com/lz4/lz4)
69-
- [lzav 5.5](https://github.com/avaneev/lzav)
69+
- [lzav 5.7](https://github.com/avaneev/lzav)
7070
- [lzf 3.6](http://software.schmorp.de/pkg/liblzf.html)
7171
- [lzfse/lzvn 1.0](https://github.com/lzfse/lzfse)
7272
- [lzg 1.0.10](https://github.com/mbitsnbites/liblzg)

bench/lzbench.h

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -203,7 +203,7 @@ static const compressor_desc_t comp_desc[] =
203203
{ "lz4", "lz4 1.10.0", 0, 0, 0, BENCH_POOL_MT, lzbench_lz4_compress, lzbench_lz4_decompress, NULL, NULL },
204204
{ "lz4fast", "lz4 1.10.0 --fast", 1, 99, 0, BENCH_POOL_MT, lzbench_lz4fast_compress, lzbench_lz4_decompress, NULL, NULL },
205205
{ "lz4hc", "lz4hc 1.10.0", 1, 12, 0, BENCH_POOL_MT, lzbench_lz4hc_compress, lzbench_lz4_decompress, NULL, NULL },
206-
{ "lzav", "lzav 5.5", 1, 2, 0, BENCH_POOL_MT, lzbench_lzav_compress, lzbench_lzav_decompress, NULL, NULL },
206+
{ "lzav", "lzav 5.7", 1, 2, 0, BENCH_POOL_MT, lzbench_lzav_compress, lzbench_lzav_decompress, NULL, NULL },
207207
{ "lzf", "lzf 3.6", 0, 1, 0, BENCH_POOL_MT, lzbench_lzf_compress, lzbench_lzf_decompress, NULL, NULL },
208208
{ "lzfse", "lzfse 2017-03-08", 0, 0, 0, BENCH_POOL_MT, lzbench_lzfse_compress, lzbench_lzfse_decompress, lzbench_lzfse_init, lzbench_lzfse_deinit },
209209
{ "lzg", "lzg 1.0.10", 1, 9, 0, BENCH_POOL_MT, lzbench_lzg_compress, lzbench_lzg_decompress, NULL, NULL },

lz/lzav/README.md

Lines changed: 36 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,12 @@ and AArch64 (Apple Silicon) architectures; Windows 10, AlmaLinux 9.3, and
8282
macOS 15.7. Full C++ compliance is enabled conditionally and automatically
8383
when the source code is compiled with a C++ compiler.
8484

85+
## Ports
86+
87+
* [C++, vcpkg](https://vcpkg.link/ports/lzav)
88+
* [FreeArc, by Shegorat](https://krinkels.org/resources/cls-lzav.579/)
89+
* [Rust, by pkolaczk](https://crates.io/crates/lzav)
90+
8591
## Customizing C++ namespace
8692

8793
In C++ environments where it is undesirable to export LZAV symbols into the
@@ -151,11 +157,11 @@ Silesia compression corpus
151157

152158
| Compressor | Compression | Decompression | Ratio % |
153159
|-----------------|-------------|---------------|---------|
154-
| **LZAV 5.5** | 615 MB/s | 3770 MB/s | 40.23 |
160+
| **LZAV 5.7** | 623 MB/s | 3790 MB/s | 39.94 |
155161
| LZ4 1.9.4 | 700 MB/s | 4570 MB/s | 47.60 |
156162
| Snappy 1.1.10 | 495 MB/s | 3230 MB/s | 48.22 |
157163
| LZF 3.6 | 395 MB/s | 800 MB/s | 48.15 |
158-
| **LZAV 5.5 HI** | 133 MB/s | 3700 MB/s | 35.12 |
164+
| **LZAV 5.7 HI** | 134 MB/s | 3700 MB/s | 35.12 |
159165
| LZ4HC 1.9.4 -9 | 40 MB/s | 4360 MB/s | 36.75 |
160166

161167
### LLVM clang 19.1.7 x86-64, AlmaLinux 9.3, Xeon E-2386G (RocketLake), 5.1 GHz
@@ -164,11 +170,11 @@ Silesia compression corpus
164170

165171
| Compressor | Compression | Decompression | Ratio % |
166172
|-----------------|-------------|---------------|---------|
167-
| **LZAV 5.5** | 610 MB/s | 3400 MB/s | 40.23 |
173+
| **LZAV 5.7** | 614 MB/s | 3490 MB/s | 39.94 |
168174
| LZ4 1.9.4 | 848 MB/s | 4980 MB/s | 47.60 |
169175
| Snappy 1.1.10 | 690 MB/s | 3360 MB/s | 48.22 |
170176
| LZF 3.6 | 455 MB/s | 1000 MB/s | 48.15 |
171-
| **LZAV 5.5 HI** | 115 MB/s | 3340 MB/s | 35.12 |
177+
| **LZAV 5.7 HI** | 115 MB/s | 3350 MB/s | 35.12 |
172178
| LZ4HC 1.9.4 -9 | 43 MB/s | 4920 MB/s | 36.75 |
173179

174180
### LLVM clang-cl 18.1.8 x86-64, Windows 10, Ryzen 3700X (Zen2), 4.2 GHz
@@ -177,11 +183,11 @@ Silesia compression corpus
177183

178184
| Compressor | Compression | Decompression | Ratio % |
179185
|-----------------|-------------|---------------|---------|
180-
| **LZAV 5.5** | 520 MB/s | 2980 MB/s | 40.23 |
186+
| **LZAV 5.7** | 525 MB/s | 3060 MB/s | 39.94 |
181187
| LZ4 1.9.4 | 675 MB/s | 4560 MB/s | 47.60 |
182188
| Snappy 1.1.10 | 415 MB/s | 2440 MB/s | 48.22 |
183189
| LZF 3.6 | 310 MB/s | 700 MB/s | 48.15 |
184-
| **LZAV 5.5 HI** | 114 MB/s | 2980 MB/s | 35.12 |
190+
| **LZAV 5.7 HI** | 116 MB/s | 2970 MB/s | 35.12 |
185191
| LZ4HC 1.9.4 -9 | 36 MB/s | 4430 MB/s | 36.75 |
186192

187193
P.S. Popular Zstd's benchmark was not included here, because it is not a pure
@@ -195,6 +201,30 @@ on Silesia dataset:
195201
| zstd 1.5.5 -1 | 460 MB/s | 1870 MB/s | 41.0 |
196202
| zstd 1.5.5 1 | 436 MB/s | 1400 MB/s | 34.6 |
197203

204+
## Datasets Benchmark
205+
206+
This section presents compression ratio comparisons for various popular
207+
datasets. Note that each file within these datasets was compressed
208+
individually, which contributed to the overall ratio.
209+
210+
| Dataset | Size, MiB | LZAV 5.7 | LZ4 1.9.4 | Snappy 1.1.10 | LZF 3.6 | Source |
211+
|-----------------------|-----------|----------|-----------|---------------|---------|--------|
212+
| 4SICS 151020 PCAP | 24.5 | 20.47 | 21.82 | 24.95 | 25.34 | [www.netresec.com](https://www.netresec.com/?page=PCAP4SICS)|
213+
| 4SICS 151022 PCAP | 200.0 | 36.45 | 37.35 | 40.24 | 41.37 | [www.netresec.com](https://www.netresec.com/?page=PCAP4SICS)|
214+
| Calgary Large | 3.1 | 44.29 | 51.97 | 51.76 | 49.07 | [data-compression.info](https://www.data-compression.info/Corpora/CalgaryCorpus/) |
215+
| Canterbury | 2.68 | 38.07 | 43.73 | 45.42 | 42.49 | [corpus.canterbury.ac.nz](https://corpus.canterbury.ac.nz/) |
216+
| Canterbury Large | 10.6 | 38.25 | 51.97 | 48.37 | 54.28 | [corpus.canterbury.ac.nz](https://corpus.canterbury.ac.nz/) |
217+
| Canterbury Artificial | 0.29 | 33.36 | 33.74 | 36.48 | 34.66 | [corpus.canterbury.ac.nz](https://corpus.canterbury.ac.nz/) |
218+
| employees_10KB.json | 0.01 | 22.55 | 24.68 | 23.92 | 23.52 | [sample.json-format.com](https://sample.json-format.com/) |
219+
| employees_100KB.json | 0.10 | 15.96 | 17.71 | 19.02 | 21.88 | [sample.json-format.com](https://sample.json-format.com/) |
220+
| employees_50MB.json | 51.5 | 10.78 | 16.42 | 18.57 | 21.44 | [sample.json-format.com](https://sample.json-format.com/) |
221+
| enwik8 | 95.4 | 44.61 | 57.26 | 56.56 | 53.95 | [www.mattmahoney.net](https://www.mattmahoney.net/dc/textdata.html) |
222+
| enwik9 | 954.7 | 39.39 | 50.92 | 50.79 | 49.30 | [www.mattmahoney.net](https://www.mattmahoney.net/dc/textdata.html) |
223+
| Manzini | 855.3 | 26.98 | 37.30 | 38.57 | 39.04 | [people.unipmn.it/manzini](https://people.unipmn.it/manzini/boosting/index.html) |
224+
| chr22.dna (Manzini) | 33.0 | 38.79 | 52.82 | 44.53 | 55.86 | [people.unipmn.it/manzini](https://people.unipmn.it/manzini/boosting/index.html) |
225+
| w3c2 HTML (Manzini) | 99.4 | 11.43 | 22.20 | 25.35 | 27.20 | [people.unipmn.it/manzini](https://people.unipmn.it/manzini/boosting/index.html) |
226+
| Silesia | 202.1 | 39.94 | 47.60 | 48.17 | 48.15 | [github.com/MiloszKrajewski](https://github.com/MiloszKrajewski/SilesiaCorpus) |
227+
198228
## Notes
199229

200230
1. LZAV API is not equivalent to LZ4 or Snappy API. For example, the `dstlen`

0 commit comments

Comments
 (0)