Skip to content

Implement search-skip optimization framework for ZSTD compressors#4645

Open
BiplabRaut wants to merge 1 commit intofacebook:devfrom
amd:zstd_ss_opt
Open

Implement search-skip optimization framework for ZSTD compressors#4645
BiplabRaut wants to merge 1 commit intofacebook:devfrom
amd:zstd_ss_opt

Conversation

@BiplabRaut
Copy link
Copy Markdown

  • Add ZSTD_ENABLE_SEARCH_SKIP_HIGH macro to portability_macros.h for conditional compilation
  • Extend compression level parameters in clevels.h with search-skip configurations
  • Integrate adaptive search skipping in zstd_fast.c, zstd_double_fast.c, and zstd_lazy.c
  • Update build configuration in libzstd.mk to support new optimization flags
  • Optimize compression speed through strategic match search reduction with minor compression ratio tradeoff

Performance Results

Test Setup: AMD Genoa (Zen4), SMT OFF, GCC 14.2, single-threaded. ~2% run-to-run variation expected.

Level 3

Dataset Baseline CS (MB/s) Optimized CS (MB/s) CS Speedup Ratio Baseline Ratio Optimized Ratio Change DS Change
silesia.tar 299.1 319.3 +6.8% 3.206 3.165 -1.3% +2.6%
calgary.tar 246.1 258.8 +5.2% 3.107 3.095 -0.4% +3.6%
canterbury.tar 306.3 309.5 +1.0% 4.461 4.468 +0.2% +2.0%
freeBSD-13.2 698.5 876.8 +25.5% 1.598 1.588 -0.6% -0.6%

Level 5

Dataset Baseline CS (MB/s) Optimized CS (MB/s) CS Speedup Ratio Baseline Ratio Optimized Ratio Change DS Change
silesia.tar 152.1 258.7 +70.1% 3.378 3.302 -2.2% +0.7%
calgary.tar 131.1 214.1 +63.3% 3.220 3.179 -1.3% +4.2%
canterbury.tar 173.2 261.8 +51.2% 4.577 4.539 -0.8% +3.8%
freeBSD-13.2 555.2 763.5 +37.5% 1.605 1.599 -0.4% -2.0%

Level 6

Dataset Baseline CS (MB/s) Optimized CS (MB/s) CS Speedup Ratio Baseline Ratio Optimized Ratio Change DS Change
silesia.tar 111.5 153.6 +37.8% 3.462 3.375 -2.5% -6.4%
calgary.tar 94.3 133.7 +41.8% 3.309 3.218 -2.7% -6.9%
canterbury.tar 111.2 178.6 +60.7% 4.708 4.580 -2.7% -5.8%
freeBSD-13.2 421.3 560.6 +33.1% 1.610 1.604 -0.4% -3.1%

Summary

  • Level 3: 1-26% compression speed gain with negligible ratio impact
  • Level 5: 37-70% compression speed gain with 0.4-2.2% ratio cost
  • Level 6: 33-61% compression speed gain with 2.5-2.7% ratio cost and 3-7% decompression regression
  • Best suited for applications where compression throughput is the primary requirement

- Add ZSTD_ENABLE_SEARCH_SKIP_HIGH macro to portability_macros.h for conditional compilation
- Extend compression level parameters in clevels.h with search-skip configurations
- Integrate adaptive search skipping in zstd_fast.c, zstd_double_fast.c, and zstd_lazy.c
- Update build configuration in libzstd.mk to support new optimization flags
- Optimize compression speed through strategic match search reduction with minor compression ratio tradeoff
@meta-cla meta-cla Bot added the CLA Signed label Apr 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants