ZStandard: A very fast compression Alogrithm

ZStandard the new compression algorithm

  • Facebook’s new compression algorithm
  • Has very good performance numbers (speed) compared to gzip and the ’traditional’ unix tools
  • Linux binaries are available for all popular distributions search for zstd compression.
  • brew install zstd to install on Mac.

Comparison of file sizes with various compressions.

  • zstd was run with -20 (highest compression)
  • gzip with (-9)
  • gzip was 7 times slower.
    -rw-r--r--  1 vivekv  staff   301M May 23 20:38 ipdata.sqlite3
    -rw-r--r--  1 vivekv  staff   103M May 23 20:38 ipdata.sqlite3.gz
    -rw-r--r--  1 vivekv  staff    66M May 23 20:38 ipdata.sqlite3.zst