QOI and XZ

QOI, the Quite OK Image format, has been gaining in popularity. Chris Wellons offers a great analysis.

QOI's key advantages is its simplicity. Being a byte-oriented format without entropy encoding, it can be further compressed with generic codecs like lz4, xz, and zstd. PNG, on the other hand, uses DEFLATE compression internally and is typically resistant to further compression. By applying a stronger compression algorithm on QOI output, you can often achieve a smaller file size compared to PNG.

Lasse Collin has shared some effective options for compressing uncompressed BMP/TIFF files. I tested them on the QOI benchmark images.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
% cat ~/tmp/b.sh
#!/bin/zsh -ue
f() {
  /tmp/p/qoi/qoiconv $1 ${1/.png/.qoi}
  convert $1 ${1/.png/.bmp}
  convert $1 -compress none ${1/.png/.tiff}
  xz --lzma2=pb=0 -fk ${1/.png/.qoi}
  if [[ $(file $1) =~ RGBA ]]; then
    pnm=${1/.png/.pam}
    convert $1 $pnm
    xz --delta=dist=4 --lzma2=lc=4 -fk $pnm
    xz --delta=dist=4 --lzma2=lc=4 -fk ${1/.png/.bmp}
    xz --delta=dist=4 --lzma2=lc=4 -fk ${1/.png/.tiff}
  else
    pnm=${1/.png/.ppm}
    convert $1 $pnm
    xz --delta=dist=3 --lzma2=pb=0 -fk $pnm
    xz --delta=dist=3 --lzma2=pb=0 -fk ${1/.png/.bmp}
    xz --delta=dist=3 --lzma2=pb=0 -fk ${1/.png/.tiff}
  fi
  stat -c '%n %s' $1 ${1/.png/.qoi.xz} $pnm.xz ${1/.png/.bmp.xz} ${1/.png/.tiff.xz}
}

f $1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
% cd /tmp/dc-img/images/
% ls -1 **/*.png | rush ~/tmp/b.sh '"{}"'
% for i in *; do echo ===$i===; (cd $i; ruby -e 'png=qoi=bmp=pnm=tiff=0; Dir.glob("**/*.qoi.xz").each{|f| png+=File.size(f.sub(/\.qoi.xz/,".png")); qoi+=File.size(f); bmp+=File.size(f.sub(/\.qoi/,".bmp")); ppm=f.sub(/\.qoi/,".ppm");  pnm+=File.exists?(ppm) ? File.size(ppm) : File.size(f.sub(/\.qoi/,".pam")); tiff+=File.size(f.sub(/\.qoi/,".tiff")); }; puts ".png:     #{png}\n.qoi.xz:  #{qoi}\n.bmp.xz:  #{bmp}\n.tiff.xz: #{tiff}\n.p?m.xz:  #{pnm}"';); done
===icon_64===
.png:     828119
.qoi.xz:  708480
.bmp.xz:  730472
.tiff.xz: 757760
.p?m.xz:  735296
===icon_512===
.png:     11154424
.qoi.xz:  7476640
.bmp.xz:  8042032
.tiff.xz: 8064476
.p?m.xz:  8039192
===photo_kodak===
.png:     15394305
.qoi.xz:  12902852
.bmp.xz:  13612440
.tiff.xz: 13616140
.p?m.xz:  13610844
===photo_tecnick===
.png:     237834256
.qoi.xz:  213268188
.bmp.xz:  210591724
.tiff.xz: 210508596
.p?m.xz:  210468412
===photo_wikipedia===
.png:     88339751
.qoi.xz:  86679696
.bmp.xz:  86380124
.tiff.xz: 86274480
.p?m.xz:  86241296
===pngimg===
.png:     229608249
.qoi.xz:  193382668
.bmp.xz:  187439556
.tiff.xz: 187718460
.p?m.xz:  187349124
===screenshot_game===
.png:     266238855
.qoi.xz:  218915316
.bmp.xz:  217036084
.tiff.xz: 217177880
.p?m.xz:  217084420
===screenshot_web===
.png:     40272678
.qoi.xz:  21321460
.bmp.xz:  21458496
.tiff.xz: 21532360
.p?m.xz:  21533432
===textures_photo===
.png:     37854634
.qoi.xz:  28967008
.bmp.xz:  30054968
.tiff.xz: 30064236
.p?m.xz:  30059784
===textures_pk===
.png:     43523493
.qoi.xz:  54117600
.bmp.xz:  70302536
.tiff.xz: 45385684
.p?m.xz:  73617008
===textures_pk01===
.png:     18946769
.qoi.xz:  14950836
.bmp.xz:  14835648
.tiff.xz: 14853420
.p?m.xz:  14839312
===textures_pk02===
.png:     102962935
.qoi.xz:  82279000
.bmp.xz:  79374112
.tiff.xz: 79348768
.p?m.xz:  79336276
===textures_plants===
.png:     51765329
.qoi.xz:  43681548
.bmp.xz:  45045392
.tiff.xz: 45154436
.p?m.xz:  45196092

With the help of a delta filter, .bmp.xz can often beat .qoi.xz. This suggests that QOI is a good filter but probably no better than a plain delta filter.

Drop LZ match finders

QOI_OP_INDEX essentially does length-1 LZ77 using a conceptual window that contains 64 unique pixels. When further compressed, another match finder seems to help very little.

1
2
3
4
5
6
7
8
% cat ~/tmp/qoi-weak-xz.sh
#!/bin/zsh
/tmp/p/qoi/qoiconv $1 ${1/.png/.qoi}
xz --lzma2=pb=0 -fk ${1/.png/.qoi}
xz --lzma2=dict=4KiB,mode=fast,nice=2,mf=hc3,depth=1 -c ${1/.png/.qoi} > ${1/.png/.qoi.weak-lz.xz}
% cd /tmp/dc-img/images
% ls -1 **/*.png | rush ~/tmp/qoi-weak-xz.sh '"{}"'
% ruby -e 'Dir.glob("*").each{|dir| strong=weak=0; Dir.glob("#{dir}/*.qoi.weak-lz.xz").each{|f| weak+=File.size(f); strong+=File.size(f.sub(/\.weak-lz/,""));}; puts "#{dir}\t#{strong}\t#{weak}\t#{(100.0*weak/strong-100).round(2)}%" }'
1
2
3
4
5
6
7
8
9
10
11
12
13
14
subdirectory    strong  weak     increase
icon_512        7476640 8629900 15.42%
icon_64 708480  735036  3.75%
photo_kodak     12902852        13464072        4.35%
photo_tecnick   213268188       217460392       1.97%
photo_wikipedia 86679696        88609716        2.23%
pngimg  193382668       206679224       6.88%
screenshot_game 218915316       234889060       7.3%
screenshot_web  21321460        24820020        16.41%
textures_photo  28967008        31249492        7.88%
textures_pk     54117600        57956168        7.09%
textures_pk01   14950836        15749556        5.34%
textures_pk02   82279000        87747576        6.65%
textures_plants 43681548        45494084        4.15%

Combining delta coding and a fast entropy encoder can achieve smaller file sizes compared to compressed QOI.