Outputs: speed and size

For simple image processing, writing the output can take a significant proportion of the overall time.

• Sample input images
• TIFF file output
• MIFF file output
• MPC file output
• PNG file output
• JPG file output
• Summary
• Conclusions
• Scripts

Tests were performed on a laptop with Intel Core i7 processor and 12 GB RAM and 1 TB internal disk, under Windows 8.1, with the pre-built integer Q16 ImageMagick binary.

Sample input images

Image compression algorithms are designed for specific image types, so their effectiveness varies according to how closely the image fits the type. We will test with two images: a large digital photograph, and a small graphic with flat colours. Both these images will be subjected to a range of compression methods available in ImageMagick.

Sample photo

zp_sus_sat.tiff is fairly typical of my photographs.

%IM%identify zp_sus_sat.tiff 

zp_sus_sat.tiff TIFF 4924x7378 4924x7378+0+0 16-bit sRGB 167.9MB 0.000u 0:00.015

for /F "usebackq" %%S in (`%IM%identify ^
  -precision 15 ^
  -format "%%[fx:w*h*6]" ^
  zp_sus_sat.tiff`) do set PH_BYTES=%%S

echo PH_BYTES=%PH_BYTES% 

PH_BYTES=217975632 

PH_BYTES is the number of bytes required for the pixels, not compressed.

Here is a reduced-size version:

%IM%convert zp_sus_sat.tiff -resize 600x400 ss_ph_sm.jpg

Sample graphic

For a graphic image, with flat colours but anti-aliasing around shapes, we will create a smaller image.

%IM%convert ^ -size 600x400 xc:khaki ^ -fill #f92 ^ -draw "circle 200,200 200,350" ^ ( -size 600x400 xc:None ^ -fill #29f ^ -draw "polygon 200,100 550,200 300,50" ^ -fill #2f9 ^ -draw "rectangle 50,275 550,375" ^ -channel A -evaluate Multiply 0.8 +channel ^ ) ^ -composite ^ -alpha off ^ ss_gr_in.png

%IM%identify ss_gr_in.png 

ss_gr_in.png PNG 600x400 600x400+0+0 16-bit sRGB 15.6KB 0.000u 0:00.000

for /F "usebackq" %%S in (`%IM%identify ^
  -precision 15 ^
  -format "%%[fx:w*h*6]" ^
  ss_gr_in.png`) do set GR_BYTES=%%S

echo GR_BYTES=%GR_BYTES% 

GR_BYTES=1440000 

GR_BYTES is the number of bytes required for the pixels, not compressed.

TIFF file output

The script compSpd.bat measures the speed of writing a file at various compressions. It does this by timing the appropriate convert command, and subtracting the time required for the same command that outputs to NULL:.

It also compares the output with the original, giving a number between zero and one. As a rule of thumb, a difference of less than 0.01 (1%) looks the same as the original.

TIFF of photo

del ph_summ.csv >nul

call %PICTBAT%compSpd zp_sus_sat.tiff ss_tiff tiff 5 ph_summ.csv

The smallest lossless compression method is ZIP. But it is slow.

-compress None is fast. JPEG compression takes twice as long, but the file size is much smaller.

When ZIP compression is enabled, writing to TIFF is also sensitive to -quality:

call %PICTBAT%qual99Spd zp_sus_sat.tiff ss_tiff_z tiff 5 ph_summ.csv

-quality 0 is equivalent to -quality 70. Lower quality numbers give shorter times and larger files. The quality with the shortest time is highlighted.

TIFF of graphic

del gr_summ.csv >nul

call %PICTBAT%compSpd ss_gr_in.png ss_gr_tiff tiff 100 gr_summ.csv

For this graphic image, ZIP is slightly faster than -compress None, and the filesize is reduced by about 99%. LZW can be slightly faster, but larger, than ZIP at default quality.

When ZIP compression is enabled, writing to TIFF is also sensitive to -quality:

call %PICTBAT%qual99Spd ss_gr_in.png ss_gr_tiff_z tiff 100 gr_summ.csv

-quality 0 is equivalent to -quality 70. Lower quality numbers give shorter times and larger files.

MIFF file output

MIFF of photo

call %PICTBAT%compSpd zp_sus_sat.tiff ss_miff miff 5 ph_summ.csv

For MIFF, all compressions are lossless. ZIP and LZW give some compression, but it is very slight, with a large penalty in speed (roughly factor 40).

When ZIP compression is enabled, writing to MIFF is also sensitive to -quality:

call %PICTBAT%qual99Spd zp_sus_sat.tiff ss_miff_z miff 5 ph_summ.csv

Changing the quality changes the size. However, for this photograph, there is not much change in size or procesing time. The smallest size is from -quality 1, no compression. Between settings that give compression, there is little difference in speed or size.

MIFF of graphic

call %PICTBAT%compSpd ss_gr_in.png ss_gr_miff miff 100 gr_summ.csv

As with the photograph, -compress None is the quickest, more or less. For this graphic, ZIP is about one-fortieth of the uncompressed size, with a speed penalty factor of less than 5. LZW is slightly faster than ZIP.

When ZIP compression is enabled, writing to MIFF is also sensitive to -quality:

call %PICTBAT%qual99Spd ss_gr_in.png ss_gr_miff_z miff 100 gr_summ.csv

For this graphic, changing quality has a significant effect on size and processing time. The smallest size, and longest time, are from -quality 90.

MPC file output

For the MPC format, the size shown is of the .CACHE file. The MPC file is always about 550 bytes.

MPC of photo

call %PICTBAT%compSpd zp_sus_sat.tiff ss_mpc mpc 5 ph_summ.csv

Like MIFF, all MPC compressions are lossless.

Why is there a speed variation, even though there is no size variation? I don't know.

For -compress None, MPC is about half the speed of MIFF. The files are 33% larger, because it also stores alpha values. So, for this photograph, MPC has no advantage over MIFF.

MPC of graphic

call %PICTBAT%compSpd ss_gr_in.png ss_gr_mpc mpc 100 gr_summ.csv

For this graphic, some times are negative, so writing the file took less time than writing to NULL:.

PNG file output

For the PNG file format, compression is controlled by the -quality setting. PNG compression is always lossless. The -quality setting affects the speed and size. For a description of this setting see the official ImageMagick: Command-line Options: quality.

The script qual99Spd.bat tries 100 quality settings, from 0 to 99. The table show two numbers for each quality setting: the time in seconds, and the file size in bytes.

PNG of photo

call %PICTBAT%qual99Spd zp_sus_sat.tiff ss_png png 5 ph_summ.csv

The default quality setting of 75 gives roughly the smallest file, and takes roughly the longest. For the quickest time, -quality 8 is a good choice. It takes about 15 times as long as TIFF with no compression, and 50 times as long as MIFF with no compression.

For this photograph, the ratio between the worst and best times is more than 8:1.

PNG of graphic

call %PICTBAT%qual99Spd ss_gr_in.png ss_gr_png png 100 gr_summ.csv

For this graphic image, the default quality 75 is worse in both speed and size than some other qualities, such as 47, 57, ... 97. The quickest quality is higlighted. Good choices for speed include 10, 11, 20, 21, 30, 31.

If the very smallest PNG is desired, the pngcrush program can be used.

JPG file output

Just for fun, I repeat the test for JPG compression. I don't use jpegs unless I really have to, or for final output to the web. Jpeg files are only 8 bits/channel/pixel, and compression is always lossy.

Lossy compresson means the compressed image is different to the input. The table shows the difference at each quality setting. So the three numbers are:

• Elapsed time in seconds
• File size in bytes
• RMSE difference from the input image, between zero and one.

JPG of photo

call %PICTBAT%qual99Spd zp_sus_sat.tiff ss_jpg jpg 5 ph_summ.csv

Except for -quality 0, increasing the quality number increases the file size and elapsed time, more or less. -quality 0 gives the same file size as -quality 92.

My rule-of-thumb "less than 1% difference" is achieved for quality levels at or above 90. If we tolerate a 2% difference, a quality of 25 or above will give this.

JPG of graphic

call %PICTBAT%qual99Spd ss_gr_in.png ss_gr_jpg jpg 100 gr_summ.csv

As for the photo, my rule-of-thumb "less than 1% difference" is achieved for quality levels at or above 90. If we tolerate a 2% difference, we need to specify a quality of 46 or above.

JPG compression is not suited to graphics. It makes smooth colours rougher, and introduces ringing at edges.

Summary

We summarise the findings for this photograph and graphic.

cPrefix /p0 /iph_summ.csv /t"Format,Option,Time,Compression,Difference"
cSort   /p0 /iph_summ.csv /oph_summ.csv /h /kTime
call csv2tab ph_summ

For this photograph, the fastest methods are uncompressed MPC, MIFF and TIFF. Low-quality JPG is also fast but the quality is horrible.

The fastest PNG hardly saves any space compare to TIFF with no compression, and takes ten times as long. PNG is not recommended, except for output to web pages that require no loss (making JPG unsuitable).

cPrefix /p0 /igr_summ.csv /t"Format,Option,Time,Compression,Difference"
cSort   /p0 /igr_summ.csv /ogr_summ.csv /h /kTime
call csv2tab gr_summ

For this graphic, the fastest methods are uncompressed MPC and MIFF. The fastest "portable" format is PNG, -quality 10.

Conclusions

I am interested only in lossless compression.

For writing photographic images:

• MIFF with no compression is fastest. I use this format for temporary, intermediate results.
• For long-term storage of photographs, TIFF is a reasonable choice. ZIP gives maximum compression. Uncompressed is about 30% larger but 70 times quicker.
• PNG is much slower.

For writing graphic images:

• MPC with no compression is fastest. This is a good format for temporary, intermediate results.
• However, for convenience, I use MIFF for temporary, intermediate results.
• With a suitable choice of -quality, PNG is twice as fast as TIFF, and much smaller. Quality 10 is good, or 20 or 30.

Scripts

For convenience, .bat scripts are also available in a single zip file. See Zipped BAT files.

The scripts use ls and stat, as provided by Cygwin. They also use my own timecc.exe, cProject.exe and cEcho.exe. I don't publish source or executables of these.

compSpd.bat

rem Speeds of writing compressed outputs.

@rem %1 is input image file.
@rem %2 is prefix for output data files.
@rem %3 is output sufix, eg png, tiff, jpg.
@rem %4 is number of times to execute command [default 1].
@rem %5 is output summary list.


set INFILE=%1

set OUT_PREFIX=%2
if "%OUT_PREFIX%"=="" set OUT_PREFIX=ts

set EXT=%3
if "%EXT%"=="" set EXT=tiff

set N_TIMES=%4
if "%N_TIMES%"=="" set N_TIMES=1

set BASE_CMD=%IM%convert %INFILE% -quiet

set DATA_EXT=%EXT%
if /i "%DATA_EXT%"=="mpc" set DATA_EXT=cache

set OUTFILE=%OUT_PREFIX%_cs.%EXT%
set DATAFILE=%OUT_PREFIX%_cs.%DATA_EXT%

set SUM_LIST=
if not "%5"=="" set SUM_LIST=%5


for /F "usebackq" %%B in (`timecc ^
  /n%N_TIMES% /f"\t" ^
  /c"%BASE_CMD% -compress None NULL:"`) do set BASE_TIME=%%B

echo BASE_TIME=%BASE_TIME%

echo -compress,Seconds,Compression %%,Difference >%OUT_PREFIX%.csv

rem FIXME: also do one with no "-compress" setting.

for /F "usebackq" %%C in (`%IM%convert -list compress`) do (
  echo %%C

  del %OUTFILE% >nul

  rem timecc ^
  rem   /n%N_TIMES% /f"%%C,\t," ^
  rem   /O%OUT_PREFIX%.csv /b%BASE_TIME% ^
  rem   /c"%BASE_CMD% -compress %%C %OUTFILE%"

  for /F "usebackq" %%T in (`timecc ^
    /n%N_TIMES% /f"\t" ^
    /o- /b%BASE_TIME% ^
    /c"%BASE_CMD% -compress %%C %OUTFILE%"`) do set TIME=%%T

  if exist %OUTFILE% (

    for /F "usebackq tokens=2 delims=() " %%D in (`%IM%compare ^
      -quiet -metric RMSE %INFILE% %OUTFILE% NULL: 2^>^&1`) do set DIFF=%%D

    for /F "usebackq" %%B in (`stat --printf=%%s %DATAFILE%`) do set BYTES=%%B
    echo BYTES=%BYTES%

    for /F "usebackq" %%C in (`%IM%identify ^
      -ping ^
      -format "%%[fx:int(!BYTES!*100/w/h/6*100+0.5)/100]" ^
      %OUTFILE%`) do set COMPR=%%C

    cEcho /O%OUT_PREFIX%.csv /X /s"%%C,!TIME!s,!COMPR!%%,!DIFF!"

    if not "%SUM_LIST%"=="" if /I "%%C"=="None" cEcho /O%SUM_LIST% /s"%EXT%,-compress None,!TIME!,!COMPR!,!DIFF!"

  ) else (
    echo No output file >>%OUT_PREFIX%.csv
  )
)

del %OUTFILE% >nul
del %DATAFILE% >nul

type %OUT_PREFIX%.csv

call csv2tab %OUT_PREFIX%

qual99Spd.bat

rem Speeds of writing outputs with quality 00 to 99.
rem Also measures size of output files, in bytes.
rem For TIFF and MIFF, tests just 00, 01, 10, 11, 20, 21 etc.
rem For JPG format, also measures difference from input image.
rem Writes output in HTML table format.

@rem %1 is input image file.
@rem %2 is prefix for output data files.
@rem %3 is output sufix, eg png, tiff, jpg.
@rem %4 is number of times to execute command [default 1].
@rem %5 is output summary list.

@rem Updated:
@rem   26-September-2016  Bug in comparison fixed.


set INFILE=%1

set OUT_PREFIX=%2
if "%OUT_PREFIX%"=="" set OUT_PREFIX=q9s

set EXT=%3
if "%EXT%"=="" set EXT=tiff

set N_TIMES=%4
if "%N_TIMES%"=="" set N_TIMES=1

set SUM_LIST=
if not "%5"=="" set SUM_LIST=%5

set DO_DIFF=0
if /i %EXT%==jpg set DO_DIFF=1
if /i %EXT%==jpeg set DO_DIFF=1

set OUTFILE=%OUT_PREFIX%_q9s.%EXT%
set TIME_FILE=%OUT_PREFIX%_q9s.lis
del %TIME_FILE% >nul
set TMP_LST=%OUT_PREFIX%_q9s_c.csv
del %TMP_LST% >nul

set COMP=
if /i %EXT%==miff set COMP=-compress Zip
if /i %EXT%==tiff set COMP=-compress Zip
if /i %EXT%==tif set COMP=-compress Zip

if "%COMP%"=="" (
  set MAX_J=9
) else (
  set MAX_J=1
)

set BASE_CMD=%IM%convert %INFILE% -quiet %COMP%

for /F "usebackq" %%B in (`timecc ^
  /n%N_TIMES% /f"\t" ^
  /f"\t" /c"%BASE_CMD% -quality 0 NULL:"`) do set BASE_TIME=%%B

echo BASE_TIME=%BASE_TIME%

set DIFF=
del %OUT_PREFIX%.csv >nul

if %MAX_J%==9 (
  cEcho /O%OUT_PREFIX%.csv /f\r\n /s"-quality,0,1,2,3,4,5,6,7,8,9"
) else (
  cEcho /O%OUT_PREFIX%.csv /f\r\n /s"-quality,0,1..9"
)

for /L %%i in (0,1,9) do (
  cEcho /O%OUT_PREFIX%.csv /f\r /s%%i0,
  for /L %%j in (0,1,%MAX_J%) do (

    for /F "usebackq" %%T in (`timecc ^
      /n%N_TIMES% /f"\t" ^
      /o- /b%BASE_TIME% ^
      /c"%BASE_CMD% -quality %%i%%j %OUTFILE%"`) do set TIME=%%T

    for /F "usebackq" %%B in (`stat --printf=%%s %OUTFILE%`) do set BYTES=%%B

    for /F "usebackq" %%C in (`%IM%identify ^
      -ping ^
      -format "%%[fx:int(!BYTES!*100/w/h/6*100+0.5)/100]" ^
      %OUTFILE%`) do set COMPR=%%C

    set nDIFF=0
    set sDIFF=

    if %DO_DIFF%==1 (
      for /F "usebackq tokens=2 delims=() " %%D in (`%IM%compare ^
        -quiet -metric RMSE %INFILE% %OUTFILE% NULL: 2^>^&1`) do set nDIFF=%%D

      set sDIFF=\^(br /\^)!nDIFF!
    )

    echo TIME=!TIME! BYTES=!BYTES! COMPR=!COMPR! nDIFF=!nDIFF!

    echo !TIME! >>%TIME_FILE%

    if %%j==%MAX_J% (
      cEcho /O%OUT_PREFIX%.csv /X /f\r /s"!TIME!s\^(br /\^)!COMPR!%%!sDIFF!\n"
    ) else (
      cEcho /O%OUT_PREFIX%.csv /X /f\r /s"!TIME!s\^(br /\^)!COMPR!%%!sDIFF!,"
    )

    cEcho /O%TMP_LST% /s"%EXT%,%COMP% -quality %%i%%j,!TIME!,!COMPR!,!nDIFF!"
  )
)

echo BASE_TIME=%BASE_TIME%

call csv2tab %OUT_PREFIX%

del %OUTFILE% >nul

type %TIME_FILE%

for /F "usebackq" %%T in (`cSort ^
  /p0 /k0 /i%TIME_FILE% /o- ^| cHead /p0 /i- /h1 /o-`) do set MIN_TIME=%%T

echo MIN_TIME=%MIN_TIME%

chStrs /i%OUT_PREFIX%.htm /o%OUT_PREFIX%.htm /f\(td\)%MIN_TIME% /t"\(td class=\"good\"\)%MIN_TIME%"

cSort /i%TMP_LST% /o- /k2 | cHead /i- /h1 /O%SUM_LIST%

%IM%identify -version

Version: ImageMagick 6.9.5-3 Q16 x86 2016-07-22 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2015 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Visual C++: 180040629
Features: Cipher DPC Modules OpenMP 
Delegates (built-in): bzlib cairo flif freetype jng jp2 jpeg lcms lqr openexr pangocairo png ps rsvg tiff webp xml zlib

Page version v1.0 1-Jun-2015.

Page created 27-Sep-2016 21:56:09.

Copyright © 2016 Alan Gibson.