snibgo's ImageMagick pages

Details, details

Detail draws the eye into a photograph, or distracts the gaze. It is also useful for aligning (registering) photos or video frames.

I show one use for detail at Adding zing to photographs: Light Blur Difference Mask, with a script that is light where the source image has detail, by finding the difference with a blur, and blurring again more heavily. It is a smoothed rectified high-pass filter.

Detail is also useful for image alignment.

This page shows some alternative methods of finding detail in an image. A related problem is that of measuring detail.

I bracket some commands with "call StopWatch". This shows approximately how long the command took, in the format "d hh:mm:ss", ie days, hours, minutes and seconds. This is for an input image approximately 5000x5000 pixels.

Set up initial conditions:

set WEB_SIZE=-resize 500x500

rem set DT_SRC=\web\im\fromNef\AGA_1468_ssd.tiff
set DT_SRC=\pictures\20130713\aga_1468_srgb.tiff

if not exist %DT_SRC% (
  echo Can't find DT_SRC [%DT_SRC%]
)

set DT_CROP=-crop 5632x4924+0+0 +repage

%IM%convert %DT_SRC% %DT_CROP% -compress Zip dt_src.tiff

set DT_SRC=dt_src.tiff

set nlDEBUG=1

call %PICTBAT%getPointsize %DT_SRC% 50

set nlPOINTSIZE=%gpPointsize%
set nlSTROKEWIDTH=10

A source image

The source photograph for this page has had minimal processing from the raw camera image. It has been given a reasonable gamma and sigmoidal contrast, and is cropped. It needs a slight warming up (colour shift towards red) and saturation boost.

The (manual) focus isn't great.

On this page, all operations are performed on the full-size image, roughly 5000x5000 pixels. Results are resized for display on the web.

A source image.

%IM%convert ^
  %DT_SRC% ^
  %WEB_SIZE% ^
  dt_src_sm.jpg
dt_src_sm.jpg

The techniques

Some of the techniques shown here include "-auto-level -auto-gamma". This is only for display on the web page. In normal use, "-auto-level" is unnecessary and prevents true comparisons, and "-auto-gamma" may create false results (as some values may get squished together).

Use a difference from blur to find detail.

call StopWatch
call %PICTBAT%ltBlrDiff %DT_SRC%
call StopWatch

%IM%convert ^
  dt_src_lbd.tiff ^
  %WEB_SIZE% ^
  dt_src_lbd_sm.jpg
0 00:00:11
dt_src_lbd_sm.jpg

Use standard deviation to find detail.

call StopWatch
%IM%convert ^
  %DT_SRC% ^
  -statistic standard-deviation 10x10 ^
  -grayscale RMS -auto-level ^
  -write dt_stddev.tiff ^
  -auto-gamma ^
  %WEB_SIZE% ^
  dt_stddev_sm.jpg
call StopWatch
0 00:02:06

-auto-gamma might lose a little bit of data,
so we don't apply it to the full-size image.

dt_stddev_sm.jpg

For the two methods, we can find the ten lightest pixels. For each pixel found, an exclusion zone is made, so pixels within a certain radius will not be considered. (The default radius is 10% of the shortest image dimension.) The exclusion circles are shown. The brightest pixel is labelled "0", the next is labelled "1", etc.

Find the lightest pixels of the blur-difference.

call %PICTBAT%nLightest dt_src_lbd.tiff

%IM%convert ^
  dt_src_lbd_nl_dbg.tiff ^
  %WEB_SIZE% ^
  dt_src_lbd_nl_dbg_sm.jpg
dt_src_lbd_nl_dbg_sm.jpg

Find the lightest pixels of the standard deviation.

call %PICTBAT%nLightest dt_stddev.tiff

%IM%convert ^
  dt_stddev_nl_dbg.tiff ^
  %WEB_SIZE% ^
  dt_stddev_nl_dbg_sm.jpg
dt_stddev_nl_dbg_sm.jpg

The script nLightest.bat takes three parameters:

  1. The input filename.
  2. The maximum number of points to be found. Default: 10.
  3. The exclusion circle radius, expressed as a percentage of the minimum image dimension. Default: 10.

This image is roughly square so, at the default radius of 10%, the maximum number of points that will be found is roughly 5 * 5 = 25.

If the input image was entirely black, the script would find no points. The primary output is an list of (x,y) coordinates, echoed to stdout. For example, the output for blur-difference is:

4713,3713
4860,3042
4804,4384
4190,2261
4370,4116
4844,1662
9,628
4509,816
2742,3228
5248,2032

When the script is run with set nlDEBUG=1, it creates the images shown above, with the red circles and numbers. This slows down the script.

We can use a difference of two blurs to find detail. This will eliminate very fine detail, as well as large-scale variations. It is a mid-pass filter.

Use a difference between two blurs to find detail.

call StopWatch
call %PICTBAT%twoBlrDiff %DT_SRC%
call StopWatch
0 00:00:17

What are the ten lightest points?

call %PICTBAT%nLightest dt_src_2bd.tiff

%IM%convert ^
  dt_src_2bd_nl_dbg.tiff ^
  %WEB_SIZE% ^
  dt_src_2bd_nl_dbg_sm.jpg
dt_src_2bd_nl_dbg_sm.jpg

DoG Difference of Gaussians:

Use a DoG to find detail. This is quite slow.

call StopWatch
%IM%convert ^
  %DT_SRC% ^
  -bias 50%% -morphology Convolve DoG:0,4.0,6.0 ^
  -grayscale RMS -auto-level ^
  dt_src_dog.tiff
call StopWatch
0 00:01:19

What are the ten lightest points?

call %PICTBAT%nLightest dt_src_dog.tiff

%IM%convert ^
  dt_src_dog_nl_dbg.tiff ^
  %WEB_SIZE% ^
  dt_src_dog_nl_dbg_sm.jpg
dt_src_dog_nl_dbg_sm.jpg

LoG Laplacian of Gaussians:

Use a LoG to find detail. This is quite slow.

call StopWatch
%IM%convert ^
  %DT_SRC% ^
  -bias 50%% -morphology Convolve LoG:0x4 ^
  -grayscale RMS -auto-level ^
  dt_src_log.tiff
call StopWatch
0 00:00:43

What are the ten lightest points?

call %PICTBAT%nLightest dt_src_log.tiff

%IM%convert ^
  dt_src_log_nl_dbg.tiff ^
  %WEB_SIZE% ^
  dt_src_log_nl_dbg_sm.jpg
dt_src_log_nl_dbg_sm.jpg

Edge creates a limited range of gray, with a large proportion being white, so we blur the result.

According to the source code effect.c, the function EdgeImage uses morphology convolve with a Laplacian kernel of the given radius, 1 iteration, 0.0 bias.

Use Edge to find detail.

call StopWatch
%IM%convert ^
  %DT_SRC% ^
  -edge 2 ^
  -grayscale RMS ^
  -blur 0x2 ^
  -auto-level ^
  dt_src_edge2.tiff
call StopWatch
0 00:00:11

What are the ten lightest points?

call %PICTBAT%nLightest dt_src_edge2.tiff

%IM%convert ^
  dt_src_edge2_nl_dbg.tiff ^
  %WEB_SIZE% ^
  dt_src_edge2_nl_dbg_sm.jpg
dt_src_edge2_nl_dbg_sm.jpg

Use Edge to find detail.

call StopWatch
%IM%convert ^
  %DT_SRC% ^
  -edge 10 ^
  -grayscale RMS ^
  -blur 0x10 ^
  -auto-level ^
  dt_src_edge10.tiff
call StopWatch
0 00:00:33

What are the ten lightest points?

call %PICTBAT%nLightest dt_src_edge10.tiff

%IM%convert ^
  dt_src_edge10_nl_dbg.tiff ^
  %WEB_SIZE% ^
  dt_src_edge10_nl_dbg_sm.jpg
dt_src_edge10_nl_dbg_sm.jpg

For the purpose of finding detail, "-edge 10" is a disaster.

Canny makes thin white lines on a black background. Blurring this will emphasise intersections.

Use Canny to find detail.

call StopWatch
%IM%convert ^
  %DT_SRC% ^
  -canny 0x4+10%%+30%% ^
  -grayscale RMS ^
  -blur 0x10 ^
  -auto-level ^
  dt_src_can.tiff
call StopWatch
0 00:00:25

What are the ten lightest points?

call %PICTBAT%nLightest dt_src_can.tiff

%IM%convert ^
  dt_src_can_nl_dbg.tiff ^
  %WEB_SIZE% ^
  dt_src_can_nl_dbg_sm.jpg
dt_src_can_nl_dbg_sm.jpg

This is a good result. Tweaking the "-canny" parameters might also find some features in the face.

We can use "-morhpology Convolve Sobel" to find a gradient magnitude. The result is similar to the difference of two blurs, but this doesn't give a zero result in the centre of edges, so doesn't need a further blur.

Slope magnitude.

call StopWatch

call %PICTBAT%slopeMag %DT_SRC%

call StopWatch
0 00:00:25

What are the ten lightest points?

call %PICTBAT%nLightest %smOUTFILE%

%IM%convert ^
  %nlDEBUG_FILE% ^
  %WEB_SIZE% ^
  dt_src_sm_nl_dbg_sm.jpg
dt_src_sm_nl_dbg_sm.jpg

Prewitt

See Convolution of Images: Prewitt.

Slope magnitude by Prewitt.

call StopWatch

%IM%convert ^
  %DT_SRC% ^
  -alpha off ^
  -define convolve:scale="50%%^!" -bias 50%% ^
  ( -clone 0 -morphology Convolve Prewitt:0 ) ^
  ( -clone 0 -morphology Convolve Prewitt:90 ) ^
  -delete 0 -solarize 50%% -level 50,0%% ^
  +level 0,70.7%% ^
  -evaluate Pow 2 ^
  -compose plus -composite  ^
  -evaluate Pow 0.5 ^
  -separate -evaluate-sequence Max ^
  -auto-level -auto-gamma ^
  dt_src_prew.tiff

call StopWatch
0 00:00:14

What are the ten lightest points?

call %PICTBAT%nLightest dt_src_prew.tiff

%IM%convert ^
  %nlDEBUG_FILE% ^
  %WEB_SIZE% ^
  dt_src_prew_sm.jpg
dt_src_prew_sm.jpg

Compass

See Convolution of Images: Compass.

Slope magnitude by Compass.

call StopWatch

%IM%convert ^
  %DT_SRC% ^
  -alpha off ^
  -define convolve:scale="50%%^!" -bias 50%% ^
  ( -clone 0 -morphology Convolve Compass:0 ) ^
  ( -clone 0 -morphology Convolve Compass:90 ) ^
  -delete 0 -solarize 50%% -level 50,0%% ^
  +level 0,70.7%% ^
  -evaluate Pow 2 ^
  -compose plus -composite  ^
  -evaluate Pow 0.5 ^
  -separate -evaluate-sequence Max ^
  -auto-level -auto-gamma ^
  dt_src_comp.tiff

call StopWatch
0 00:00:14

What are the ten lightest points?

call %PICTBAT%nLightest dt_src_comp.tiff

%IM%convert ^
  %nlDEBUG_FILE% ^
  %WEB_SIZE% ^
  dt_src_comp_sm.jpg
dt_src_comp_sm.jpg

Roberts

See Convolution of Images: Roberts.

Slope magnitude by Roberts.

call StopWatch

%IM%convert ^
  %DT_SRC% ^
  -define convolve:scale="50%%^!" -bias 50%% ^
  -define morphology:compose=Lighten ^
  -morphology Convolve "Roberts:@" ^
  -grayscale RMS ^
  -auto-level -auto-gamma ^
  dt_src_rob.tiff

call StopWatch
0 00:00:25

What are the ten lightest points?

call %PICTBAT%nLightest dt_src_rob.tiff

%IM%convert ^
  %nlDEBUG_FILE% ^
  %WEB_SIZE% ^
  dt_src_rob_sm.jpg
dt_src_rob_sm.jpg

Kirsch

See Convolution of Images: Kirsch.

Slope magnitude by Kirsch.

call StopWatch

%IM%convert ^
  %DT_SRC% ^
  -define convolve:scale="50%%^!" -bias 50%% ^
  -define morphology:compose=Lighten ^
  -morphology Convolve "Kirsch:@" ^
  -grayscale RMS ^
  -auto-level -auto-gamma ^
  dt_src_kir.tiff

call StopWatch
0 00:00:14

What are the ten lightest points?

call %PICTBAT%nLightest dt_src_kir.tiff

%IM%convert ^
  %nlDEBUG_FILE% ^
  %WEB_SIZE% ^
  dt_src_kir_sm.jpg
dt_src_kir_sm.jpg

Summary and Conclusions

We have seen techniques for expressing the level of detail, expressed as a grayscale image, where "lighter" means "more detail".

The basic techniques are:

We can rank these in order of speed, with the fastest first:

cGrep /p0 /idt_sw_*.lis /s*:* /f\r,\m\n |cSort /p0 /i- /k0 /o- |chStrs /p0 /i- /odt_rank.lis /f"dt_sw_"
0 00:00:11,blrDiff
0 00:00:11,edge2
0 00:00:14,comp
0 00:00:14,rob
0 00:00:14,prew
0 00:00:14,kir
0 00:00:17,2bd
0 00:00:25,sm
0 00:00:25,can
0 00:00:33,edge10
0 00:00:43,log
0 00:01:19,dog
0 00:02:06,stdDev

We have seen that the results of these methods can be processed to find positions of high detail. We want the process to identify the positions that a human viewer would consider interesting, and not to identify "boring" positions. I would rank the process in this order, with the best first:

Tweaking the parameters to each method would change the rankings of speed and quality.

For my purposes, difference between two blurs works well.

Human viewers tend to be most interested in faces. None of these methods rank facial detail within the top ten.

Other possibilities include: Laplacian:{type} (where type=0, 1, 2, 3, 5, 7, 15, 19), Roberts:{angle}, Prewitt:{angle}, Compass:{angle}, Kirsch:{angle}, Frei-Chen:[{type},][{angle}] (type=1, 2, 10, 11-19).

Application: trimming an image

The debug image is intended as a visual aid, but it can be used for other purposes. It is greyscale, apart from the red circles and numbers. By taking the difference between the red and green channels and trimming, we find the boundary box of the first ten circles. This will be at least 10/25 of the area of a square source image.

Crop the source image to the boundary of the red circles.

for /F "usebackq" %%L ^
in (`%IM%convert ^
  dt_src_lbd_nl_dbg.tiff ^
  -separate -delete 2 -compose Difference -composite ^
  -bordercolor Black -border 1 ^
  -format "TRIM_CROP=%%@" ^
  info:`) ^
do set %%L

echo TRIM_CROP="TRIM_CROP" 
TRIM_CROP="TRIM_CROP" 
%IM%convert ^
  %DT_SRC% ^
  -crop %TRIM_CROP% ^
  %WEB_SIZE% ^
  dt_trimmed_sm.jpg
dt_trimmed_sm.jpg

We have trimmed the image back to where detail apears, more or less. But the bottom of the image doesn't appear to have any detail. It is included because of red circle number 2, which is caused by a small patch of white on the otherwise black leggings, which isn't visible in the reduced image.

Application: simple image alignment

When we want to align one image to another, we generally start by matching points in one image with points in the other. We can use one of the above methods to identify features in one image, ie areas of high variation, high entropy. These are likely to be distinctive in the other image, reducing false matches.

This is developed on the Simple alignment by matching areas page.

Cleanup

We don't need to keep all those large TIFF or MIFF files, so delete them.

rem del dt_*_*.tiff
del dt_stddev.tiff
del dt_*_*.miff
del %TEMP%\dt_*nl.miff
del %TEMP%\dt_*nl_dbgdrw.txt

Scripts

For convenience, .bat scripts are also available in a single zip file. See Zipped BAT files.

StopWatch.bat

@set STOP_WATCH_SUB=
@if not "%STOP_WATCH_TIME%"=="" set STOP_WATCH_SUB=/S "%STOP_WATCH_TIME%"

@cDate %STOP_WATCH_SUB% /f"\q \H:\M:\s"

@for /F "usebackq tokens=*" %%L IN (`cDate /f"\d-\b-\Y \H:\M:\s"`) do @set STOP_WATCH_TIME=%%L
@rem echo %STOP_WATCH_TIME%

@if not "%1"=="" @echo %STOP_WATCH_TIME% %1 %2 %3 %4 %5 %6 %7 %8 %9>>stopwatch.lis

getPointsize.bat

rem %1 is image, %2 is height in pixels, sets gpPointsize to height in points.

call %PICTBAT%getDpi %1

if "%gdDpi%"=="0" (
  set gpPointsize=%2
) else (
  for /F "usebackq" %%L in (`%IM%identify -format "gpPointsize=%%[fx:%2*%gdDpi%/72]" xc:`) do set %%L
)

getDpi.bat

for /F "usebackq" %%L in (`%IM%identify -format "gdDpi=%%x\ngdUnits=%%U" %1`) do set %%L

if /I "%gdUnits%"=="PixelsPerCentimeter" (
  for /F "usebackq" %%L in (`%IM%identify -format "gdDpi=%%[fx:%gdDpi%*2.54]" xc:`) do set %%L
)

nLightest.bat

@rem From image %1, finds (%2) lightest points.
@rem
@rem After each point is found,
@rem    blacks out pixels within radius (r) of that point
@rem where r = min(width,height) * %3/100
@rem %3 defaults to 10 (percent).
@rem
@rem Returns number of points found. Could be < n, or even zero (?).
@rem
@rem Also uses:
@rem   nlDEBUG if 1, creates debugging image.
@rem   nlPOINTSIZE optional, pointsize for debug images.
@rem   nlSTROKEWIDTH for debug.
@rem   nlONLY_WHITE if 1, doesn't auto-level,
@rem      so finds only points that are exactly white.
@rem
@rem Returns:
@rem   nl_nFound number of points found, integer >= 0
@rem   echos list of coordinates.
@rem
@rem Updated:
@rem   24-May-2016 v7 needs -channel RGB for -auto-level.



@if "%1"=="" findstr /B "rem @rem" %~f0 & exit /B 1

@setlocal enabledelayedexpansion

@call echoOffSave

call %PICTBAT%setInOut %1 nl

for /F "usebackq" %%L in (`cygpath %TEMP%`) do set CYGTEMP=%%L

@set TMPFILE=%TEMP%\%~n1_%sioCODE%.miff
@set CYGTMPFILE=%CYGTEMP%\%~n1_%sioCODE%.miff
@set LISTFILE=%BASENAME%_%sioCODE%.lis
@set TMPDEBUGFILE=%BASENAME%_%sioCODE%_dbg.miff
@set DEBUGFILE=%BASENAME%_%sioCODE%_dbg%EXT%
@set DEBUGDRAW=%TEMP%\%~n1_%sioCODE%_dbgdrw.txt

del %DEBUGDRAW% 2>nul

if "%nlPOINTSIZE%"=="" set nlPOINTSIZE=20

if "%nlSTROKEWIDTH%"=="" set nlSTROKEWIDTH=1

set MAX_FIND=%2
if "%MAX_FIND%"=="" set MAX_FIND=10

set PC_RAD=%3
if "%PC_RAD%"=="" set PC_RAD=10

set AUTOLEV=-channel RGB -auto-level +channel
if "%nlONLY_WHITE%"=="1" set AUTOLEV=

for /F "usebackq" %%L in (`%IM%convert ^
  -ping ^
  %INFILE% ^
  -format "PIX_RAD=%%[fx:int(min(w,h)*%PC_RAD%/100+0.5)]" ^
  info:`) do set %%L

%IM%convert %INFILE% -colorspace Gray %AUTOLEV% %TMPFILE%

if "%nlDEBUG%"=="1" %IM%convert %INFILE% %TMPDEBUGFILE%

set /A nFound=0

:loop

set MAX=
set whiteX=

for /F "usebackq tokens=1-3 delims=:, " %%W ^
in (`%IMDEV%convert ^
    %TMPFILE% ^
    -process onewhite ^
    NULL: 2^>^&1`) ^
do (
  if "%%W"=="onewhite" (
    set MAX=%%W
    set whiteX=%%X
    set whiteY=%%Y
    set /A whiteX2=%%X+%PIX_RAD%
  )
)

if "!MAX!"=="onewhite" if "!whiteX!" neq "none" (
  %IM%convert ^
    %TMPFILE% ^
    -fill #000 -draw "circle %whiteX% %whiteY% %whiteX2% %whiteY%" ^
    %AUTOLEV% ^
    %TMPFILE%

  if "%nlDEBUG%"=="1" (
    (
      echo fill None stroke #f00 circle %whiteX%,%whiteY%,%whiteX2%,%whiteY%
      echo fill #f00 stroke None text %whiteX%,%whiteY% '!nFound!'
    ) >>%DEBUGDRAW%
  )

  set /A nFound+=1
  echo %whiteX%,%whiteY%
  if !nFound! LSS %MAX_FIND% goto loop
)

rem if "%nlDEBUG%"=="1" %IM%convert %TMPDEBUGFILE% %DEBUGFILE%

if "%nlDEBUG%"=="1" if exist %DEBUGDRAW% %IM%convert ^
  %INFILE% ^
  -strokewidth %nlSTROKEWIDTH% ^
  -pointsize %nlPOINTSIZE% ^
  -draw @%DEBUGDRAW% ^
  %DEBUGFILE%


call echoRestore

@endlocal & set nl_nFound=%nFound%& set nlDEBUG_FILE=%DEBUGFILE%

slopeMag.bat

This script is based on Convolution of Images: Directional Kernels, under "vector addition of the two X and Y derivatives".

Each Sobel gives grey where there is no detail, and black or white at maximum detail. Solarize and level make this black where no detail, and white at maximum detail. Calling these X and Y, we then set result = sqrt(X^2+Y^2). This could give a result of 2. To ensure the result is <=1, we initially multiply both X and Y by sqrt(2)/2 = 0.7071 before squaring.

Bias 50% puts no-detail at mid-grey, so we also normalize the kernel and scale it by 50%.

rem Slope magnitude
rem 
rem Returns the magnitude of the slope.
@rem
@rem  If %2 is given, uses this as the output filename.
@rem
@rem Also uses:
@rem   smAUTO if 0, doesn't "-auto-level -auto-gamma"


@if "%1"=="" findstr /B "rem @rem" %~f0 & exit /B 1

@setlocal

@call echoOffSave

call %PICTBAT%setInOut %1 sm

if not "%2"=="" set OUTFILE=%2

if "%smAUTO%"=="0" (
  set AUTO=
) else (
  set AUTO=-auto-level -auto-gamma
)

rem This works best with Q32 HDRI.

if "%IM32f%"=="" call %PICTBAT%setIm8.bat

%IM32f%convert ^
  %INFILE% ^
  -alpha off ^
  -define convolve:scale="50%%^!" -bias 50%% ^
  ( -clone 0 -morphology Convolve Sobel:0 ) ^
  ( -clone 0 -morphology Convolve Sobel:90 ) ^
  -delete 0 -solarize 50%% -level 50,0%% ^
  +level 0,70.71067811865475%% ^
  -evaluate Pow 2 ^
  -compose plus -composite  ^
  -evaluate Pow 0.5 ^
  -separate -evaluate-sequence Max ^
  %AUTO% ^
  +depth ^
  -define quantum:format=integer ^
  %OUTFILE%

if ERRORLEVEL 1 exit /B 1

call echoRestore

endlocal & set smOUTFILE=%OUTFILE%

All images on this page were created by the commands shown, using:

%IM%identify -version
Version: ImageMagick 6.9.5-3 Q16 x86 2016-07-22 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2015 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Visual C++: 180040629
Features: Cipher DPC Modules OpenMP 
Delegates (built-in): bzlib cairo flif freetype jng jp2 jpeg lcms lqr openexr pangocairo png ps rsvg tiff webp xml zlib

Source file for this web page is detail.h1. To re-create this web page, run "procH1 detail".


This page, including the images, is my copyright. Anyone is permitted to use or adapt any of the code, scripts or images for any purpose, including commercial use.

Anyone is permitted to re-publish this page, but only for non-commercial use.

Anyone is permitted to link to this page, including for commercial use.


Page version v1.0 29-May-2014.

Page created 11-Dec-2016 05:25:07.

Copyright © 2016 Alan Gibson.