snibgo's ImageMagick pages

Alignment by Gaussian pyramid

Brute-force alignment of corresponding levels of Gaussian pyramids finds a good translation-only alignment fairly quickly.

The following are building-blocks for this page:

Sample images

All work on this page is performed on full-size camera images, about 7500x5000 pixels. The results are shrunk and converted to JPG for this web page.

The camera was hand-held. Auto-exposure was used, so the exposure for overlapping parts of the photos varies slightly.

set WEB_SIZE=-resize 600x400
set SRCDIR=%PICTLIB%20151008\

rem set SRC1=%SRCDIR%AGA_2595.JPG

set SRC1=algs_src1.jpg

if not exist %SRC1% copy %SRCDIR%AGA_2595.JPG %SRC1%

%IMG7%magick %SRC1% %WEB_SIZE% algs_src1_sm.jpg
rem set SRC2=%SRCDIR%AGA_2596.JPG

set SRC2=algs_src2.jpg

if not exist %SRC2% copy %SRCDIR%AGA_2596.JPG %SRC2%

%IMG7%magick %SRC2% %WEB_SIZE% algs_src2_sm.jpg

The process

Aligning two images by Gaussian pyramid works roughly as a human would do the job. The images are (mentally) simplified, and these simplified versions are aligned. Then the process is refined, using more detail from the original images, until we finally align at the pixel level.

We start by making pyramids from the images. The level (smallest) level of the pyramid needs to contain a reasonable number of pixels. If it is too small, we get false matches, and these errors will be propagated down the lower levels. Large sizes will be more reliable, but slower. I find that a minimum size of 10 in each direction is generally sufficient, and (so far) 25 has always been sufficient.

set pyMIN_BLK_WH=25
set pyPREFIX=algs_pyr_

call %PICTBAT%mkGausPyr %SRC1% algs_g1.tiff
set pyWR_VAR=1
call %PICTBAT%mkGausPyr %SRC2% algs_g2.tiff
set pyWR_VAR=
call %PICTBAT%alignGausPyr algs_g1.tiff algs_g2.tiff

That script has written a text file, and has put the name of the file into environment variable %agpDATAFILE%.

agpDATAFILE=abf_data3.csv agpDODGY=1 

The text file now contains this:


We can read that file into environment variables:

for /F "tokens=1-9 delims=," %%A in (%agpDATAFILE%) do (
  set OV_COMP=%%A
  set OV_X=%%B
  set OV_Y=%%C
  set OV_W=%%D
  set OV_H=%%E
  set OV_X1=%%F
  set OV_Y1=%%G
  set OV_X2=%%H
  set OV_Y2=%%I

This gives us the width and height of the overlap, and the offsets of the overlap in the two images.

echo OV_W=%OV_W% OV_H=%OV_H% OV_X1=%OV_X1% OV_Y1=%OV_Y1% OV_X2=%OV_X2% OV_Y2=%OV_Y2% 
OV_W=3533 OV_H=4752 OV_X1=3827 OV_Y1=160 OV_X2=0 OV_Y2=0 

From that, we can crop the overlaps from the two images:

%IMG7%magick ^
  %SRC1% ^
  -crop %OV_W%x%OV_H%+%OV_X1%+%OV_Y1% ^
  +repage ^
  +write algs_over1.miff ^
  %WEB_SIZE% ^
%IMG7%magick ^
  %SRC2% ^
  -crop %OV_W%x%OV_H%+%OV_X2%+%OV_Y2% ^
  +repage ^
  +write algs_over2.miff ^
  %WEB_SIZE% ^

We see that there is no major problem. Visually, the two images roughly match. A Difference composite shows us more clearly whether the match is exact:

%IMG7%magick ^
  algs_over1.miff ^
  algs_over2.miff ^
  -compose Difference -composite ^
  -auto-level ^
  %WEB_SIZE% ^

The variables OV_X and OV_Y are the offsets of the second image with respect to the first.

echo OV_X=%OV_X% OV_Y=%OV_Y% 
OV_X=3827 OV_Y=160 

We can use this to join the images into a panorama:

%IMG7%magick ^
  %SRC1% ^
  ( %SRC2% ^
    -page +%OV_X%+%OV_Y% ^
  ) ^
  -layers Mosaic ^
  %WEB_SIZE% ^


The returned score is the simple RMSE. In real life, we would give more weight when matching images contain significant detail rather than, say, a clear blue sky matching another clear blue sky. The scripts may be enhanced to provide this.

The process works, but it is slow: with 7400x5000 pixel images, 8 minutes on my computer.

See also


For convenience, .bat scripts are also available in a single zip file. See Zipped BAT files.


rem Find translation-only alignment of two Gaussian pyramids %1 and %2 with same structure.
rem Assumes %pyPREFIX% is the prefix of an appropriate blk.lis file.
@rem Optional:
@rem   %3 Factor for initial search window eg 0.66.
@rem Updated:
@rem   22-August-2022 Upgraded for IM v7.

@if "%2"=="" findstr /B "rem @rem" %~f0 & exit /B 1

@setlocal enabledelayedexpansion

@call echoOffSave

call %PICTBAT%setInOut %1 agp

set SRC1=%1
set SRC2=%2

set SRCH_WIND=%3
if "%SRCH_WIND%"=="." set SRCH_WIND=
if "%SRCH_WIND%"=="" set SRCH_WIND=0.6667

if "%pyPREFIX%"=="" (
  echo %0: pyPREFIX not set
  exit /B 1

for /F "tokens=*" %%L in (%pyPREFIX%blk.lis) do set %%L

if "%NUM_OCTAVES%"=="" (
  echo %0: NUM_OCTAVES not set in %pyPREFIX%blk.lis
  exit /B 1

set /A iOct=%NUM_OCTAVES%-1

for /F "usebackq" %%L in (`%IMG7%magick identify ^
  -format "SrchW=%%[fx:int(!N_BLK_W.%iOct%!*%SRCH_WIND%+0.5)]\nSrchH=%%[fx:int(!N_BLK_H.%iOct%!*%SRCH_WIND%+0.5)]" ^
  xc:`) do set %%L

rem Get dims multiplied by magic overlap factor for first search.

call %PICTBAT%alignBF ^
  %SRC1%[%iOct%] %SRC2%[%iOct%] ^
  !N_BLK_W.%iOct%! !N_BLK_H.%iOct%! -%SrchW% -%SrchH% %SrchW% %SrchH%

set DODGY=%abfDODGY%

call :ReadBlkLis

for /L %%i in (%iOct%,-1,1) do (

  echo %~n0: BESTX=!BESTX! BESTY=!BESTY!
  echo %~n0: N_BLK_W.%%i=!N_BLK_W.%%i!
  echo %~n0: N_BLK_H.%%i=!N_BLK_H.%%i!

  set /A PrevOct=%%i-1

  call :GetPrev %%i !PrevOct!

  echo %~n0: FirstX=!FirstX! FirstY=!FirstY! LastX=!LastX! LastY=!LastY!

  call %PICTBAT%alignBF ^
    %SRC1%[!PrevOct!] %SRC2%[!PrevOct!] ^
    !BW! !BH! !FirstX! !FirstY! !LastX! !LastY!

  if "!abfDODGY!"=="1" set DODGY=1

  call :ReadBlkLis

echo %~n0: COMP=%COMP% abfDATAFILE=%abfDATAFILE%

call echoRestore

@endlocal & set agpDATAFILE=%abfDATAFILE%& set agpDODGY=%DODGY%

exit /B 0

::---------------- Subroutines ----------------

  rem type %abfDATAFILE%

  for /F "tokens=1-9 delims=," %%A in (!abfDATAFILE!) do (
    set COMP=%%A
    set BESTX=%%B
    set BESTY=%%C
    set W=%%D
    set H=%%E
    set X1=%%F
    set Y1=%%G
    set X2=%%H
    set Y2=%%I
  echo %~n0: !COMP! !X! !Y! X1=!X1! Y1=!Y1! X2=!X2! Y2=!Y2!

  exit /B

  echo %~n0: N_BLK_W.%2=!N_BLK_W.%2!
  echo %~n0: N_BLK_H.%2=!N_BLK_H.%2!

  set /A BW=!N_BLK_W.%2!
  set /A BH=!N_BLK_H.%2!
  set /A FirstX=^(!BESTX!-1^)*!N_BLK_W.%2!/!N_BLK_W.%1!
  set /A FirstY=^(!BESTY!-1^)*!N_BLK_H.%2!/!N_BLK_H.%1!
  set /A LastX=^(!BESTX!+1^)*!N_BLK_W.%2!/!N_BLK_W.%1!
  set /A LastY=^(!BESTY!+1^)*!N_BLK_H.%2!/!N_BLK_H.%1!

  exit /B

All images on this page were created by the commands shown, using:

%IMG7%magick -version
Version: ImageMagick 7.1.0-47 Q16-HDRI x64 15861e0:20220827
Copyright: (C) 1999 ImageMagick Studio LLC
Features: Cipher DPC HDRI OpenCL 
Delegates (built-in): bzlib cairo freetype gslib heic jng jp2 jpeg jxl lcms lqr lzma openexr pangocairo png ps raqm raw rsvg tiff webp xml zip zlib
Compiler: Visual Studio 2022 (193331629)
%IM7DEV%magick -version
Version: ImageMagick 7.1.0-20 Q32-HDRI x86_64 2021-12-29
Copyright: (C) 1999-2021 ImageMagick Studio LLC
Features: Cipher DPC HDRI Modules OpenMP(4.5) 
Delegates (built-in): bzlib cairo fontconfig fpx freetype jbig jng jpeg lcms ltdl lzma pangocairo png raqm rsvg tiff webp wmf x xml zip zlib
Compiler: gcc (11.2)

To improve internet download speeds, some images may have been automatically converted (by ImageMagick, of course) from PNG or TIFF or MIFF to JPG.

Source file for this web page is aligngaus.h1. To re-create this web page, run "procH1 aligngaus".

This page, including the images, is my copyright. Anyone is permitted to use or adapt any of the code, scripts or images for any purpose, including commercial use.

Anyone is permitted to re-publish this page, but only for non-commercial use.

Anyone is permitted to link to this page, including for commercial use.

Page version v1.0 20-March-2016.

Page created 25-Sep-2022 04:54:56.

Copyright © 2022 Alan Gibson.