Poisson image editing, and related techniques.
This page describes the Poisson image editing technique for blending two or more images. I implement the basic Poisson process as a fairly simple script, relaxFillMS.bat. However, this can be used in a number of ways.
This page is concerned with images, not mathematics. (I am a picture maker, not a mathematician.) Interested readers may feast on the mathematics in the references.
CAUTION: In v1.0 of this page (23October2014), slope and divergence images were offset so zero was represented by pixel values of 50%.
However, this caused problems due to the Fifty percent issue, so now zero slopes are represented by pixel values of 0%, and negative values are common, and HDRI should be used.
After I wrote this page in 2017, IM has implemented operations "compose seamlessblend composite" and "compose saliencyblend composite". See Seamlessblend and Saliencyblend below.
This page draws heavily on:
Other related papers, though not directly considered on this page:
This page uses a version of IM assumed to be HDRI.
We use two images from the following paper:
set BEAR=perez_bear.jpg 

set WATER=perez_water.jpg 
The images are not my copyright. I think the animal is a bear, but I could be wrong.
The goal is to cutout the bear and paste it over the water. The water in the foreground bear photo is generally darker than the required background.
For the cutout, we will use a mask that I made with Gimp, roughly following the example in the Pérez paper. It is white where we want the bear image; otherwise black.
set MASK=bear_mask.png rem goto skip 
We will offset the bear image:
set sOFFSET=+40+30 set sGEOM=geometry %sOFFSET%
Get the dimensions of the original bear image:
for /F "usebackq" %%L in (`%IMG7%magick identify ^ format "WW=%%w\nHH=%%h\nWm1=%%[fx:w1]\nHm1=%%[fx:h1]" ^ %BEAR%`) do set %%L echo WW=%WW% HH=%HH%
WW=297 HH=163
We can simply cut out the bear and composite over the water.
%IMG7%magick ^ %WATER% ^ ( %BEAR% %MASK% alpha off ^ compose CopyOpacity composite ^ ) ^ %sGEOM% ^ compose Over composite ^ spm_simpmont.jpg 
The boundary between the images is obvious. We want a seamless boundary. We could blur the boundary, but the difference in the water would still be obvious.
The basic method in the Pérez paper is called "Poisson interpolation", "seamless boundary by Poisson image editing" or even "seamless boundary by Poisson equation with Dirichlet boundary conditions". For short, I call it simply "Poisson pasting".
The simple version of the method (with no explicit "guidance vector field") adds an adjustment (or "correction") image to the foreground bear image before compositing it over the water background. The adjustment image has pixels corresponding to the mask boundary exactly equal to the water image minus the bear image. Hence, when added to the bear image, those pixels will be equal to the water background.
The pixels in the adjustment image outside the mask boundary won't be used so we don't care what values they have. The pixels inside the mask boundary do matter. Values should change smoothly, interpolating between values at the boundary. The smoother they change, the better the result. Put it another way: we should smooth the pixels of the adjustment image, and keep smoothing until they no longer change. This is exactly what relaxFillMS.bat does. (See Filling holes: relaxation.)
Calculate background minus foreground.
%IMG7%magick ^ %WATER% ^ %BEAR% ^ %sGEOM% ^ compose Mathematics ^ define compose:args=0,0.5,0.5,0.5 ^ composite ^ crop %WW%x%HH%%sOFFSET% +repage ^ +depth ^ +write spm_minus.png ^ ( %MASK% negate ) ^ alpha off compose CopyOpacity composite ^ spm_minus_mskd.png 

Fill the hole by relaxation. call %PICTBAT%relaxFillMS ^ spm_minus_mskd.png . spm_add1.png 

Add the result to the bear image.
%IMG7%magick ^ %BEAR% ^ spm_add1.png ^ compose Mathematics ^ define compose:args=0,2,1,1 ^ composite ^ +depth ^ +write spm_bear_adj.png ^ %MASK% alpha off ^ compose CopyOpacity composite ^ %WATER% ^ +swap ^ %sGEOM% ^ compose Over composite ^ spm_s2.png 
The result is good. We have the bear's reflection in the water, and the water's colour is taken from the correct image. Note that the water has lightened, but so has the bear. If we want to prevent the bear from becoming lightened, see Inner Dirichlet boundary below.
We can express the pixels that fill the hole as:
fill_pixels = bear + relax (water  bear)
At the boundary:
relax (water  bear) = water  bear
Hence, at the boundary:
fill_pixels = bear + water  bear = water
ASIDE
Instead of relaxFillMS.bat, we could use blurFill.bat. But this creates a discontinuity at the centre. For this example, the discontinuity is a roughly horizontal line that coincides with the base of the bear, so the problem it creates isn't visible.
call %PICTBAT%blurFill ^ spm_minus_mskd.png . spm_blr_add.png 
ASIDE 2
For the two mathematics composites, we could use compose:args=0,1,1,0.5 and compose:args=0,1,1,0.5 respectively. But if we don't use HDRI, this can clip the first result. The parameters used above have the same effect but ensure we have no clipping (at the expense of one bit of precision).
Above, we have adjusted the bear image to match the required background water. This has visibly lightened the bear's fur. Instead, we can adjust the water to match the bear.
%IMG7%magick ^ %WATER% ^ %BEAR% ^ %sGEOM% ^ compose Mathematics ^ define compose:args=0,0.5,0.5,0.5 ^ composite ^ +depth ^ +write spm_minus_r.png ^ alpha transparent ^ ( %MASK% alpha off ) ^ %sGEOM% ^ compose CopyOpacity composite ^ spm_minus_mskd_r.png 

call %PICTBAT%relaxFillMS ^ spm_minus_mskd_r.png . ^ spm_add1_r.png 1e6 1000 

%IMG7%magick ^ %WATER% ^ spm_add1_r.png ^ compose Mathematics ^ define compose:args=0,2,1,1 ^ composite ^ +depth ^ +write spm_bear_adj_r.png ^ ( %MASK% negate ) alpha off ^ %sGEOM% ^ set option:compose:outsideoverlay false ^ compose CopyOpacity composite ^ %BEAR% ^ %sGEOM% ^ compose DstOver composite ^ +depth ^ spm_s2_r.png 
So we can paste a foreground to a background with seamless boundary, either by adjusting the foreground or background. Of course, we could blend the two results.
The code above has used a boundary as exactly specified by a mask. Instead, a mask could define a range of possible boundaries, eg by describing a ring 20 pixels thick. Code could then find the darkest path around the ring, and we use that as the actual boundary.
%IMG7%magick ^ %MASK% ^ alpha off ^ morphology EdgeOut disk:20 ^ spm_ring_mask.png 
We use shapeDp.bat to find the darkest path in the difference between the images, around this ring. (See Awkward boundaries with dark paths.) This needs two input images and the ring mask, all the same size.
%IMG7%magick ^ %WATER% ^ crop %WW%x%HH%%sOFFSET% +repage ^ spm_back_crp.png 

set sdpBLUR_SIG=0 set sdpMASK_FILE=spm_dp_mask.png call %PICTBAT%shapeDp ^ spm_back_crp.png %BEAR% spm_ring_mask.png 
shapeDp.bat outputs the second image composited over the first, via a calculated mask. We use that calculated mask.
set MASK2=spm_dp_mask.png %IMG7%magick ^ %WATER% ^ %BEAR% ^ %sGEOM% ^ compose Mathematics ^ define compose:args=0,0.5,0.5,0.5 ^ composite ^ crop %WW%x%HH%%sOFFSET% +repage ^ +depth ^ +write spm_minus_rng.png ^ ( %MASK2% negate ) ^ alpha off ^ compose CopyOpacity composite ^ spm_minus_mskd_rng.png 

call %PICTBAT%relaxFillMS ^ spm_minus_mskd_rng.png . spm_add1_rng.png 

%IMG7%magick ^ %BEAR% ^ spm_add1_rng.png ^ compose Mathematics ^ define compose:args=0,2,1,1 ^ composite ^ +depth ^ +write spm_bear_adj_rng.png ^ %MASK2% alpha off ^ compose CopyOpacity composite ^ %WATER% ^ +swap ^ %sGEOM% ^ compose Over composite ^ +depth ^ spm_s2_rng.png 
The only noticable difference is that the water around the bear seems calmer. This is simply because the boundary is larger.
We implement the above in the script poissonPaste.bat. For simplicity, it requires the two input images and the mask to be the same size, with no offsets.
The script will adjust part of the second input image to match the first at the mask boundary. Using the mask, it then cuts out the second image and composites that over the first.
(Thus, for the common case where we adjust part of a small image to paste over a larger image, the relaxFillMS.bat wastes time blurring a large area that will be overwritten. The script could be optimised, eg by trimming the mask. Or we could have a wrapper that finds the trim of the mask, crops all three images to that, calls poissonPaste.bat, and finally reassembles the pieces.)
In this example, we extend the bear and mask to be the same size as the water.
%IMG7%magick ^ %WATER% ^ +depth ^ fill Black colorize 100 ^ ( clone 0 ^ %BEAR% ^ %sGEOM% ^ compose Over composite ^ +write spm_bear_ext.png ^ +delete ^ ) ^ ( clone 0 ^ %MASK% ^ %sGEOM% ^ compose Over composite ^ +write spm_mask_ext.png ^ +delete ^ ) ^ NULL: 
We can paste the bear over the water, or the water over the bear. The second image is adjusted to match the first, the "under" image, at the boundary and pasted over it.
Poissonpaste the bear over the water: call %PICTBAT%poissonPaste ^ %WATER% spm_bear_ext.png spm_mask_ext.png ^ spm_pp_out1.png 

Poissonpaste the water over the bear: call %PICTBAT%poissonPaste ^ spm_bear_ext.png %WATER% spm_mask_ext.png ^ spm_pp_out2.png 
In the next two examples, we negate the effect of the mask. As before, the second image is adjusted to match the first and pasted over it. But the adjustment and pasting is where the mask is black, instead of white.
Poissonpaste the bear under the water: call %PICTBAT%poissonPaste ^ %WATER% spm_bear_ext.png spm_mask_ext.png ^ spm_pp_out3.png 1 

Poissonpaste the water under the bear: call %PICTBAT%poissonPaste ^ spm_bear_ext.png %WATER% spm_mask_ext.png ^ spm_pp_out4.png 1 
An improved boundary for the Poisson pasting (relaxFillMs.bat) can be found by isolating the bear from its background.
Theorem: A 3D object with a constant colour, illuminated by white light, appears in a photograph to have varying lightness, somewhat varying saturation (or chroma), but constant hue.
Corollary: if we want to segment a photograph by objects, the hue channel is all we need.
In real life, objects rarely have exactly constant colour. And they are illuminated not merely by the main light source but also by light reflected by other objects, and this reflected light will be coloured. This is particularly evident when a lowsaturation (grayish) object is close to a highsaturation object (eg bright red). The gray object easily picks up red light from the reflected red object.
And objects that have very low saturation often have widely varying hues, eg a gray object may have adjacent pixels that are sRGB(50.001%,50%,50%) and sRGB(50%,50%,50.001%). This happens at the bear's nose, which is nearly black.
And reducing an image to just hues can cause objects that are clearly distinct to merge into each other.
But it is often a useful first approximation.
Here, we set the Lightness of all pixels to 50%. We could set all Chroma to 100%, but this gives false colours to pixels at the nose, so we set very low Chroma to zero, other Chroma to 100%, with a short ramp between them.
%IMG7%magick ^ %BEAR% ^ colorspace HCL ^ channel 1 level 3,5%% ^ channel 2 evaluate set 50%% ^ +channel ^ colorspace sRGB ^ spm_hue.png 
We want just two colours. In general, when we want just (N) colours, using "colors N" doesn't work well because many pixels we would want in in a particular group actually fall into the other. By using "colors N+x", virtually all the pixels allocated to one of the groups will be in the correct group. If N+x is too large, we fragment the natural groups too much that the next step won't work. N+2 often works well, when N is small.
%IMG7%magick ^ spm_hue.png ^ +dither colors 4 ^ spm_hue_4.png 
The sea has essentially two colours. Where the water is tilted up by a wave towards the camera, we see the colour of the water, which is somewhat green. Where the water is tilted away, the far side of a wave, we can't see into the water and instead see reflected light from the sky, which is more blue.
The sea also has a heavily distorted reflection and refraction of the bear. When reduced to four colours, a couple of small shapes have the bear's colour.
The script connCompLimArea.bat removes very small components (here, those that are less than 0.1% of the total image area).
call %PICTBAT%connCompLimArea ^ spm_hue_4.png spm_hue_4a.png 0.1c 
The script connCompLimNum.bat here limits the image to two components, mapping the other pixels to the colours of those two components. We have successfully lassooed the bear. Starting from the bear surrounded by water, we have found an outline for the bear.
call %PICTBAT%connCompLimNum ^ spm_hue_4a.png spm_hue_4a2.png 2 
(An alternative method for simplifying spm_hue_4.png is: find the unique colours in the four edges, remap the image to these; any pixels that have changed were a different colour to all edge pixels.)
Change the colours so we have a white background and black object.
%IMG7%magick ^ spm_hue_4a2.png ^ fill White ^ draw "color 0,0 floodfill" ^ fill Black +opaque White ^ spm_hue_bw.png 
Combine with the usersupplied mask:
%IMG7%magick ^ spm_hue_bw.png ^ %MASK% ^ compose Darken composite ^ spm_hue_bw2.png 
Now we have an outer and inner bound for a minimum error boundary cut (MEBC). The outer bound has been supplied by the user; the inner bound is from the lassoo. We find the optimum boundary within those limits.
set sdpBLUR_SIG=0 set sdpMASK_FILE=spm_dp_mask2.png call %PICTBAT%shapeDp ^ spm_back_crp.png %BEAR% spm_hue_bw2.png ^ spm_ring_mask2.png 
We can now use Poisson pasting with this mask.
Poissonpaste the bear over the water: %IMG7%magick ^ %WATER% ^ +depth ^ fill Black colorize 100 ^ spm_dp_mask2.png ^ %sGEOM% ^ compose Over composite ^ +write spm_mask2_ext.png call %PICTBAT%poissonPaste ^ %WATER% spm_bear_ext.png spm_mask2_ext.png ^ spm_pp_out5.png 
(Alternative: use Pp with spm_hue_bw.png, possibly dilated.)
See DragandDrop Pasting Jiaya Jia, Jian Sun, ChiKeung Tang, HeungYeung Shum, 2009.
poissonPaste.bat adjusts one image to match another at their boundary, by adding an adjustment image. The adjustment image is a relaxed ("membranised") version of the difference between the two input images. If the input images were identical at the boundary, the adjustment image would be zero at the boundary, so the relaxation would also be black and there would be no adjustment (nor any need for an adjustment). If the boundary pixels in the adjustment image were identical to each other, the relaxation would be a constant, flat colour, so the adjustment would be a constant colourshift.
We see this in the above example: the bear is lightened by the process.
Difficulties arise when one of the inputs contains detail (i.e. gradients) at the boundary that the other doesn't. This creates a variation in the boundary pixels of the adjustment image, and this variation is propagated somewhat into the adjustment, which causes localised colourshifts.
This isn't obvious in the above examples because both images have variation at the boundary, and the difference in the variation is of a similar magnitude. We can construct an artificial example:
%IMG7%magick ^ %BEAR% ^ crop 2x2@ ^ ( clone 0 fill White colorize 100 ) ^ delete 0 ^ ( clone 0 fill Lime colorize 100 ) ^ delete 0 ^ ( clone 0 fill Red colorize 100 ) ^ delete 0 ^ ( clone 0 fill Black colorize 100 ) ^ delete 0 ^ layers merge ^ spm_chq.png 

call %PICTBAT%poissonPaste ^ spm_chq.png %BEAR% %MASK% ^ spm_chq_out.png 
The four background colours have clearly propagated into the bear. In this example, no alternative boundary is available, so we can't avoid the effect.
When we have a choice of boundaries, represented in examples above by a white ring, we have chosen the path that minimises the total difference between the input images.
Instead, we can choose the path that minimises the difference in the slopes (a.k.a. gradients; see the Slopes page) of the inputs. For both inputs, we calculate the slopes and record x and ydeltas for each colour channel in two images (because we need six channels). For each channel we then calculate the difference between the slopes, and use the maximum magnitude of these as the input to the darkestpntpnt process module.
(Instead of using six channels, we could instead use the slopes of the grayscaled images, needing just two channels.)
For the slope*.bat scripts, see the Slopes page.
Below, "sigmoidalcontrast" and "autolevel" are merely to aid visibility.
Get the slope of the bear image. call %PICTBAT%slopeXY ^ %BEAR% spm_bear_slp.miff %IMG7%magick spm_bear_slp.miff[0] ^ evaluate Divide 2 ^ evaluate Add 50%% ^ sigmoidalcontrast 10,50%% ^ spm_bear_slp_0.jpg %IMG7%magick spm_bear_slp.miff[1] ^ evaluate Divide 2 ^ evaluate Add 50%% ^ sigmoidalcontrast 10,50%% ^ spm_bear_slp_1.jpg 

Get the slope of the water image. call %PICTBAT%slopeXY ^ spm_back_crp.png spm_water_slp.miff %IMG7%magick spm_water_slp.miff[0] ^ evaluate Divide 2 ^ evaluate Add 50%% ^ sigmoidalcontrast 10,50%% ^ spm_water_slp_0.jpg %IMG7%magick spm_water_slp.miff[1] ^ evaluate Divide 2 ^ evaluate Add 50%% ^ sigmoidalcontrast 10,50%% ^ spm_water_slp_1.jpg 

Subtract: slope(water)  slope(bear) call %PICTBAT%slopeXYminus ^ spm_bear_slp.miff spm_water_slp.miff ^ spm_minus_slp.miff %IMG7%magick spm_minus_slp.miff[0] ^ evaluate Divide 2 ^ evaluate Add 50%% ^ sigmoidalcontrast 10,50%% ^ spm_minus_slp_0.jpg %IMG7%magick spm_minus_slp.miff[1] ^ evaluate Divide 2 ^ evaluate Add 50%% ^ sigmoidalcontrast 10,50%% ^ spm_minus_slp_1.jpg 

Get the magnitude of the slope difference. call %PICTBAT%slopeXYmag ^ spm_minus_slp.miff ^ spm_mag_slp.miff %IMG7%magick spm_mag_slp.miff ^ autolevel ^ spm_mag_slp.jpg 

Find the line of minimum total magnitude within the ring. call %PICTBAT%minDp ^ spm_mag_slp.miff spm_hue_bw2.png ^ spm_mag_msk.png 

Poissonpaste. call %PICTBAT%poissonPaste ^ spm_back_crp.png %BEAR% spm_mag_msk.png ^ spm_mag_out.png 
In some ways, this result is more accurate than previous results, creating less bleeding of features from the background to the new foreground. However, the boundary in this example is also tighter, closer to the bear. We have less of the bear's reflection in the water.
Ordinary (nonguided) Poisson Image Editing creates a result that has a gradient very close to g, the required foreground ("source"). The result is made by adding a membrane to the foreground, where the membrane was made by relaxing a boundary of (background minus foreground).
Guided Poisson Image Editing creates a result that has a gradient very close to v, called a "guidance vector field". v is set to some function of g and f* (the original background). The result is made from the membrane itself, not added to anything, and the membrane is made by relaxing a boundary of simply the background. But the method of relaxation is more complicated.
The guidance vector field might come from the gradient of a single image, or from a mixture of gradients of different images (aka mixed gradients).
Below, spm_ws_div.miff is the right hand side of the equation:
f(x1,y) + f(x+1,y) + f(x,y1) + f(x,y+1)  4*f(x,y) = rhs(x,y)
So each iteration is:
f(x,y) = f(x1,y) + f(x+1,y) + f(x,y1) + f(x,y+1)  rhs(x,y) 4 4
In IM terms: convolve once, subtract something, convolve again, subtract the same thing, and keep going.
For example, we take two other images from the Pérez paper:
A scribble image. set SCRIBBLE=perez_scribble.jpg 

A wall image, cropped to the same size as the scribble. %IMG7%magick ^ %SCRIBBLE% ^ perez_wall.jpg ^ compose Over composite ^ spm_wall_crp.png set WALL=spm_wall_crp.png 

Make a mask in Gimp, roughly the same as the Pérez paper. set WS_MASK=perez_ws_mask.png 
For the result: where the mask is black, we want the wall. Where the mask is white, we want the gradient of either the wall or the scribble, whichever is more significant at that pixel.
To do that, we find the gradients (on x and y directions) of both images; find the magitudes of each gradient; find which magnitude is greater; for each pixel, use the slope of the image with the greatest magnitude; find the divergence of that.
Slope of the scribble. call %PICTBAT%slopeXY ^ %SCRIBBLE% spm_scrb_sxy.miff %IMG7%magick ^ spm_scrb_sxy.miff[0] ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_scrb_sxy_0.png %IMG7%magick ^ spm_scrb_sxy.miff[1] ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_scrb_sxy_1.png 

Slope of the wall. call %PICTBAT%slopeXY ^ %WALL% spm_wall_sxy.miff %IMG7%magick ^ spm_wall_sxy.miff[0] ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_wall_sxy_0.png %IMG7%magick ^ spm_wall_sxy.miff[1] ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_wall_sxy_1.png 

Magnitude of the scribble slope. call %PICTBAT%slopeXYmag ^ spm_scrb_sxy.miff spm_scrb_mag.miff 

Magnitude of the wall slope. call %PICTBAT%slopeXYmag ^ spm_wall_sxy.miff spm_wall_mag.miff 

Which magnitude is greatest? White for scribble, black for wall. %IMG7%magick ^ spm_wall_mag.miff spm_scrb_mag.miff ^ colorspace Gray ^ compose MinusDst composite ^ threshold 0 ^ spm_ws_mag_diff.png 

Use the slope that has the greatest magnitude. rem FIXME: Can't use mpr?? %IMG7%magick ^ define compose:clamp=off ^ ( spm_wall_sxy.miff[0] ^ spm_scrb_sxy.miff[0] ^ spm_ws_mag_diff.png +write mpr:MSK ^ alpha off ^ compose Over composite ^ ) ^ ( spm_wall_sxy.miff[1] ^ spm_scrb_sxy.miff[1] ^ spm_ws_mag_diff.png ^ alpha off ^ compose Over composite ^ ) ^ define quantum:format=floatingpoint ^ spm_ws_mxslp.miff %IMG7%magick ^ spm_ws_mxslp.miff[0] ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_ws_mxslp_0.png %IMG7%magick ^ spm_ws_mxslp.miff[1] ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_ws_mxslp_1.png 

Find the divergence. call %PICTBAT%slopeXYdiv ^ spm_ws_mxslp.miff spm_ws_div.miff %IMG7%magick ^ spm_ws_div.miff ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_ws_div.jpg This will be subtracted after each blur. 
This divergence is the required slope of the slope of the result.
With the mask, make a hole in the wall. %IMG7%magick ^ %WALL% ^ ( %WS_MASK% negate ) ^ alpha off ^ set option:compose:outsideoverlay false ^ compose CopyOpacity composite ^ spm_wall_msk.miff 
We relaxfill the hole, using the divergence as the guidance field.
call %PICTBAT%relaxFillMS ^ spm_wall_msk.miff . spm_wall_msk_rfs2.miff ^ 0.001 1000 . spm_ws_div.miff 0 00:06:50 
This is essentially the same result as in the Pérez paper. The blue scribble has become purple, i.e. it has increased in redness. This is because the wall is redder than the scribble, at the boundary.
We can test the "null" fill by calculating the div of the wall, and using that as the guidance to fill the hole in the wall.
Find the divergence. call %PICTBAT%slopeXYdiv ^ spm_wall_sxy.miff spm_wall_div.miff %IMG7%magick ^ spm_wall_div.miff ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_wall_div.jpg This will be subtracted after each blur. 

Relaxfill with this divergence. call %PICTBAT%relaxFillMS ^ spm_wall_msk.miff . spm_wall_fill_wall.miff ^ 0.001 1000 . spm_wall_div.miff We have reconstructed the wall. 
The result is very similar to the original wall image, but slightly blurred, as if the wall has been given "blur 0.46".
Other results:
Make a black image, and holed black image. %IMG7%magick ^ %WALL% ^ alpha off ^ fill Black colorize 100 ^ +depth ^ +write spm_ws_gray.png ^ ( %WS_MASK% negate ) ^ alpha off compose CopyOpacity composite ^ spm_ws_gray_m.png 

Relax the holed wall. call %PICTBAT%relaxFillMS ^ spm_wall_msk.miff . spm_wall_msk_rf.miff 

Relax the holed wall, guided by gray(50%), ie zero. call %PICTBAT%relaxFillScr ^ spm_wall_msk.miff . spm_wall_msk_gr_rfs.miff ^ 0.001 100 . spm_ws_gray.png The result is the same (but slower). 

Relax the holed gray(50%), guided by the divergence. call %PICTBAT%relaxFillScr ^ spm_ws_gray_m.png . spm_gray_rfs.miff ^ 0.001 100 . spm_ws_div.miff 

Poissonpaste, with userdefined mask. call %PICTBAT%poissonPaste ^ %WALL% %SCRIBBLE% perez_ws_mask.png ^ spm_ws_out2.png 

Poissonpaste, with maxmagnitude mask. call %PICTBAT%poissonPaste ^ %WALL% %SCRIBBLE% spm_ws_mag_diff.png ^ spm_ws_out4.png 
Another image from the Pérez paper. %IMG7%magick ^ perez_flower.jpg ^ crop 350x350+0+0 +repage ^ spm_flower.png set FLOWER=spm_flower.png 

A loose mask, made in Gimp. set FLR_MASK=flower_mask.png 
The mask is very loose, entirely surrounding the flower; we haven't bothered to try and capture just the tendrils.
The goal is to make the background (the nonflower) monochrome, without changing the flower.
Gray version of flower. %IMG7%magick ^ %FLOWER% ^ colorspace Gray ^ +depth ^ spm_flr_gr.png 

Gray version, with hole. %IMG7%magick ^ spm_flr_gr.png ^ ( %FLR_MASK% negate ) ^ alpha off ^ compose CopyOpacity composite ^ spm_flr_gr_h.png 

Slope of colour flower. call %PICTBAT%slopeXY ^ %FLOWER% spm_flr_sxy.miff %IMG7%magick ^ spm_flr_sxy.miff[0] ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_flr_sxy_0.png %IMG7%magick ^ spm_flr_sxy.miff[1] ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_flr_sxy_1.png 

Divergence of slope of flower. call %PICTBAT%slopeXYdiv ^ spm_flr_sxy.miff spm_flr_div.miff %IMG7%magick ^ spm_flr_div.miff ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_flr_div.png We will use this as the guidance field. 

Relaxfill the holed grayscale version, with guidance. call %PICTBAT%relaxFillMS ^ spm_flr_gr_h.png . spm_flr_bwc.png ^ 0.001 1000 . spm_flr_div.miff A good result, but the entire flower has become slightly gray. 
This is essentially the same result as shown in the Pérez paper.
If the result seems too good to be true, well, it is.
Where the mask is black, the result is grayscale, of course. Where the mask is white, the result is the colour input with green "subtracted" (speaking approximately). So the flower has become less green: values in the green channel have decreased, while values in the red and blue channels have increased. This has happened because that is the change at the boundary. Similarly, yellow colour has leached out from the flower to the grayscale areas between the tendrils. This is addressed in Inner Dirichlet boundary below.
The guidance field might be from the background image, but modulated in some way.
Yet another image from the Pérez paper:
set BOY=perez_boy.jpg 

With Gimp, make a mask that includes just the facial features. set BOY_MASK=boy_mask.png 

Get the slope of BOY. call %PICTBAT%slopeXY ^ %BOY% spm_boy_sxy.miff Modify the slope. %IMG7%magick ^ spm_boy_sxy.miff ^ fuzz 2%% fill Black opaque Black ^ define quantum:format=floatingpoint ^ spm_boy_sxy2.miff Get the divergence of the modified slope. call %PICTBAT%slopeXYdiv ^ spm_boy_sxy2.miff spm_boy_div2.miff %IMG7%magick ^ spm_boy_div2.miff ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_boy_div2.png 

Poissonpaste. call %PICTBAT%poissonPaste ^ %BOY% . %BOY_MASK% ^ spm_boy_out2.png ^ . spm_boy_div2.miff 
We have adjusted the slope so where it is within ±2% of 50% (i.e. zero), it becomes 50%. This eliminates small changes in slope. It has a sideeffect of reducing overall contrast, because slope accumulates to make contrast, [[ so we compensate very approximately with "level 80%%,80%%", but this is a kludge.]] Instead, we could match the result to the input image, using either imgGainBias.bat or matchHisto.bat. Another option would be to match the adjusted slope to the nonadjusted slope.
In the next example, after eliminating small slopes, we sharpen the remaining slopes. This increases local contrast.
Modify the slope. %IMG7%magick ^ spm_boy_sxy.miff ^ fuzz 2%% fill Black opaque Black ^ unsharp 0x3+1+0 ^ define quantum:format=floatingpoint ^ spm_boy_sxy3.miff Get the divergence of the modified slope. call %PICTBAT%slopeXYdiv ^ spm_boy_sxy3.miff spm_boy_div3.miff %IMG7%magick ^ spm_boy_div3.miff ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_boy_div3.png 

Poissonpaste. call %PICTBAT%poissonPaste ^ %BOY% . %BOY_MASK% ^ spm_boy_out3.png ^ . spm_boy_div3.miff 
This seems useful, so we put that in a script, poissonSmooth.bat. The script creates its own mask, which is black at the four edges, and otherwise white. At large values of sigma, it is effective at removing paper texture from scanned documents. This is a form of adaptive blur; see also Adaptive blur and sharpen.
call %PICTBAT%poissonSmooth ^ toes.png spm_psm1.png 1 

call %PICTBAT%poissonSmooth ^ toes.png spm_psm2.png 3 

call %PICTBAT%poissonSmooth ^ toes.png spm_psm3.png 5 

call %PICTBAT%poissonSmooth ^ toes.png spm_psm4.png 10 
A different effect is obtained by increasing the contrast of divergence, around the 50% level. This changes the slope of the slope of the result (increasing overall contrast), so a small change here goes a long way.
rem FIXME blah %IMG7%magick ^ spm_boy_div2.miff ^ evaluate Divide 2 evaluate Add 50%% ^ sigmoidalcontrast 2,50%% ^ evaluate Subtract 50%% evaluate Multiply 2 ^ define quantum:format=floatingpoint ^ spm_boy_div4.miff %IMG7%magick ^ spm_boy_div4.miff ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_boy_div4.png 

Poissonpaste. call %PICTBAT%poissonPaste ^ %BOY% . %BOY_MASK% ^ spm_boy_out4.png ^ . spm_boy_div4.miff 
More generally, poissonPaste.bat gives us a method of modifying the pixels in part of an image and blending the modified part into the original; an alternative to alpha blending.
See Image Editing without Color Inconsistency Using Modified Poisson Equation, Qin Chuan, Wang Shuozhong, Zhang Xinpeng, 2008.
In examples above, the colour difference at the boundary has influenced all result pixels inside the boundary, so the bear has become lightened, the scribble has become purple, and the flower has become less green. This is also known as "colour inconsistency" or "colour contamination". Sometimes we want this effect, but sometimes we don't.
The pixels to be filled already have an outer boundary condition, where the adjustment to be added is "water" minus "bear", so when this is added to "bear" the result at the outer boundary is "water". Chuan et al suggest we add an additional condition at the bear pixels, setting the adjustment to be added to "zero", so when this is added to "bear" the result at the inner boundary is "bear".
The script poissonPasteIDB.bat takes two masks, one for each of the outer and inner boundaries. The inner boundary mask is white where we want the bear unchanged, and black elsewhere.
Chuan et al suggest the user should create the bear mask. But we have created one automatically, so we can use that. We shrink spm_hue_bw.png by a few pixels. Black now has pixels that are in the bear. Add this to MASK, so now we have a white ring that is to be solved by Poisson equation.
The two masks together form what is often called a trimap; a map that labels some pixels as certainly inside the object, some as certainly outside, and others as unknown.
poissonPasteIDB does the following for nonguided Poissonpasting:
For guided pasting, poissonPasteIDB does the following:
As a happy sideeffect: the area to be filled is now the ring between the outer and inner masks. This ring shape is smaller than before, so fewer pixels need filling by relaxFillMS.bat, so this is faster than without an inner mask.
Create the inner boundary mask. %IMG7%magick ^ spm_hue_bw.png ^ morphology Dilate disk:3 ^ negate ^ spm_bear_sm.png 

call %PICTBAT%poissonPasteIDB ^ spm_back_crp.png %BEAR% ^ %MASK% spm_bear_sm.png ^ spm_idb_out.png 

Another example:
call %PICTBAT%poissonPasteIDB ^ spm_chq.png %BEAR% ^ %MASK% spm_bear_sm.png ^ spm_idb_out2.png 
The result has pixels at the outer boundary the same colour as the background, and pixels at the inner boundary the same colour as the bear. Hence we now have colour consistency.
We can do the same trick with the flower.
Create the inner boundary mask with Gimp.
set FLR_IN_MASK=flower_inner_mask.png %IMG7%magick ^ %FLR_IN_MASK% ^ alpha off ^ %FLR_IN_MASK% 

Check the masks. %IMG7%magick ^ %FLR_MASK% ^ ( %FLR_IN_MASK% negate ) ^ compose Darken composite ^ %FLOWER% ^ +swap ^ alpha off ^ compose CopyOpacity composite ^ alpha Background ^ +depth ^ spm_flr_chk.png 

Poissonpaste the grayscale version. call %PICTBAT%poissonPasteIDB ^ spm_flr_gr.png %FLOWER% ^ %FLR_MASK% %FLR_IN_MASK% ^ spm_flr_bwc3.png The tendrils have lost some colour, become whiter. 

Poissonpaste the grayscale version, with guidance. call %PICTBAT%poissonPasteIDB ^ spm_flr_gr.png %FLOWER% ^ %FLR_MASK% %FLR_IN_MASK% ^ spm_flr_bwc4.png ^ . spm_flr_div.miff The tendrils are a good colour, but have slightly blurred. 
However, the tendrils have become somewhat gray and the monochrome areas between tendrils have become somewhat yellow. Chuan et al suggest a solution. In the guidance field, where the pixels represent either the yellow flower or the grayscale background, the values are low (i.e. close to zero). At the border between the flower and background, the gradient field has relatively high values (i.e. more distant from zero). If we make the high values even higher, we will get a steeper gradient between the two, so less colour bleed. Where the gradient field is above a certain threshold, they increase the values by a certain factor. For the threshold, they use 80% of the maximum value M. For the factor, they use K. They calculate:
ForeIn: mean of foreground at the inner boundary
ForeOut: mean of foreground at the outer boundary
BackOut: mean of background at the outer boundary
K = ForeIn  BackOut ForeIn  ForeOut M = max(guidance_field)
I suppose the same multiplier is applied to all three channels. But what if K is less than one? Less than zero?
More simply, we can sharpen the guidance field:
Sharpen the guidance. %IMG7%magick ^ spm_flr_div.miff ^ unsharp 0x0.5+1+0.05 ^ define quantum:format=floatingpoint ^ spm_flr_div_u.miff %IMG7%magick ^ spm_flr_div_u.miff ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_flr_div_u.png 

Poissonpaste the grayscale version, with sharpened guidance. call %PICTBAT%poissonPasteIDB ^ spm_flr_gr.png %FLOWER% ^ %FLR_MASK% %FLR_IN_MASK% ^ spm_flr_bwc5.png ^ . spm_flr_div_u.miff This has reduced the bleed of yellow into the monochrome,

We can restore the sharpness with a small "unsharp" applied just in the transition area, between the two masks:
%IMG7%magick ^ spm_flr_bwc4.png ^ ( +clone unsharp 0x0.5+1+0 ) ^ ( %FLR_IN_MASK% negate ^ %FLR_MASK% ^ compose Darken composite ^ ) ^ compose Over composite ^ spm_flr_bwc4_u.png 
I regard this as the best result for the yellow flower on grayscale background.
Above, we have seen that colourchange at the boundary is propagated inwards, creating a colour cast. We have seen this can be prevented by an inner Dirichlet boundary. On the other hand, we can use this to our advantage.
Poisson pasting can be used to correct colours when photographs are to be joined with overlaps ("stitched"), such as panoramas or aerial photography. Suppose we have these three input images:
#1: %IMG7%magick ^ size 150x100 xc: ^ sparsecolor bilinear ^ 0,0,#67f,0,%%[fx:h1],#6f7,%%[fx:w1],0,#45d ^ gravity Center pointsize 30 ^ fill Black annotate +0+0 "#1" ^ fill White annotate 22 "#1" ^ spm_pan_in1.png 

#2: %IMG7%magick ^ size 150x100 xc: ^ sparsecolor bilinear ^ 0,0,#f88,0,%%[fx:h1],#7f6,%%[fx:w1],0,#faa ^ fill White gravity Center pointsize 30 ^ fill Black annotate +0+0 "#2" ^ fill White annotate 22 "#2" ^ spm_pan_in2.png 

#3: %IMG7%magick ^ size 150x100 xc: ^ sparsecolor bilinear ^ 0,0,#a6f,0,%%[fx:h1],#8d7,%%[fx:w1],0,#c7f ^ fill White gravity Center pointsize 30 ^ fill Black annotate +0+0 "#3" ^ fill White annotate 22 "#3" ^ spm_pan_in3.png 
Suppose they should overlap like this:
:skip set OFFS_2=+130+10 set OFFS_3=+25010 %IMG7%magick ^ spm_pan_in1.png ^ ( spm_pan_in2.png repage %OFFS_2% ) ^ ( spm_pan_in3.png repage %OFFS_3% ) ^ background None ^ layers merge +repage ^ spm_pan_out1.png set CROP_2=150x100%OFFS_2% 
There are many possible ways to blend these images, including:
We will use the third way.
The boundary condition, where we require the images to match, are at most of the left edge of #2 and part of the top (an inverted "L"), and an "L" shape near the right edge of #2.
We make a mask of the "L" shapes. The following is messy but depends only on the image sizes, their order, and the "repage" arguments. "fuzz" is used in case twice gray(50%) isn't exactly white. "layers merge" puts offsets in the output, in this case +010, so we can later use #2 offsets directly for cropping.
"set colorspace sRGB" is needed for IM v7, to get three channels. In IM v7, "border" uses the current "compose" method, so we need to explicitly use "plus".
%IMG7%magick ^ size 150x100 ^ xc:gray(50%%) set colorspace sRGB ^ fuzz 0.1%% ^ ( xc:Red ^ shave 1x1 bordercolor gray(50%%) border 1 ^ repage %OFFS_2% ^ ) ^ background Black ^ compose Plus layers merge +repage ^ fill gray(50%%) opaque Red ^ ( xc:Red ^ shave 1x1 ^ compose Over ^ bordercolor gray(50%%) border 1 ^ repage %OFFS_3% ^ ) ^ background Black ^ compose Plus layers merge ^ fill Black +opaque White ^ spm_Lboth.png Note: the output has offsets. 

Crop to #2: %IMG7%magick ^ spm_Lboth.png ^ crop %CROP_2% +repage ^ spm_L_cr.png 
We use this to blend image #2 with the other two images.
Subtract #2 from the others: %IMG7%magick ^ spm_pan_in1.png ^ ( spm_pan_in3.png repage %OFFS_3% ) ^ background None ^ layers merge ^ ( spm_pan_in2.png repage %OFFS_2% ) ^ compose Mathematics ^ define compose:args=0,0.5,0.5,0.5 ^ layers merge ^ spm_pan_sub.png Note: output has offsets. 

Crop to #2: %IMG7%magick ^ spm_pan_sub.png ^ crop %CROP_2% +repage ^ spm_pan_sub_cr.png 

Mask by the Lshapes: %IMG7%magick ^ spm_pan_sub_cr.png ^ spm_L_cr.png ^ compose CopyOpacity composite ^ spm_pan_sub_lcr.png 

Relaxfill to get an adjustment image: call %PICTBAT%relaxFillMS ^ spm_pan_sub_lcr.png . spm_pan_rf.png ^ 1e6 1000 

Add the adjustment image to #2: %IMG7%magick ^ spm_pan_in2.png ^ spm_pan_rf.png ^ compose Mathematics ^ define compose:args=0,2,1,1 composite ^ spm_pan_2adj.png 

Paste: %IMG7%magick ^ spm_pan_in1.png ^ ( spm_pan_2adj.png repage %OFFS_2% ) ^ ( spm_pan_in3.png repage %OFFS_3% ) ^ background None compose Over ^ layers merge +repage ^ spm_pan_123.png 
As required, the images blend seamlessly. Image #2 has been adjusted to exactly match two adjacent images. If image #2 had been entirely surrounded by other images, similar processing could be done.
The script colCorr2.bat composites one image over another, at a given offset, with colourcorrection to the top image. We can use the script to composite successive images over previous results. For example:
Composite #2 over #1: call %PICTBAT%colCorr2 ^ spm_pan_in1.png spm_pan_in2.png %OFFS_2% ^ spm_cc_12.png 

Composite #3 over previous result: call %PICTBAT%colCorr2 ^ spm_cc_12.png spm_pan_in3.png %OFFS_3% ^ spm_cc_123.png 
This result is different to the previous one. Instead of adjusting #2 to match both #1 and #3, we adjust #2 to match just #1, then adjust #3 to match the adjusted #2.
We can readily add further conditions. For example, we might want the central column of image #2 to be unchanged. This needs gray(50%) in the adjustment image.
Add a gray line: %IMG7%magick ^ spm_pan_sub_lcr.png ^ stroke gray(50%%) ^ draw "line 75,0,75,99" ^ spm_pan_sub_lcr2.png 

Relaxfill to get an adjustment image: call %PICTBAT%relaxFillMS ^ spm_pan_sub_lcr2.png . spm_pan_rf2.png ^ 1e6 1000 

Add the adjustment image to #2: %IMG7%magick ^ spm_pan_in2.png ^ spm_pan_rf2.png ^ compose Mathematics ^ define compose:args=0,2,1,1 composite ^ spm_pan_2adj2.png 

Paste: %IMG7%magick ^ spm_pan_in1.png ^ ( spm_pan_2adj2.png repage %OFFS_2% ) ^ ( spm_pan_in3.png repage %OFFS_3% ) ^ background None compose Over ^ layers merge +repage ^ spm_pan_1232.png 
We can do a similar trick in the script, by adding a small gray circle in the centre of the adjustment image:
Composite #2 over #1: set cc2DRAW=circle 75,50 80,50 call %PICTBAT%colCorr2 ^ spm_pan_in1.png spm_pan_in2.png %OFFS_2% ^ spm_cc_12b.png 

Composite #3 over previous result: call %PICTBAT%colCorr2 ^ spm_cc_12b.png spm_pan_in3.png %OFFS_3% ^ spm_cc_123b.png set cc2DRAW= 
The technical papers in the references above (Pérez etc) describe the mathematics of Poisson image editing. Here is a lessmathematical explanation. The goal is to create an image where some pixels have desired values (the boundary constraint) while others are such that the slope either changes as slowly as possible, or the rate of change follows a guidance.
The script relaxFill.bat aims at a zero rate of change of gradient.
At each iteration, for every pixel (and every channel), we set the new pixel value to the average value of the four neighbours to the north, south, east and west. This is a blurring operation. If we repeat this often enough, every pixel will be almost equal to the average of the four neighbours. This means the slope (gradient) of the image will be almost the same everywhere. It doesn't change quickly. Put it another way, the slope of the slope (the "second derivative") will be almost zero.
Generally, pixels won't be exactly equal to the average of the neighbours. That happens only when the result is a bilinear gradient, which only happens when the boundary values when considered as heights are coplanar. But the repeated iteration gets the overall image as close to this result as it can get.
A bit of maths (sorry): if we say that each pixel value is a function of x and y, the pixel is f(x,y), then the following is approximately true for every pixel in the relaxed result:
f(x1,y) + f(x+1,y) + f(x,y1) + f(x,y+1)  4*f(x,y) = 0
The left hand side of that equation approximates the slope of the slope of the image.
To get to that state, each iteration sets a new value for each pixel f'(x,y):
f'(x,y) = f(x1,y) + f(x+1,y) + f(x,y1) + f(x,y+1) 4
In ImageMagick code, this is:
morphology convolve:n 3x3:0,0.25,0,0.25,0,0.25,0,0.25,0
... where n is however many iterations we want to make between each test for completion.
The iterative process has to start somewhere. We need an initial guess at the result, and each iteration takes us closer to the actual result. With luck there is no local minimum so the initial guess doesn't matter; any initial guess will eventually give us the same result. But the closer the initial guess is to the actual result, the quicker we will get there.
The relaxFillScr.bat is a little more complex. Instead of aiming to set the slope of the slope to zero for every pixel, it aims at some other particular slope of the slope for each pixel. The values to be aimed at are given in the guidance image. So in the relaxed result, the following is approximately true:
f(x1,y) + f(x+1,y) + f(x,y1) + f(x,y+1)  4*f(x,y) = guide(x,y)
So each iteration is:
f'(x,y) = f(x1,y) + f(x+1,y) + f(x,y1) + f(x,y+1)  guide(x,y) 4 4
IM can't do this in a single operation. (For example, morphology can't use more than one image as input.) So each iteration has two phases: first we convolve as above, then we subtract another image. These phases fight each other: we smooth then roughen the image, and repeat this smoothing and roughening. This makes the process slow to converge.
In preparation for the script, we set the guidance image to the divergence of the required gradient, so it is the sum of the slope of the slope in the x and y directions.
[[Divergence can be negative, so we offset this for IM by 50% in the usual way so gray(50%) represents zero divergence.]]
This calls either relaxFill.bat or relaxFillScr.bat at multiple scales. For relaxFill.bat, this always improves performance. However, for relaxFillScr.bat the improvement may be small or even negative.
A recursive mechanism is used, dividing the dimensions by two while the minimum dimension is greater than 50. At each level in the recursive stack, the script resizes images and calls either relaxFill.bat or relaxFillScr.bat, using the previous result as the initial guess. In the initial levels, striving for high accuracy is a waste of effort so the target score is relaxed by a factor that is the ratio of the areas (which is roughly four) at each level.
Here is a simple example using relaxFillMS.bat to relaxfill with no explicit guidance:
Example image, with transparent centre: %IMG7%magick ^ size 400x300 xc:None ^ size 20x300 ^ xc:#f80 gravity West composite ^ xc:#ff0 gravity East composite ^ size 400x20 ^ xc:#08f gravity North composite ^ xc:#080 gravity South composite ^ spm_bnd.png 

Relaxfilled: call %PICTBAT%relaxFillMS ^ spm_bnd.png . spm_bnd_rf.png 
This is a higherlevel script. For inputs it uses a background, foreground and mask. The mask should be white where we want the adjusted foreground, and hence defines the area that will be relaxed.
The script optionally takes a guidance image.
This is like poissonPaste.bat, but takes an extra mask, that is white where we want the foreground to be unchanged. Hence the area to be relaxed is between the two masks.
If it is called with this inner mask set to dot, ".", it instead calls poissonPaste.bat.
To further illustrate the relaxation process, we make a simple example, and show the intensity graphs at a crosssection through the images, at the middle row.
Create a background. %IMG7%magick ^ size 200x200 gradient: ^ rotate 90 ^ spm_xb.png call %PICTBAT%xSection spm_xb.png 

Create a foreground. The square is 40% lighter than its surround. %IMG7%magick ^ size 200x200 xc:gray(30%%) ^ fill gray(70%%) draw "rectangle 80,80 120,120" ^ spm_xf.png call %PICTBAT%xSection spm_xf.png 

Create a mask. %IMG7%magick ^ size 200x200 xc:Black ^ fill White draw "circle 100,100 150,100" ^ spm_xm.png 
The goal is to blend the background and foreground, where the mask is white.
We show the elapsed time for the process, and the number of iterations.
Poisson paste with minimal iterations. set rfCOMP_METRIC=PAE set ppTGT_SCORE=999 set ppNUM_STEPS=1 call %PICTBAT%poissonPaste ^ spm_xb.png spm_xf.png spm_xm.png ^ spm_xout1.png call %PICTBAT%xSection spm_xout1.png 0 00:00:04 rfmsITER=2 

Poisson paste with a few iterations. set rfCOMP_METRIC=PAE set ppTGT_SCORE=0.1 set ppNUM_STEPS=10 call %PICTBAT%poissonPaste ^ spm_xb.png spm_xf.png spm_xm.png ^ spm_xout2.png call %PICTBAT%xSection spm_xout2.png 0 00:00:04 rfmsITER=20 

Poisson paste to approximate completion. set rfCOMP_METRIC=PAE set ppTGT_SCORE=0.001 set ppNUM_STEPS=1000 call %PICTBAT%poissonPaste ^ spm_xb.png spm_xf.png spm_xm.png ^ spm_xout3.png call %PICTBAT%xSection spm_xout3.png 0 00:00:53 rfmsITER=6000 In the finished result, the square is 40% lighter than its surround. 
We will repeat the above, but with an explicit guidance field. First, we make a guidance field: the divergence of the slope of the foreground.
call %PICTBAT%slopeXY spm_xf.png spm_xf_sxy.miff call %PICTBAT%slopeXYdiv spm_xf_sxy.miff spm_xf_div.miff %IMG7%magick ^ spm_xf_div.miff ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_xf_div.png call %PICTBAT%xSection ^ spm_xf_div.miff ^ spm_xf_div_xs.png 
Now we use this divergence as the guidance:
Poisson paste with minimal iterations. set rfCOMP_METRIC=PAE set ppTGT_SCORE=999 set ppNUM_STEPS=1 call %PICTBAT%poissonPaste ^ spm_xb.png spm_xf.png spm_xm.png ^ spm_xout1g.png . spm_xf_div.miff call %PICTBAT%xSection spm_xout1g.png 0 00:00:04 rfmsITER=2 

Poisson paste. set rfCOMP_METRIC=PAE set ppTGT_SCORE=0.1 set ppNUM_STEPS=10 call %PICTBAT%poissonPaste ^ spm_xb.png spm_xf.png spm_xm.png ^ spm_xout2g.png . spm_xf_div.miff call %PICTBAT%xSection spm_xout2g.png 0 00:00:05 rfmsITER=30 

Poisson paste to approximate completion. set rfCOMP_METRIC=PAE set ppTGT_SCORE=0.001 set ppNUM_STEPS=1000 call %PICTBAT%poissonPaste ^ spm_xb.png spm_xf.png spm_xm.png ^ spm_xout3g.png . spm_xf_div.miff call %PICTBAT%xSection spm_xout3g.png 0 00:03:31 rfmsITER=12000 
Using an explicit guidance field of the divergence of the slope of the foreground eventually gives almost the same result as using no explicit guidance field. But it takes eight times as long.
We can use a sharper divergence:
call %PICTBAT%shpDiv spm_xf.png spm_xf_div2.miff %IMG7%magick ^ spm_xf_div2.miff ^ evaluate Divide 2 ^ evaluate Add 50%% ^ spm_xf_div2.png call %PICTBAT%xSection spm_xf_div2.png 

Poisson paste. set rfCOMP_METRIC=PAE set ppTGT_SCORE=0.001 set ppNUM_STEPS=1000 call %PICTBAT%poissonPaste ^ spm_xb.png spm_xf.png spm_xm.png ^ spm_xout3g2.png . spm_xf_div2.miff call %PICTBAT%xSection spm_xout3g2.png 0 00:03:01 rfmsITER=9000 
There might be a large mosaic of photographs in an irregular grid, where a small number are known to be "correct", with a requirement to adjust all the others to create a smooth final result. There could be millions of images, each with millions of pixels.
A possible approach is:
After I wrote this page in 2017, IM has implemented operations "compose seamlessblend composite" and "compose saliencyblend composite". These are documented in Forum: Seamless Blending and Saliency Blending.
For both methods, three numbers can be given in "define compose:arg=". These are:
The last is relevant only when "verbose" is in effect.
%IMG7%magick ^ verbose ^ spm_back_crp.jpg ^ %BEAR% ^ %MASK% ^ alpha off ^ define compose:args=5000x0.00001+500 ^ compose seamlessblend composite ^ depth 16 ^ spm_biw.png This is very close to the result spm_bear_adj.png above. 
We do a Poissonpaste with an Inner Dirichlet Boundary, using spm_bear_sm.png. We need a black shape on a white background, so we negate the image.
set IDB=spm_bear_sm.png %IMG7%magick ^ verbose ^ %IDB% negate +write mpr:IDB +delete ^ spm_back_crp.jpg ^ ( %BEAR% readmask mpr:IDB ) ^ %MASK% ^ alpha off ^ define compose:args=5000x0.00001+500 ^ compose seamlessblend composite ^ depth 16 ^ spm_biw2.png 
The second example, with the Inner Dirichlet Boundary, has slightly darkened the bear (towards its original colour), but not as much as in the method shown above in the Inner Dirichlet boundary section.
These builtin methods are faster than the scriptbased methods shown above, partly because they don't need to read and write an image at each iteration. However, they can't be supplied with a "first guess", so we can't make a first guess from downsized inputs. They also cannot use a guidance image.
For convenience, .bat scripts are also available in a single zip file. See Zipped BAT files.
rem Seamless boundary by Poisson pasting. rem %1 is input background (without holes) rem %2 is input foreground, same size (relevant only when no guidance) rem %3 is input mask, same size rem mask is white where we want adjusted foreground, or black for background rem %4 is output. rem %5 0 or 1: whether to negate the mask. rem %6 optional guidance image rem %7 radius of final unsharp. 0=no unsharp. Default 0. @rem @rem Also uses: @rem ppTGT_SCORE target RMSE or PAE score. [default 0.001] @rem ppNUM_STEPS number of iteration steps between tests. [default 100] @rem ppPOST processing to apply to transition area at end, eg "unsharp 0x0.5+1+0". @rem @rem rfRLX_FUNC is a relaxation function. [default: average of 4 neighbours] @rem rfCOMP_METRIC default RMSE. PAE is more sensitive. @rem @rem Updated: @rem 24July2022 for IM v7. @rem @if "%3"=="" findstr /B "rem @rem" %~f0 & exit /B 1 @setlocal enabledelayedexpansion @call echoOffSave call %PICTBAT%setInOut %1 pp set IN_A=%INFILE% set IN_B=%2 set MASK=%3 if not "%4"=="" if not "%4"=="." set OUTFILE=%4 set NEG_MASK=%5 if "%NEG_MASK%"=="." set NEG_MASK= if "%NEG_MASK%"=="" set NEG_MASK=0 if not %NEG_MASK%==0 if not %NEG_MASK%==1 exit /B 1 if %NEG_MASK%==0 ( set NEG1=negate set NEG2= ) else ( set NEG1= set NEG2=negate ) set GUID=%6 if "%GUID%"=="" set GUID=. set HAS_GUID=1 if "%GUID%"=="." set HAS_GUID=0 set UNSHP=%7 if "%UNSHP%"=="." set UNSHP= if "%UNSHP%"=="" set UNSHP=0 if "%ppPOST%"=="" ( set sUNSHP= ) else ( set sUNSHP=^^^( +clone %ppPOST% ^^^) %MASK% compose Over composite ) if "%ppTGT_SCORE%"=="" set ppTGT_SCORE=0.001 if "%ppNUM_STEPS%"=="" set ppNUM_STEPS=100 set TMPDIR=\temp\ set TMP_APPRX=%TMPDIR%pp_tmp_apprx.miff set TMP1=%TMPDIR%pp_tmp1.miff set TMP2=%TMPDIR%pp_tmp2.miff :: Provide a first approximation: the background over the foreground. :: However, this seems to make performance worse. :: %IMG7%magick ^ :: %IN_B% ^ :: %IN_A% ^ :: compose Over composite ^ :: %TMP_APPRX% set TMP_APPRX=. if %HAS_GUID%==1 ( %IMG7%magick ^ %IN_A% colorspace sRGB ^ ^( %MASK% alpha off %NEG1% ^) ^ alpha off compose CopyOpacity composite ^ +depth ^ %TMP1% if ERRORLEVEL 1 exit /B 1 ) else ( %IMG7%magick ^ define compose:clamp=off ^ %IN_A% colorspace sRGB ^ %IN_B% ^ compose Mathematics ^ define "compose:args=0,0.5,0.5,0.5" composite ^ ^( %MASK% alpha off %NEG1% ^) ^ alpha off compose CopyOpacity composite ^ +depth ^ %TMP1% if ERRORLEVEL 1 exit /B 1 ) call %PICTBAT%relaxFillMS ^ %TMP1% %TMP_APPRX% %TMP2% ^ %ppTGT_SCORE% %ppNUM_STEPS% ^ . %GUID% if ERRORLEVEL 1 exit /B 1 if %HAS_GUID%==1 ( %IMG7%magick ^ %TMP2% ^ %sUNSHP% ^ +depth ^ %OUTFILE% if ERRORLEVEL 1 exit /B 1 ) else ( %IMG7%magick ^ define compose:clamp=off ^ %IN_B% ^ %TMP2% ^ compose Mathematics ^ define "compose:args=0,2,1,1" composite ^ +depth ^ ^( %MASK% alpha off %NEG2% ^) ^ alpha off ^ compose CopyOpacity composite ^ %IN_A% ^ +swap ^ compose Over composite ^ %sUNSHP% ^ +depth ^ %OUTFILE% if ERRORLEVEL 1 exit /B 1 ) call echoRestore @endlocal & set ppOUTFILE=%OUTFILE%& set rfmsITER=%rfmsITER%
rem Seamless boundary by Poisson pasting, with inner Dirichlet boundary. rem %1 is input background (without holes) rem %2 is input foreground, same size rem %3 is input mask, same size rem mask is white where we want adjusted foreground, or black for background rem %4 inner mask, white where we want exact foreground rem %5 is output. rem %6 0 or 1: whether to negate the mask. rem %7 optional guidance image @rem @rem Also uses: @rem ppTGT_SCORE target RMSE or PAE score. [default 0.001] @rem ppNUM_STEPS number of iteration steps between tests. [default 100] @rem @rem rfRLX_FUNC is a relaxation function. [default average of 4 neighbours] @rem rfCOMP_METRIC default RMSE. PAE is more sensitive. @rem @rem Updated: @rem 24July2022 for IM v7. @rem @if "%4"=="" findstr /B "rem @rem" %~f0 & exit /B 1 @setlocal enabledelayedexpansion @call echoOffSave call %PICTBAT%setInOut %1 ppi set IN_A=%INFILE% set IN_B=%2 set MASK=%3 set MASK_IDB=%4 if "%MASK_IDB%"=="." set MASK_IDB= if not "%5"=="" if not "%5"=="." set OUTFILE=%5 set NEG_MASK=%6 if "%NEG_MASK%"=="." set NEG_MASK= if "%NEG_MASK%"=="" set NEG_MASK=0 set GUID=%7 if "%GUID%"=="" set GUID=. set HAS_GUID=1 if "%GUID%"=="." set HAS_GUID=0 if %NEG_MASK%==0 ( set NEG1=negate set NEG2= ) else ( set NEG1= set NEG2=negate ) if "%MASK_IDB%"=="" ( call %PICTBAT%poissonPaste ^ %IN_A% %IN_B% %MASK% %OUTFILE% %NEG_MASK% %GUID% goto finished ) if "%ppTGT_SCORE%"=="" set ppTGT_SCORE=0.001 if "%ppNUM_STEPS%"=="" set ppNUM_STEPS=100 set TMPDIR=\temp\ set TMP1=%TMPDIR%ppi_tmp1.miff set TMP2=%TMPDIR%ppi_tmp2.miff set TMP_MASK=%TMPDIR%ppi_msk.png %IMG7%magick ^ %MASK% ^ ( %MASK_IDB% alpha off negate ) ^ alpha off ^ compose Darken composite ^ %TMP_MASK% if %HAS_GUID%==1 ( %IMG7%magick ^ define compose:clamp=off ^ compose CopyOpacity ^ ^( %IN_A% colorspace sRGB ^ ^( %MASK% alpha off %NEG1% ^) ^ alpha off composite ^ ^) ^ ^( %IN_B% ^ ^( %MASK_IDB% alpha off %NEG2% ^) ^ alpha off composite ^ ^) ^ compose Over composite ^ %TMP1% if ERRORLEVEL 1 exit /B 1 ) else ( %IMG7%magick ^ define compose:clamp=off ^ %IN_A% colorspace sRGB ^ %IN_B% ^ compose Mathematics ^ define "compose:args=0,0.5,0.5,0.5" composite ^ ^( +clone fill gray^(50%%^) colorize 100 ^) ^ ^( %MASK_IDB% ^) ^ compose Over composite ^ +depth ^ ^( %TMP_MASK% alpha off %NEG1% ^) ^ alpha off compose CopyOpacity composite ^ %TMP1% if ERRORLEVEL 1 exit /B 1 ) call %PICTBAT%relaxFillMS ^ %TMP1% . %TMP2% ^ %ppTGT_SCORE% %ppNUM_STEPS% ^ . %GUID% if ERRORLEVEL 1 exit /B 1 if %HAS_GUID%==1 ( %IMG7%magick ^ %TMP2% ^ %OUTFILE% if ERRORLEVEL 1 exit /B 1 ) else ( %IMG7%magick ^ %IN_B% ^ %TMP2% ^ compose Mathematics ^ define "compose:args=0,2,1,1" composite ^ +depth ^ ^( %MASK% alpha off %NEG2% ^) ^ alpha off ^ compose CopyOpacity composite ^ %IN_A% ^ colorspace sRGB ^ +swap ^ compose Over composite ^ +depth ^ %OUTFILE% if ERRORLEVEL 1 exit /B 1 ) :finished call echoRestore @endlocal & set ppiOUTFILE=%OUTFILE%
rem From image %1, rem make output %2 rem that is smoothed by Poissonpasting with guidance field rem that is the div of the slope rem set to zero where the slope is within %3 percent of zero. @rem @rem Updated: @rem 24July2022 for IM v7. @rem @if "%1"=="" findstr /B "rem @rem" %~f0 & exit /B 1 @setlocal @call echoOffSave call %PICTBAT%setInOut %1 psm if not "%2"=="." if not "%2"=="" set OUTFILE=%2 set SLP_LIM_PC=%3 if "%SLP_LIM_PC%"=="." set SLP_LIM_PC= if "%SLP_LIM_PC%"=="" set SLP_LIM_PC=10 set TMPDIR=\temp\ set SXY=%TMPDIR%psm_sxy.miff set SXY2=%TMPDIR%psm_sxy2.miff set DIV=%TMPDIR%psm_div.miff set MASK=%TMPDIR%psm_msk.miff call %PICTBAT%slopeXY %INFILE% %SXY% if ERRORLEVEL 1 exit /B 1 %IMG7%magick ^ %SXY% ^ fuzz %SLP_LIM_PC%%% fill Black opaque Black ^ define quantum:format=floatingpoint ^ depth 32 ^ %SXY2% call %PICTBAT%slopeXYdiv %SXY2% %DIV% if ERRORLEVEL 1 exit /B 1 %IMG7%magick ^ %INFILE% ^ fill White colorize 100 ^ gravity Center ^ shave 1x1 ^ bordercolor Black border 1 ^ %MASK% call %PICTBAT%poissonPaste ^ %INFILE% . %MASK% ^ %OUTFILE% ^ . %DIV% if ERRORLEVEL 1 exit /B 1 call echoRestore endlocal & set psmOUTFILE=%OUTFILE%
rem Given image %1, rem makes %2 rem from connected components that are at least %3 pixels. @rem %3 may have suffix 'c' or '%' for percentage or 'p' for proportion of image w*h. @rem Default 0. @rem @rem Each output connected component is the mean of its input pixels. @rem @rem Updated: @rem 24July2022 for IM v7. @rem @if "%1"=="" findstr /B "rem @rem" %~f0 & exit /B 1 @setlocal enabledelayedexpansion @call echoOffSave call %PICTBAT%setInOut %1 ccla if not "%2"=="" if not "%2"=="." set OUTFILE=%2 set THRESH=%3 if "%THRESH%"=="." set THRESH= if "%THRESH%"=="" set THRESH=0 set TH_LAST=%THRESH:~1% if "%TH_LAST%"=="^%" set TH_LAST=c if /I "%TH_LAST%"=="c" ( for /F "usebackq" %%L in (`%IMG7%magick identify ^ format "nTHRESH=%%[fx:w*h*%THRESH:~0,1%/100]" ^ %INFILE%`) do set %%L ) else if /I "%TH_LAST%"=="p" ( for /F "usebackq" %%L in (`%IMG7%magick identify ^ format "nTHRESH=%%[fx:w*h*%THRESH:~0,1%]" ^ %INFILE%`) do set %%L ) else ( set nTHRESH=%THRESH% ) if %nTHRESH%==0 ( set sTHRESH= ) else ( set "sTHRESH=define connectedcomponents:areathreshold^=%nTHRESH%" ) %IMG7%magick ^ %INFILE% ^ define connectedcomponents:verbose^=true ^ define connectedcomponents:meancolor=true ^ %sTHRESH% ^ connectedcomponents 4 ^ %OUTFILE% call echoRestore @endlocal & set cclaOUTFILE=%OUTFILE%
rem Given image %1, rem makes %2 rem from the largest %3 connected components. @rem @rem For performance, use the output from connCompLimitArea.bat @rem to reduce the number of components this script considers. @rem (or add area parameter to this script?) @rem @rem When an eliminated component is adjacent to more than one other component, @rem it may merge with the "wrong" one. @rem @rem Each output connected component is the mean of its input pixels. @rem @rem Updated: @rem 24July2022 for IM v7. @rem 29July2022 Bug in "connectedcomponents" hinders merging. "virtualpixel None" may help. @rem Also repeating "connectedcomponents" with rotated image. @rem (See https://github.com/ImageMagick/ImageMagick/issues/5368 ) @rem @if "%1"=="" findstr /B "rem @rem" %~f0 & exit /B 1 @setlocal enabledelayedexpansion rem @call echoOffSave call %PICTBAT%setInOut %1 ccln if not "%2"=="" if not "%2"=="." set OUTFILE=%2 set NUM=%3 if "%NUM%"=="." set NUM= if "%NUM%"=="" set NUM=1 set N_CONNECT=8 set N_CONNECT=4 goto skip1 %IMG7%magick ^ %INFILE% ^ define connectedcomponents:verbose^=true ^ connectedcomponents %N_CONNECT% ^ NULL: :skip1 set sKEEP= set CNT=1 set LIM_AREA=0 set nTHRESH= for /F "usebackq skip=1 tokens=4" %%A in (`%IMG7%magick ^ %INFILE% ^ define connectedcomponents:verbose^=true ^ connectedcomponents %N_CONNECT% ^ NULL:`) do ( rem echo %%A if !CNT!==%NUM% set nTHRESH=%%A set /A CNT+=1 ) echo %0: nTHRESH=%nTHRESH% if "%nTHRESH%"=="" ( echo %0: Found CNT=%CNT%, not NUM=%NUM% exit /B 1 ) %IMG7%magick ^ %INFILE% ^ virtualpixel None ^ define connectedcomponents:verbose=true ^ define connectedcomponents:meancolor=true ^ define connectedcomponents:areathreshold=%nTHRESH% ^ connectedcomponents %N_CONNECT% ^ rotate 180 ^ connectedcomponents %N_CONNECT% ^ rotate 180 ^ %OUTFILE% goto end set sKEEP= set CNT=1 set LIM_AREA=0 for /F "usebackq skip=1 tokens=1 delims=: " %%A in (`%IMG7%magick ^ %INFILE% ^ define connectedcomponents:verbose^=true ^ connectedcomponents %N_CONNECT% ^ NULL:`) do ( rem echo %%A if !CNT! LEQ %NUM% set sKEEP=!sKEEP!,%%A set /A CNT+=1 ) echo sKEEP=%sKEEP% rem Remove initial comma set sKEEP=%sKEEP:~1% echo sKEEP=%sKEEP% :: FIXME: No, this makes the others transparent!! :: We need to find the limiting area. set UNIQ=\temp\ccln_uniq.png %IMG7%magick ^ %INFILE% ^ define connectedcomponents:verbose^=true ^ define connectedcomponents:keep=%sKEEP% ^ define connectedcomponents:meancolor=true ^ connectedcomponents %N_CONNECT% ^ uniquecolors ^ %UNIQ% %IMG7%magick identify %UNIQ% :end call echoRestore @endlocal & set cclnOUTFILE=%OUTFILE%
rem Given images %1 and %2 rem %3 is an offset for #2 with respect to #1 rem makes output %4 rem that is #2 composited over #1, and colourcorrected. @rem @rem Also uses: @rem cc2DRAW if provided, draws this in gray before relaxfill. @rem @rem @rem Updated: @rem 24July2022 for IM v7. @rem @if "%3"=="" findstr /B "rem @rem" %~f0 & exit /B 1 @setlocal enabledelayedexpansion @call echoOffSave set INFILE1=%1 set INFILE2=%2 set OFFS_2=%3 set OUTFILE=%4 set TMP_FILE=\temp\cc2.miff if "%cc2DRAW%"=="" ( set sDRAW= ) else ( set sDRAW=fill gray^(50%%^) stroke gray^(50%%^) draw "%cc2DRAW%" ) %IMG7%magick ^ ( %INFILE1% +write mpr:IN1 ) ^ fill gray(50%%) colorize 100 ^ fuzz 0.1%% ^ ( %INFILE2% +write mpr:IN2 ^ fill Red colorize 100 ^ shave 1x1 bordercolor gray(50%%) border 1 ^ repage %OFFS_2% ^ ) ^ background Black ^ compose Plus layers merge +repage ^ fill Black +opaque White ^ crop %OFFS_2% +repage ^ +write mpr:LMASK ^ +delete ^ mpr:IN1 ^ ( mpr:IN2 repage %OFFS_2% ) ^ compose Mathematics ^ define compose:args=0,0.5,0.5,0.5 ^ background None ^ layers merge ^ crop %OFFS_2% +repage ^ mpr:LMASK ^ alpha off ^ compose CopyOpacity composite ^ %sDRAW% ^ %TMP_FILE% if ERRORLEVEL 1 exit /B 1 rem Relaxfill to get an adjustment image: call %PICTBAT%relaxFillMS ^ %TMP_FILE% . %TMP_FILE% ^ 1e6 1000 if ERRORLEVEL 1 exit /B 1 :skip rem Add the adjustment image to #2: %IMG7%magick ^ %INFILE2% ^ %TMP_FILE% ^ compose Mathematics ^ define compose:args=0,2,1,1 composite ^ repage %OFFS_2% ^ %INFILE1% ^ +swap ^ background None compose Over ^ layers merge +repage ^ %OUTFILE% if ERRORLEVEL 1 exit /B 1 endlocal& set cc2OUTFILE=%OUTFILE%
rem From image %1, rem make horizontal crosssection. @rem @rem Updated: @rem 24July2022 for IM v7. @rem @if "%1"=="" findstr /B "rem @rem" %~f0 & exit /B 1 @setlocal @call echoOffSave call %PICTBAT%setInOut %1 xs if not "%2"=="" if not "%2"=="." set OUTFILE=%2 set WW= for /F "usebackq" %%L in (`%IMG7%magick identify ^ format "WW=%%w\nHH=%%h\nW_2=%%[fx:int(w/2)]\nH_2=%%[fx:int(h/2)]" ^ %INFILE%`) do set %%L if "%WW%"=="" exit /B 1 set TMP=%TEMP%\xs.miff %IMG7%magick ^ %INFILE% ^ crop %WW%x1+0+%H_2% +repage ^ %TMP% call %PICTBAT%graphLineCol %TMP% . . . %OUTFILE% call echoRestore endlocal & set xsOUTFILE=%OUTFILE%
All images on this page were created by the commands shown, using:
%IMG7%magick identify version
Version: ImageMagick 7.1.049 Q16HDRI x64 7a3f3f1:20220924 https://imagemagick.org Copyright: (C) 1999 ImageMagick Studio LLC License: https://imagemagick.org/script/license.php Features: Cipher DPC HDRI OpenCL Delegates (builtin): bzlib cairo freetype gslib heic jng jp2 jpeg jxl lcms lqr lzma openexr pangocairo png ps raqm raw rsvg tiff webp xml zip zlib Compiler: Visual Studio 2022 (193331630)
To improve internet download speeds, some images may have been automatically converted (by ImageMagick, of course) from PNG or TIFF or MIFF to JPG.
Source file for this web page is seamlpm.h1. To recreate this web page, run "procH1 seamlpm".
This page, including the images except where shown otherwise, is my copyright. Anyone is permitted to use or adapt any of the code, scripts or images for any purpose, including commercial use.
Anyone is permitted to republish this page, but only for noncommercial use.
Anyone is permitted to link to this page, including for commercial use.
Page version v2.0 25June2017.
Page created 29Sep2022 04:24:11.
Copyright © 2022 Alan Gibson.