snibgo's ImageMagick pages

Inverse composites

If a known image was composited with an unknown image making a known result, how can we find the unknown?

• The problem
• Terminology
• Opaque images
• Non-opaque images

The problem

Suppose we have images A.png and R.png, and we know that...

magick A.png X.png -compose CCCC -composite R.png 

... where CCCC is a known compose option, and we want to find X.png. How do we so this?

In ordinary arithmetic, we might know that...

3 + x = 8

... and we want to find the value of x that makes this true. We solve the problem by rearranging...

x = 8 - 3

... and subtracting 3 from 8 to give 5. So:

x = 5

The operators "plus" and "minus" are inverses of each other.

When images are opaque, we can easily do the same trick with ImageMagick. When one or more of the images has transparency, many solutions may be possible.

Terminology

The general command for compositing two input images to make a result is:

magick Dest.png Src.png -compose CCCC -composite Result.png

We will consider compose operators Plus and Over.

Formulae will show how pixel values are calculated. This will be for colour components (ie colour channels) and the alpha component. We will roughly follow the SVG Compositing Specification and use these variables:

Images may have three colour components, but only one alpha component. Values will generally be 0.0 to 1.0. An alpha of 0.0 means transparent, and 1.0 means opaque. We will limit discussion to opaque Destination and Result images, hence Ra=Da=0.0.

We will also limit discussion to cases where the inputs are the same size.

Caution: "-composite" will clamp unless we use "-set option:compose:clamp false" or "-define compose:clamp false"

Opaque images

Opaque images have alpha = 1.0, which makes calculations simple.

We will use these as opaque inputs:

%IMG7%magick ^ -size 300x200 ^ gradient:#05f-#2f6 ^ ic_op_dst.png

%IMG7%magick ^ ( -size 200x300 ^ gradient:#400-#f88 ^ -rotate -90 ^ ) ^ ( -size 300x200 ^ xc:Black ^ -fill White ^ -draw "circle 150,100 150,150" ^ -blur 0x5 ^ ) ^ -alpha off ^ -compose CopyOpacity -composite ^ -background Black ^ -compose Over -layers Flatten ^ ic_op_src.png

Here are the results from plus and over:

%IMG7%magick ^ ic_op_dst.png ^ ic_op_src.png ^ -compose Plus -composite ^ ic_op_plus.png
%IMG7%magick ^ ic_op_dst.png ^ ic_op_src.png ^ -compose Over -composite ^ ic_op_over.png

Composite Plus

This is the formula for the Plus operator:

R = D + S

If the inputs range from 0.0 to 1.0, the output is from 0.0 to 2.0.

If we know the Result and one of the inputs, we can easily calculate the other from the inverse formulae:

D = R - S
S = R - D

If the inputs range from 0.0 to 1.0, the output is from -1.0 to +1.0.

These formulae easily translate to ImageMagick processes, so we can derive either the Source or Dest by subtracting the other from the result. We can also compare the derived Sourse and Dest with the original Source and Dest.

%IMG7%magick ^ ic_op_plus.png ^ ic_op_src.png ^ -compose MinusSrc -composite ^ ic_op_plus_d.png %IMG7%magick compare ^ -metric RMSE ^ ic_op_dst.png ic_op_plus_d.png ^ NULL: 509.854 (0.00777987)
%IMG7%magick ^ ic_op_plus.png ^ ic_op_dst.png ^ -compose MinusSrc -composite ^ ic_op_plus_s.png %IMG7%magick compare ^ -metric RMSE ^ ic_op_src.png ic_op_plus_s.png ^ NULL: 441.547 (0.00673757)

Composite Over

The formula for the Over operator (SrcOver, putting the Source over the Destination) is simply:

R = S

The inverse formula is:

S = R

There is no inverse formula to derive Destination.

Non-opaque images

When images have transparency, the formulae used for compose operators are more complex. Hence, the inverse formulae are also more complex, and the ImageMagick processes we need to implement the inverses are also complex.

We will use these as non-opaque inputs:

%IMG7%magick ^ -size 300x200 ^ gradient:#05f-#2f6 ^ ic_tr_dst.png

%IMG7%magick ^ ( -size 200x300 ^ gradient:#400-#f88 ^ -rotate -90 ^ ) ^ ( -size 300x200 ^ xc:Black ^ -fill White ^ -draw "circle 150,100 150,150" ^ -blur 0x5 ^ ) ^ -alpha off ^ -compose CopyOpacity -composite ^ ic_tr_src.png

Here are the results from plus and over:

%IMG7%magick ^ ic_tr_dst.png ^ ic_tr_src.png ^ -compose Plus -composite ^ ic_tr_plus.png
%IMG7%magick ^ ic_tr_dst.png ^ ic_tr_src.png ^ -compose Over -composite ^ ic_tr_over.png

ASIDE: I like the dark border around the circle. If I didn't like it, I would do the composition in linear colorspace, or CIELab, or whatever:

%IMG7%magick ^ ic_tr_dst.png ^ ic_tr_src.png ^ -colorspace RGB ^ -compose Over -composite ^ -colorspace sRGB ^ ic_tr_over_lin.png

Composite Plus

For composite Plus, the formula for the new colour component is:

R = (S+D)*Sa*Da + S*Sa * (1-Da) + D*Da * (1-Sa)

When Destination is fully opaque, then Da = 1 and:

R = (S+D)*Sa + D * (1-Sa)
  = S.Sa + D.Sa + D - D.Sa
  = S.Sa +D

If we know R, Sa and D then:

S = (R-D) / Sa

If we know R, S and D then:

Sa = (R-D) / S

In both inverse formulae, if the inputs range from 0.0 to 1.0, the output is from -infinity to +infinity.

We can also express the inverse formulae like this:

S * Sa = R - D

Known opaque images Result and Destination do not uniquely determine a Source image with transparency, but merely the product of the colour and alpha components. If we know the alpha channel values, we can calculate the colours. If, instead, we know the colours then we can calculate the alpha.

In this example, we can take a reasonable guess at either the Source colour or alpha.

Guess the Source alpha: this could be the red channel, auto-levelled. By inspection, we can see that a "-level" is needed to make the background entirely 0% and the object entirely 100%.

%IMG7%magick ^ ic_tr_over.png ^ -channel R -separate +channel ^ -auto-level ^ +write ic_tr_al0.png ^ -level 18%%,70%% ^ ic_tr_al.png

Photographs need more sophsticated methods to estimate alpha, such as remapping Result to the edge colours, differencing with Result, and auto-levelling. Or using various edge techniques.

Now we have values for alpha, we can calculate colours from the formula:

S = (R-D) / Sa

We calculate colours, and copy the alpha channel into it.

%IMG7%magick ^ ic_tr_plus.png ic_tr_dst.png ^ -compose MinusSrc -composite ^ ( ic_tr_al.png +write mpr:ALPH ) ^ -compose DivideSrc -composite ^ +write ic_tr_plus_col.png ^ mpr:ALPH ^ -compose CopyOpacity -composite ^ ic_tr_plus_colal.png

The colours are a little strange, which suggests the alpha is not quite correct, but the reconstructed Source with alpha, ic_tr_plus_colal.png, looks like the original Source. We can use the reconstructed Source composited over the background, and compare with the original result.

%IMG7%magick ^ ic_tr_dst.png ^ ic_tr_plus_colal.png ^ -compose Plus -composite ^ +write ic_tr_plus_recon.png ^ ic_tr_plus.png ^ -metric RMSE -compare ^ -format "%%[distortion]" ^ info: 0.00528014

The reconstruction is 0.5% from the original, which isn't bad.

There is a glitch. Images have three colour channels but only one alpha. Which colour channel should we use to calculate alpha? What if the three colour channels result in different alphas?

Composite Over

For "over" (Src over Dst):

Ra = Sa + Da - Sa * Da

R.Ra = S*Sa*Da + S*Sa * (1-Da) + D*Da * (1-Sa)

Opaque inputs

When the result R is opaque, Ra==1 so either Sa==1 or Da==1.

When dest is fully opaque, Da==1 and:

Ra = Sa + 1 - Sa
   = 1

The formula for "-compose Over -composite" simplifies to:

R = S*Sa + D * (1-Sa)
  = S.Sa + D - D.Sa
  = S.Sa + D.(1-Sa)

Of the four variable R, D, S and Sa, if we know three values we can calculate the fourth.

If we know R, S and D then we can find Sa:

Sa = (R-D) / (S-D)

If we know R, Sa and D then we can find S:

S = (R-D.(1-Sa)) / Sa
  = (R - D + D.Sa) / Sa
  = (R - D)/Sa + D

Even when we know the Result and Dest are opaque, if we don't know either the Source colours (S) or alpha (Sa), there is a virtually infinite number of possible solutions. This is because at each pixel that is different between Result and Dest, there is a virtually infinite number of colours between these two, so there is an infinite range of Source colours that, when composited with the appropriate alpha, makes the Result colour. But if we know (or can guess) the colour at each pixel then we can calculate the alpha values. Or we might know (or can guess) the alpha at each pixel then we can calculate the colours; see Deriving a semi-transparent watermark.

Finally, if we know S, Sa and R then we can find D:

R = S.Sa + D - D.Sa

D - D.Sa = R - S.Sa

D.(1-Sa) = R - S.Sa

D = (R - S.Sa) / (1-Sa)

Note that if Sa==1, we have a divide by zero, and D is indeterminate.

This also has an application in de-watermarking. If a watermark with no opaque pixels (S and Sa) is composited over an opaque input image (D) to make a watermarked image (R), we can reconstruct the original D from R, S and Sa. See the Watermarks page.

Where the watermark contains opaque pixels, Sa==1 so the original pixels cannot be derived.

Inputs with transparency

In the more general case, the D image may contain transparency, so we don't know that Da==1. We can still calculate Da and D, with a slight increase in complexity. First, find Da:

Ra = Sa + Da - Sa * Da

Hence:

Da = (Ra - Sa) / (1-Sa)

Special cases:

• When Ra==1, Da==1.
• When Sa==0, Da==Ra.
• When Sa==1, Da is indeterminate.

Note that Ra >= Sa and Ra >= Da at every pixel.

Now we can find D:

R*Ra = S*Sa*Da + S*Sa * (1-Da) + D*Da * (1-Sa)

Hence:

D = (R*Ra - S*Sa*Da - S*Sa * (1-Da)) / (Da * (1-Sa))
  = (R*Ra - S*Sa*Da - S*Sa * (1-Da)) / (Ra - Sa)
  = (R*Ra - S*Sa*(Da + (1-Da)) / (Ra - Sa)
  = (R*Ra - S*Sa) / (Ra - Sa)

Code to implement the inverse of "-compose Over -composite" is in the Watermarks page.

%IMG7%magick -version

Version: ImageMagick 7.1.0-42 Q16-HDRI x64 396d87c:20220709 https://imagemagick.org
Copyright: (C) 1999 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI OpenCL 
Delegates (built-in): bzlib cairo freetype gslib heic jng jp2 jpeg jxl lcms lqr lzma openexr pangocairo png ps raqm raw rsvg tiff webp xml zip zlib
Compiler: Visual Studio 2022 (193231332)

Page version v1.0 6-April-2016.

Page created 12-Aug-2022 02:47:57.

Copyright © 2022 Alan Gibson.