snibgo's ImageMagick pages

Inverse composites

If a known image was composited with an unknown image making a known result, how can we find the unknown?

The problem

Suppose we have images A.png and R.png, and we know that...

convert A.png X.png -compose CCCC -composite R.png 

... where CCCC is a known compose option, and we want to find X.png. How do we so this?

In ordinary arithmetic, we might know that...

3 + x = 8

... and we want to find the value of x that makes this true. We solve the problem by rearranging...

x = 8 - 3

... and subtracting 3 from 8 to give 5. So:

x = 5

The operators "plus" and "minus" are inverses of each other.

When images are opaque, we can easily do the same trick with ImageMagick. When one or more of the images has transparency, many solutions may be possible.

Terminology

The general command for compositing two input images to make a result is:

convert Dest.png Src.png -compose CCCC -composite Result.png

We will consider compose operators Plus and Over.

Formulae will show how pixel values are calculated. This will be for colour components (ie colour channels) and the alpha component. We will roughly follow the SVG Compositing Specification and use these variables:

R colour component of a Result pixel
D colour component of a Destination pixel
S colour component of a Source pixel
Ra alpha component of a Result pixel
Da alpha component of a Destination pixel
Sa alpha component of a Source pixel

Images may have three colour components, but only one alpha component. Values will generally be 0.0 to 1.0. An alpha of 0.0 means transparent, and 1.0 means opaque. We will limit discussion to opaque Destination and Result images, hence Ra=Da=0.0.

We will also limit discussion to cases where the inputs are the same size.

Caution: "-composite" will clamp unless we use "-set option:compose:clamp false" or "-define compose:clamp false"

Opaque images

Opaque images have alpha = 1.0, which makes calculations simple.

We will use these as opaque inputs:

%IM%convert ^
  -size 300x200 ^
  gradient:#05f-#2f6 ^
  ic_op_dst.png
ic_op_dst.png
%IM%convert ^
  ( -size 200x300 ^
    gradient:#400-#f88 ^
    -rotate -90 ^
  ) ^
  ( -size 300x200 ^
    xc:Black ^
    -fill White ^
    -draw "circle 150,100 150,150" ^
    -blur 0x5 ^
  ) ^
  -alpha off ^
  -compose CopyOpacity -composite ^
  -background Black ^
  -compose Over -layers Flatten ^
  ic_op_src.png
ic_op_src.png

Here are the results from plus and over:

%IM%convert ^
  ic_op_dst.png ^
  ic_op_src.png ^
  -compose Plus -composite ^
  ic_op_plus.png
ic_op_plus.png
%IM%convert ^
  ic_op_dst.png ^
  ic_op_src.png ^
  -compose Over -composite ^
  ic_op_over.png
ic_op_over.png

Composite Plus

This is the formula for the Plus operator:

R = D + S

If the inputs range from 0.0 to 1.0, the output is from 0.0 to 2.0.

If we know the Result and one of the inputs, we can easily calculate the other from the inverse formulae:

D = R - S
S = R - D

If the inputs range from 0.0 to 1.0, the output is from -1.0 to +1.0.

These formulae easily translate to ImageMagick processes, so we can derive either the Source or Dest by subtracting the other from the result. We can also compare the derived Sourse and Dest with the original Source and Dest.

%IM%convert ^
  ic_op_plus.png ^
  ic_op_src.png ^
  -compose MinusSrc -composite ^
  ic_op_plus_d.png

%IM%compare ^
  -metric RMSE ^
  ic_op_dst.png ic_op_plus_d.png ^
  NULL: 
509.86 (0.00777997)
ic_op_plus_d.png
%IM%convert ^
  ic_op_plus.png ^
  ic_op_dst.png ^
  -compose MinusSrc -composite ^
  ic_op_plus_s.png

%IM%compare ^
  -metric RMSE ^
  ic_op_src.png ic_op_plus_s.png ^
  NULL: 
1747.69 (0.026668)
ic_op_plus_s.png

Composite Over

The formula for the Over operator (SrcOver, putting the Source over the Destination) is simply:

R = S
The Destination can be anything at all; the Result will always be a copy of the Source.

The inverse formula is:

S = R

There is no inverse formula to derive Destination.

Non-opaque images

When images have transparency, the formulae used for compose operators are more complex. Hence, the inverse formulae are also more complex, and the ImageMagick processes we need to implement the inverses are also complex.

We will use these as non-opaque inputs:

%IM%convert ^
  -size 300x200 ^
  gradient:#05f-#2f6 ^
  ic_tr_dst.png
ic_tr_dst.png
%IM%convert ^
  ( -size 200x300 ^
    gradient:#400-#f88 ^
    -rotate -90 ^
  ) ^
  ( -size 300x200 ^
    xc:Black ^
    -fill White ^
    -draw "circle 150,100 150,150" ^
    -blur 0x5 ^
  ) ^
  -alpha off ^
  -compose CopyOpacity -composite ^
  ic_tr_src.png
ic_tr_src.png

Here are the results from plus and over:

%IM%convert ^
  ic_tr_dst.png ^
  ic_tr_src.png ^
  -compose Plus -composite ^
  ic_tr_plus.png
ic_tr_plus.png
%IM%convert ^
  ic_tr_dst.png ^
  ic_tr_src.png ^
  -compose Over -composite ^
  ic_tr_over.png
ic_tr_over.png

ASIDE: I like the dark border around the circle. If I didn't like it, I would do the composition in linear colorspace, or CIELab, or whatever:

%IM%convert ^
  ic_tr_dst.png ^
  ic_tr_src.png ^
  -colorspace RGB ^
  -compose Over -composite ^
  -colorspace sRGB ^
  ic_tr_over_lin.png
ic_tr_over_lin.png

Composite Plus

For composite Plus, the formula for the new colour component is:

R = (S+D)*Sa*Da + S*Sa * (1-Da) + D*Da * (1-Sa)

When Destination is fully opaque, then Da = 1 and:

R = (S+D)*Sa + D * (1-Sa)
  = S.Sa + D.Sa + D - D.Sa
  = S.Sa +D

If we know R, Sa and D then:

S = (R-D) / Sa

If we know R, S and D then:

Sa = (R-D) / S

In both inverse formulae, if the inputs range from 0.0 to 1.0, the output is from -infinity to +infinity.

We can also express the inverse formulae like this:

S * Sa = R - D

Known opaque images Result and Destination do not uniquely determine a Source image with transparency, but merely the product of the colour and alpha components. If we know the alpha channel values, we can calculate the colours. If, instead, we know the colours then we can calculate the alpha.

In this example, we can take a reasonable guess at either the Source colour or alpha.

Guess the Source alpha: this could be the red channel, auto-levelled. By inspection, we can see that a "-level" is needed to make the background entirely 0% and the object entirely 100%.

%IM%convert ^
  ic_tr_over.png ^
  -channel R -separate +channel ^
  -auto-level ^
  +write ic_tr_al0.png ^
  -level 18%%,70%% ^
  ic_tr_al.png
ic_tr_al0.png ic_tr_al.png

Photographs need more sophsticated methods to estimate alpha, such as remapping Result to the edge colours, differencing with Result, and auto-levelling. Or using various edge techniques.

Now we have values for alpha, we can calculate colours from the formula:

S = (R-D) / Sa

We calculate colours, and copy the alpha channel into it.

%IM%convert ^
  ic_tr_plus.png ic_tr_dst.png ^
  -compose MinusSrc -composite ^
  ( ic_tr_al.png +write mpr:ALPH ) ^
  -compose DivideSrc -composite ^
  +write ic_tr_plus_col.png ^
  mpr:ALPH ^
  -compose CopyOpacity -composite ^
  ic_tr_plus_colal.png
ic_tr_plus_col.png ic_tr_plus_colal.png

The colours are a little strange, which suggests the alpha is not quite correct, but the reconstructed Source with alpha, ic_tr_plus_colal.png, looks like the original Source. We can use the reconstructed Source composited over the background, and compare with the original result.

%IM%convert ^
  ic_tr_dst.png ^
  ic_tr_plus_colal.png ^
  -compose Plus -composite ^
  +write ic_tr_plus_recon.png ^
  ic_tr_plus.png ^
  -metric RMSE -compare ^
  -format "%%[distortion]" ^
  info: 
0.00528038
ic_tr_plus_recon.png

The reconstruction is 0.5% from the original, which isn't bad.

ImageMagick's Divide operator:

There is a glitch. Images have three colour channels but only one alpha. Which colour channel should we use to calculate alpha? What if the three colour channels result in different alphas?

Composite Over

For "over" (Src over Dst):

Ra = Sa + Da - Sa * Da

R.Ra = S*Sa*Da + S*Sa * (1-Da) + D*Da * (1-Sa)

Opaque inputs

When the result R is opaque, Ra==1 so either Sa==1 or Da==1.

When dest is fully opaque, Da==1 and:

Ra = Sa + 1 - Sa
   = 1

The formula for "-compose Over -composite" simplifies to:

R = S*Sa + D * (1-Sa)
  = S.Sa + D - D.Sa
  = S.Sa + D.(1-Sa)

Of the four variable R, D, S and Sa, if we know three values we can calculate the fourth.

If we know R, S and D then we can find Sa:

Sa = (R-D) / (S-D)

If we know R, Sa and D then we can find S:

S = (R-D.(1-Sa)) / Sa
  = (R - D + D.Sa) / Sa
  = (R - D)/Sa + D

Even when we know the Result and Dest are opaque, if we don't know either the Source colours (S) or alpha (Sa), there is a virtually infinite number of possible solutions. This is because at each pixel that is different between Result and Dest, there is a virtually infinite number of colours between these two, so there is an infinite range of Source colours that, when composited with the appropriate alpha, makes the Result colour. But if we know (or can guess) the colour at each pixel then we can calculate the alpha values. Or we might know (or can guess) the alpha at each pixel then we can calculate the colours; see Deriving a semi-transparent watermark.

Finally, if we know S, Sa and R then we can find D:

R = S.Sa + D - D.Sa

D - D.Sa = R - S.Sa

D.(1-Sa) = R - S.Sa

D = (R - S.Sa) / (1-Sa)

Note that if Sa==1, we have a divide by zero, and D is indeterminate.

This also has an application in de-watermarking. If a watermark with no opaque pixels (S and Sa) is composited over an opaque input image (D) to make a watermarked image (R), we can reconstruct the original D from R, S and Sa. See the Watermarks page.

Where the watermark contains opaque pixels, Sa==1 so the original pixels cannot be derived.

Inputs with transparency

In the more general case, the D image may contain transparency, so we don't know that Da==1. We can still calculate Da and D, with a slight increase in complexity. First, find Da:

Ra = Sa + Da - Sa * Da

Hence:

Da = (Ra - Sa) / (1-Sa)

Special cases:

Note that Ra >= Sa and Ra >= Da at every pixel.

Now we can find D:

R*Ra = S*Sa*Da + S*Sa * (1-Da) + D*Da * (1-Sa)

Hence:

D = (R*Ra - S*Sa*Da - S*Sa * (1-Da)) / (Da * (1-Sa))
  = (R*Ra - S*Sa*Da - S*Sa * (1-Da)) / (Ra - Sa)
  = (R*Ra - S*Sa*(Da + (1-Da)) / (Ra - Sa)
  = (R*Ra - S*Sa) / (Ra - Sa)

Code to implement the inverse of "-compose Over -composite" is in the Watermarks page.


All images on this page were created by the commands shown, using:

%IM%identify -version
Version: ImageMagick 6.9.5-3 Q16 x86 2016-07-22 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2015 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Visual C++: 180040629
Features: Cipher DPC Modules OpenMP 
Delegates (built-in): bzlib cairo flif freetype jng jp2 jpeg lcms lqr openexr pangocairo png ps rsvg tiff webp xml zlib

To improve internet download speeds, some images may have been automatically converted (by ImageMagick, of course) from PNG to JPG.

Source file for this web page is invcomp.h1. To re-create this web page, execute "procH1 invcomp".


This page, including the images, is my copyright. Anyone is permitted to use or adapt any of the code, scripts or images for any purpose, including commercial use.

Anyone is permitted to re-publish this page, but only for non-commercial use.

Anyone is permitted to link to this page, including for commercial use.


Page version v1.0 6-April-2016.

Page created 14-May-2017 16:24:09.

Copyright © 2017 Alan Gibson.