snibgo's ImageMagick pages

Squishing xyY into shape

An automated method for squishing xyY image colours into the CIE horseshoe, a triangle of primaries, or any shape.

This script sqshxyY.bat is designed to manipulate colours from a raw camera image before it has been encoded with an RGB profile (such as sRGB, Rec2020, or ACES2065-1). The camera may have apparently captured chromaticities that do not fit within an RGB profile, or even within the locus of observable colours -- the CIE "horseshoe" (aka "tongue"). Manipulating xyY encodings before conversion to RGB may give more satisfactory final results.

This script has a similar purpose to the process module barymap described in Colours as barycentric coordinates, but is more automatic in operation.

This changes chroma only, ie the x and y channels of xyY images. The Y channel is unchanged, and hue is unchanged. The alpha channel, if any, is ignored and unchanged.

Some literature refers to "Yxy" colorspace. This is the same as xyY, with the channels in a different order.

This page originated from the pixls.us discussion Camera gamut outside horseshoe?.

References

See also references in the source code barymap.c.

The script

The script sqshxyY.bat takes the following parameters:

Option Description
%1 Input image in xyY space.
%2 Output image in xyY space.
%3 Shape. One of:
    blank or dot.
    horseshoe
    named triangle of primaries eg sRGB
    six numbers eg 0.39,0.41,0.39,0.41,0.39,0.41
Default: sRGB.
%4 White point. One of:
    blank or dot.
    named white point eg D50
    two numbers eg 0.39,0.41
%5 Method. One of:
    clamp
    filmic
    divide
    null
Default: filmic.

Lists of named triangle of primaries, and of white points, is available from:

%IM7DEV%magick xc: -process 'barymap list primaries' NULL: 
sRGB
sRgbD50
Rec709
P3D65
DisplayP3
AdobeRGB
Rec2020
ProPhotoRGB
ACEScg
ACES2065-1
xyY
XYZ
%IM7DEV%magick xc: -process 'barymap list illuminants' NULL: 
A
D50
D60
D65
D65s
D75
D100
D200
D300
D400
E
Aa
D50a
D55a
D65a
D75a
Ea
D50i

If the shape is horseshoe, the white point will be according to the sRGB standard, which is D50. If the shape is specified by name such as sRGB, this also sets the white point according to that standard. (But beware: there are different standards with slightly different values.)

If parameter %4 is given, this will override the WP set by the shape.

The null method does all the work of converting the (x,y,Y) image to (relative_rho,theta,Y), then doesn't tweak the rho, and converts it back. This can be used to verify parts of the script.

The script also uses these environment variables:

Variable Description
sqxCLUTDIR Directory for reading and witing cluts.
sqxHiLimit hiLimit parameter for oogbox.
sqxMAXSTAT_FILE If not blank, write maximum relative chroma to this file.
sqxFORCE_MAX If not blank, assume this maximum relative chroma instead of calculted value.

Creating the large clut isn't quick, even though the resulting files are small. Hence the script will only create the clut if it doesn't exist in the directory named in %sqxCLUTDIR%. It also writes any created cluts file to that directory. If %sqxCLUTDIR% is blank, the directory named in %ICCPROF% is used.

dir %ICCPROF%\sqx_clut* |findstr sqx_clut 
17/01/2019  17:02            16,678 sqx_clut_4096_horseshoe_0_3127_0_329.tiff
17/01/2019  00:08            16,678 sqx_clut_4096_horseshoe_0_31_0_32.tiff
13/12/2018  21:32            16,678 sqx_clut_4096_horseshoe_0_34567_0_3585.tiff
20/12/2018  16:01            16,678 sqx_clut_4096_sRGB_0_3127_0_329.tiff
13/12/2018  21:48            16,678 sqx_clut_4096_sRGB_0_34567_0_3585.tiff
13/12/2018  12:20             2,342 sqx_clut_512_horseshoe_0_34567_0_3585.tiff
13/12/2018  12:28             2,342 sqx_clut_512_sRGB_0_34567_0_3585.tiff

The script calculates the maximum relative chroma, and uses this in the filmic and divide methods. Optionally, it writes this number to the file named in sqxMAXSTAT_FILE. Optionally, it ignores that number and uses sqxFORCE_MAX instead.

Examples

We take a raw image from a camera, convert it to XYZ with dcraw, and from there to xyY.

set SRCRAW=%PICTLIB%20171029\AGA_3443.NEF

%DCRAW% -v -4 -w -W -o 5 -T -O sxs_src.tiff %SRCRAW%

%IM7DEV%magick ^
  sxs_src.tiff ^
  -set colorspace XYZ ^
  -colorspace xyY ^
  -depth 32 ^
  -define quantum:format=floating-point ^
  sxs_xyy.miff

We demonstrate squishing colours into the horseshoe, and into the sRGB triangle. For each, we show the three methods: clamp, filmic and divide.

Squish into horseshoe with clamp method:

rem goto skip

call %PICTBAT%sqshxyY ^
  sxs_xyy.miff ^
  sxs_hc.miff ^
  horseshoe . clamp
if ERRORLEVEL 1 exit /B 1

call %PICTBAT%xyyHorse ^
  sxs_hc.miff ^
  sxs_hc.png
sxs_hc.png sxs_hc_sm.jpg

Squish into horseshoe with filmic method:

call %PICTBAT%sqshxyY ^
  sxs_xyy.miff ^
  sxs_hf.miff ^
  horseshoe . filmic
if ERRORLEVEL 1 exit /B 1

call %PICTBAT%xyyHorse ^
  sxs_hf.miff ^
  sxs_hf.png
sxs_hf.png sxs_hf_sm.jpg

Squish into horseshoe with divide method:

call %PICTBAT%sqshxyY ^
  sxs_xyy.miff ^
  sxs_hd.miff ^
  horseshoe . divide
if ERRORLEVEL 1 exit /B 1

call %PICTBAT%xyyHorse ^
  sxs_hd.miff ^
  sxs_hd.png
sxs_hd.png sxs_hd_sm.jpg

Squish into sRGB triangle with clamp method:

Create a text file for drawing the primaries and WP:

%IM7DEV%magick xc: -process 'barymap v' NULL: 
call %PICTBAT%sqshxyY ^
  sxs_xyy.miff ^
  sxs_sc.miff ^
  sRGB . clamp
if ERRORLEVEL 1 exit /B 1

call %PICTBAT%xyyHorse ^
  sxs_sc.miff ^
  sxs_sc.png

call %PICTBAT%drPrimWp ^
  sxs_sc.png ^
  sxs_sc.png ^
  sxs_primwp.lis
sxs_sc.png sxs_sc_sm.jpg

Squish into sRGB triangle with filmic method:

call %PICTBAT%sqshxyY ^
  sxs_xyy.miff ^
  sxs_sf.miff ^
  sRGB . filmic
if ERRORLEVEL 1 exit /B 1

call %PICTBAT%xyyHorse ^
  sxs_sf.miff ^
  sxs_sf.png

call %PICTBAT%drPrimWp ^
  sxs_sf.png ^
  sxs_sf.png ^
  sxs_primwp.lis
sxs_sf.png sxs_sf_sm.jpg

Squish into sRGB triangle with divide method:

call %PICTBAT%sqshxyY ^
  sxs_xyy.miff ^
  sxs_sd.miff ^
  sRGB . divide
if ERRORLEVEL 1 exit /B 1

call %PICTBAT%xyyHorse ^
  sxs_sd.miff ^
  sxs_sd.png

call %PICTBAT%drPrimWp ^
  sxs_sd.png ^
  sxs_sd.png ^
  sxs_primwp.lis
sxs_sd.png sxs_sd_sm.jpg

How does it work?

We tweak the chroma of colours in an image to put them inside a given shape (a horseshoe, triangle, whatever).

Step 1: In cartesian coordinates (x,y) of xyY space, given a shape and a centre within that shape, we calculate the polar coordinates (rho, theta) of each colour, and normalise rho so it is 1.0*QuantumRange at the edge of the shape. Larger values mean the colour is outside the shape. Smaller values mean the colour is inside the shape.

When the shape is the CIE horseshoe, normalised rho is the excitation purity.

Step 2: Then we tweak values of relative rho to be no greater than 1.0, using one of these methods:

  1. Clamp relative rho to 1.0. All chromas at or below 1.0 are unchanged.
  2. Apply a filmic curve. See Putting OOG back in the box. For example, suppose the maximum chroma is at 1.20 of the shape radius at that hue. The script takes all the values that were between 0.70 and 1.20, squishing them all into the range 0.70 to 1.00. All chromas at or below 0.70 are unchanged. This uses the oogbox process module, which makes no changes when all values are within the range 0.0 to 1.0.
  3. Divide relative rho by max(relative rho). All chromas are changed, in the same proportion. If the maximum relative rho is less than 1.0, chroma will increase for all colours.

Step 3: Then we invert the rho normalisation, and convert the polar coordinates (rho, theta) back to cartesian coordinates (x,y).

The three steps are detailed below, after some preparation work.

Preparation: making cluts

We need a clut image with Nx1 pixels that gives the radius of the shape at any given hue.

We create a square image and draw the horseshoe, or other shape, in white on a black background. Unroll the image, centred on the white point, setting Rmax to the diagonal of the input. ]]Trim it to remove lines that are entirely black from the top, and]] scale to one row. The result is a Nx1 clut file. It represents the radius of the shape at different angles from north, clockwise, as a proportion of the image diagonal. [[The value is 100% at the maximum radius.]]

set HSESHOE=m 426.8 358.8 c-106.3 -106.3 -177.2 -174.2 -222.7 -219.7 -39.2 -39.2 -94 -73.4 -114.3 -73.4 -28.7 0 -39.5 45.1 -39.5 98.5 0 59.3 18.4 227.4 66 306.4 0 0 10.3 15.6 21.7 20 l 288.8 -131.8 z

set DR_HSESHOE=translate -47.8,19.2 path '%HSESHOE%'

set WPx=0.31
set WPy=0.32

%IM7DEV%magick ^
  -size 500x500 xc:Black ^
  -fill White -stroke None ^
  -draw "scale %%[fx:w/512],%%[fx:h/512] stroke-width %%[fx:512/w] %DR_HSESHOE%" ^
  +write sxs_shape.png ^
  -rotate 180 ^
  -virtual-pixel Black ^
  -distort depolar %%[fx:hypot(w,h)/sqrt(2)],0,%%[fx:w-w*%WPx%],%%[fx:h*%WPy%] ^
  -scale "x1^!" ^
  -flop ^
  -define quantum:format=floating-point ^
  -depth 32 ^
  sxs_clut.miff

sxs_shape.png

sxs_shape.png

Show a graph of the clut:

call %PICTBAT%graphLineCol ^
  sxs_clut.miff . . sxs_clut_glc.png
sxs_clut_glc.png

The script writes cluts with a filename "sqx_clut_*.tiff". The filename is specific to the clut size, the shape, and the white point.

Step 1: Convert to polar and normalise rho

[Putting OOH into the box: from an image in xyY, we can express chromaticity as (rho,theta) roughly (chroma,hue) where rho is normalised to be 1.0 at the horseshoe. We can also do the inverse.

This readily gives the maximum rho, hence a divisor for barymap gain. Better, a scheme such as oogbox.htm, a "filmic" curve.]

From the xyY image: subtract the xy of the white point, convert the cartesian coordinates (x and y) channels to polar (rho and theta), and save the three channels.

rho ranges from 0.0 to 100% of Quantum, increasing anti-clockwise, at zero and 100% where x=WPx and y<WPy, ie the line "south" of the WP on the conventional diagram.

FIXME: also check image has embedded XYZ profile.

%IM7DEV%magick ^
  sxs_xyy.miff ^
  -set colorspace sRGB ^
  -strip ^
  +profile "*" ^
  -define quantum:format=floating-point ^
  -depth 32 ^
  -process 'rhotheta offset %WPx%,%WPy%' ^
  -channel RGB -separate +channel ^
  ( -clone 0 ^
    -write sxs_rho.miff ^
    +delete ) ^
  -delete 0 ^
  ( -clone 0 ^
    -write sxs_theta.miff ^
    -negate ^
    +delete ) ^
  -delete 0 ^
  sxs_Y.miff

Find the radius of the shape at every hue:

%IM7DEV%magick ^
  sxs_theta.miff ^
  sxs_clut.miff ^
  -clut ^
  -colorspace Gray ^
  -define quantum:format=floating-point ^
  -depth 32 ^
  sxs_rho0.miff

sxs_rho0.miff has the radius of the shape at that hue. [[A value of 1.0*QuantumRange is the largest radius.]]

Find the relative rho. That is, at each pixel, rho divided by the radius of the shape at that hue.

%IM7DEV%magick ^
  sxs_rho.miff ^
  sxs_rho0.miff ^
  -define compose:clamp=off ^
  -compose DivideSrc -composite ^
  -colorspace Gray ^
  -define quantum:format=floating-point ^
  -depth 32 ^
  sxs_rho_rel.miff
call %PICTBAT%rhoStats sxs_rho_rel.miff 
MIN=6.8530497e-08 MEAN=0.17991904 MAX=1.1387836
PROPOUT=2.4222891e-06

Step 2: Tweak rho

 %IM7DEV%magick ^
  sxs_rho_rel.miff ^
  -process 'oogbox h 0.7 channel 0 v' ^
  -define quantum:format=floating-point ^
  -depth 32 ^
  sxs_rho_rel2.miff

%IM7DEV%magick ^
  sxs_rho_rel.miff ^
  -clamp ^
  -define quantum:format=floating-point ^
  -depth 32 ^
  sxs_rho_rel2.miff

%IM7DEV%magick ^
  sxs_rho_rel.miff ^
  -evaluate Divide %%[fx:maxima] ^
  -define quantum:format=floating-point ^
  -depth 32 ^
  sxs_rho_rel2.miff

Step 3: Denormalise rho and convert to cartesian

This is the inverse of step 1.

Multiply the [adjusted] relative rho by the rho at that hue to get the absolute rho. Combine this with the unchanged theta and Y to get a (rho,theta,Y) image. The inverse rhotheta module converts this to xyY, and adds the origin xy.

%IM7DEV%magick ^
  sxs_rho_rel2.miff ^
  sxs_rho0.miff ^
  -define compose:clamp=off ^
  -compose Multiply -composite ^
  -define quantum:format=floating-point ^
  -depth 32 ^
  sxs_theta.miff ^
  sxs_Y.miff ^
  -combine ^
  -set colorspace xyY ^
  -process 'rhotheta offset %WPx%,%WPy% inverse' ^
  sxs_xyy_new.miff

Show the xy chart and a JPG sRGB version:

call %PICTBAT%xyyHorse ^
  sxs_xyy_new.miff sxs_xyy_new_out.png
sxs_xyy_new_out.png sxs_xyy_new_sm.jpg

Show a sRGB version:

%IM7DEV%magick ^
  sxs_xyy_new.miff ^
  -set colorspace xyY -colorspace XYZ ^
  -profile sRGB.icc ^
  -resize 600x600 ^
  sxs_srgb.jpg
sxs_srgb.jpg

Making and using HALD clut

A hald clut might transform:

  1. from xyY to xyY, as above, with filmic curve or whatever;
  2. or from xyY directly to sRGB or other colorspace;
  3. or from XYZ directly to sRGB or other colorspace.

The filmic and divide methods automatically use the maximum relative chroma of the image. To create a hald clut that does the same processing, we need to find the maximum that was used, and force it to be used when creating the hald.

:skip

set sqxMAXSTAT_FILE=sxs_max_stat.lis

call %PICTBAT%sqshxyY ^
  sxs_xyy.miff ^
  NULL: ^
  sRGB . filmic
if ERRORLEVEL 1 exit /B 1

for /F %%A in (%sqxMAXSTAT_FILE%) do set maxChroma=%%A

echo maxChroma=%maxChroma% 
maxChroma=2.1404362 

We make an identity hald clut in xyY colorspace:

%IM7DEV%magick ^
  hald:16 ^
  -set colorspace xyY ^
  -define quantum:format=floating-point ^
  -depth 32 ^
  sxs_hald.miff
call %PICTBAT%xyyHorse ^
  sxs_hald.miff sxs_hald.png
sxs_hald.png sxs_hald_sm.jpg

We squish values:

set sqxFORCE_MAX=%maxChroma%

call %PICTBAT%sqshxyY ^
  sxs_hald.miff ^
  sxs_hald2.miff ^
  sRGB . filmic
if ERRORLEVEL 1 exit /B 1

set sqxFORCE_MAX=
call %PICTBAT%xyyHorse ^
  sxs_hald2.miff sxs_hald2.png
sxs_hald2.png sxs_hald2_sm.jpg

We could use this result, sxs_hald2.miff, to convert xyY images to squished xyY images. However, we might use it on an image with relative chroma greater than %maxChroma%, so we should ensure the hald will gracefully clamp those colours.

call %PICTBAT%sqshxyY ^
  sxs_hald2.miff ^
  sxs_hald3.miff ^
  sRGB . clamp
call %PICTBAT%xyyHorse ^
  sxs_hald3.miff sxs_hald3.png
sxs_hald3.png sxs_hald3_sm.jpg

[Why is the XYZ triangle not an exact triangle?]

Now the hald contains only chromaticities that are within the sRGB triangle.

%IM7DEV%magick ^
  sxs_xyy.miff ^
  sxs_hald3.miff ^
  -strip ^
  -hald-clut ^
  -alpha off ^
  -set colorspace xyY ^
+write sxs_s_xyy.miff ^
  -colorspace sRGB ^
  sxs_s.miff

[Do we still have the XYZ profile?]

%IM7DEV%magick ^
  sxs_s_xyy.miff ^
  -verbose ^
  info: 
 Image:
  Filename: sxs_s_xyy.miff
  Format: MIFF (Magick Image File Format)
  Class: DirectClass
  Geometry: 4924x7378+0+0
  Resolution: 300x300
  Print size: 16.4133x24.5933
  Units: PixelsPerInch
  Colorspace: xyY
  Type: TrueColor
  Endianness: Undefined
  Depth: 32-bit
  Channels: 3.0
  Channel depth:
    Channel 0: 32-bit
    Channel 1: 32-bit
    Channel 2: 32-bit
  Channel statistics:
    Pixels: 36329272
    Channel 0:
      min: 7.4853658e+08  (0.17428225)
      max: 2.4135498e+09 (0.56194836)
      mean: 1.3335688e+09 (0.31049567)
      median: 1.3486943e+09 (0.31401736)
      standard deviation: 1.4157904e+08 (0.032963938)
      kurtosis: 1.0115166
      skewness: -0.66618521
      entropy: 0.91122605
    Channel 1:
      min: 4.2037536e+08  (0.097876266)
      max: 2.4851904e+09 (0.57862848)
      mean: 1.4969683e+09 (0.3485401)
      median: 1.4723882e+09 (0.3428171)
      standard deviation: 2.2563594e+08 (0.052534961)
      kurtosis: 0.75882076
      skewness: 0.27167952
      entropy: 0.9231443
    Channel 2:
      min: 4456515.5  (0.0010376134)
      max: 4.2949673e+09 (1)
      mean: 4.4218551e+08 (0.10295434)
      median: 1.4306728e+08 (0.033310447)
      standard deviation: 8.9787389e+08 (0.20905256)
      kurtosis: 10.170213
      skewness: 3.3411297
      entropy: 0.80890199
  Image statistics:
    Overall:
      min: 4456515.5  (0.0010376134)
      max: 4.2949673e+09 (1)
      mean: 1.0909075e+09 (0.2539967)
      median: 9.8804993e+08 (0.2300483)
      standard deviation: 4.2169629e+08 (0.098183819)
      kurtosis: 3.9801834
      skewness: 0.98220801
      entropy: 0.88109078
  Rendering intent: Perceptual
  Gamma: 1
  Chromaticity:
    red primary: (0.64,0.33,0.03)
    green primary: (0.3,0.6,0.1)
    blue primary: (0.15,0.06,0.79)
    white point: (0.3127,0.329,0.3583)
  Matte color: grey74
  Background color: white
  Border color: srgb(223,223,223)
  Transparent color: black
  Interlace: None
  Intensity: Undefined
  Compose: Over
  Page geometry: 4924x7378+0+0
  Dispose: Undefined
  Iterations: 0
  Compression: None
  Orientation: TopLeft
  Properties:
    date:create: 2024-06-16T11:17:08+00:00
    date:modify: 2024-06-16T11:17:08+00:00
    date:timestamp: 2024-06-16T11:17:12+00:00
    exif:ExposureTime: 0.033333
    exif:FNumber: 8.000000
    exif:FocalLength: 20.000000
    exif:ISOSpeedRatings: 400
    icc:copyright: auto-generated by dcraw
    icc:description: XYZ
    signature: aedd19af3ad7e967b90b0345cdad28c5deb8d89b6e25dadc72f6fe068d45b060
    tiff:alpha: unspecified
    tiff:artist: Alan Gibson                         
    tiff:endian: lsb
    tiff:make: Nikon
    tiff:model: D800
    tiff:photometric: RGB
    tiff:rows-per-strip: 1
    tiff:software: dcraw v9.27
    tiff:timestamp: 2017:10:29 15:03:38
  Tainted: False
  Filesize: 415.75631MiB
  Number pixels: 36329272
  Pixel cache type: Memory
  Pixels per second: 351.46215MP
  Time-to-live: 0:0:0:0  2024-06-16T11:17:12Z
  User time: 0.109u
  Elapsed time: 0:01.103
  Version: ImageMagick 7.1.1-27 (Beta) Q32-HDRI x86_64 570a9a048:20240108 https://imagemagick.org
%IM7DEV%magick ^
  sxs_s.miff ^
  -verbose ^
  info: 
 Image:
  Filename: sxs_s.miff
  Format: MIFF (Magick Image File Format)
  Class: DirectClass
  Geometry: 4924x7378+0+0
  Resolution: 300x300
  Print size: 16.4133x24.5933
  Units: PixelsPerInch
  Colorspace: sRGB
  Type: TrueColor
  Endianness: Undefined
  Depth: 32-bit
  Channels: 3.0
  Channel depth:
    Red: 32-bit
    Green: 32-bit
    Blue: 32-bit
  Channel statistics:
    Pixels: 36329272
    Red:
      min: 123168.21  (2.8677334e-05)
      max: 5.0079713e+09 (1.1660092)
      mean: 1.0511811e+09 (0.24474718)
      median: 7.9514374e+08 (0.18513383)
      standard deviation: 8.5554316e+08 (0.19919666)
      kurtosis: 7.3604713
      skewness: 2.7260088
      entropy: 0.89221996
    Green:
      min: 68238288  (0.015887965)
      max: 4.3477868e+09 (1.012298)
      mean: 1.1749112e+09 (0.27355532)
      median: 8.8363098e+08 (0.20573637)
      standard deviation: 8.6788035e+08 (0.20206914)
      kurtosis: 5.0127444
      skewness: 2.3318772
      entropy: 0.90631246
    Blue:
      min: 3405485  (0.00079290126)
      max: 4.3210604e+09 (1.0060753)
      mean: 1.1175159e+09 (0.26019196)
      median: 7.5795264e+08 (0.1764746)
      standard deviation: 9.8650797e+08 (0.22968929)
      kurtosis: 3.5820002
      skewness: 2.1433949
      entropy: 0.90163109
  Image statistics:
    Overall:
      min: 123168.21  (2.8677334e-05)
      max: 5.0079713e+09 (1.1660092)
      mean: 1.1145361e+09 (0.25949815)
      median: 8.1224245e+08 (0.18911493)
      standard deviation: 9.0331049e+08 (0.21031836)
      kurtosis: 5.3184053
      skewness: 2.4004269
      entropy: 0.9000545
  Rendering intent: Perceptual
  Gamma: 0.454545
  Chromaticity:
    red primary: (0.64,0.33,0.03)
    green primary: (0.3,0.6,0.1)
    blue primary: (0.15,0.06,0.79)
    white point: (0.3127,0.329,0.3583)
  Matte color: grey74
  Background color: white
  Border color: srgb(223,223,223)
  Transparent color: black
  Interlace: None
  Intensity: Undefined
  Compose: Over
  Page geometry: 4924x7378+0+0
  Dispose: Undefined
  Iterations: 0
  Compression: None
  Orientation: TopLeft
  Properties:
    date:create: 2024-06-16T11:17:11+00:00
    date:modify: 2024-06-16T11:17:11+00:00
    date:timestamp: 2024-06-16T11:17:28+00:00
    exif:ExposureTime: 0.033333
    exif:FNumber: 8.000000
    exif:FocalLength: 20.000000
    exif:ISOSpeedRatings: 400
    icc:copyright: auto-generated by dcraw
    icc:description: XYZ
    signature: fbb39298b1c4979415e72f779dbb9da9eb478b982f36bf09c217d91fce3200d3
    tiff:alpha: unspecified
    tiff:artist: Alan Gibson                         
    tiff:endian: lsb
    tiff:make: Nikon
    tiff:model: D800
    tiff:photometric: RGB
    tiff:rows-per-strip: 1
    tiff:software: dcraw v9.27
    tiff:timestamp: 2017:10:29 15:03:38
  Tainted: False
  Filesize: 415.75632MiB
  Number pixels: 36329272
  Pixel cache type: Memory
  Pixels per second: 377.93034MP
  Time-to-live: 0:0:0:0  2024-06-16T11:17:28Z
  User time: 0.094u
  Elapsed time: 0:01.096
  Version: ImageMagick 7.1.1-27 (Beta) Q32-HDRI x86_64 570a9a048:20240108 https://imagemagick.org

We may as well incorporate the next stage, which is converting the xyY encoding to an RGB encoding.

As I use dcraw to make XYZ images, and dcraw embeds an XYZ ICC profile, I will convert to XYZ then assign that profile to the hald, then convert the hald to sRGB.

%IM7DEV%magick ^
  sxs_hald3.miff ^
  -strip ^
  -colorspace XYZ ^
  -set profile "%ICCPROFFWD%/dcrXYZ.icc" ^
  -profile "%ICCPROFFWD%/sRGB-elle-V4-srgbtrc.icc" ^
  sxs_hald4.tiff
%IM7DEV%magick ^
  sxs_hald3.miff ^
  -strip ^
  -set colorspace xyY ^
  -colorspace sRGB ^
  sxs_hald4a.miff

[BUG: hald4 has negative values.]

%IM7DEV%magick ^
  sxs_xyy.miff ^
  sxs_hald4a.miff ^
  -hald-clut ^
  -alpha off ^
  -strip ^
  -set colorspace sRGB ^
  sxs_x4a.miff

Now we can convert xyY images to sRGB with that hald, and assign the sRGB profile:

%IM7DEV%magick ^
  sxs_xyy.miff ^
  sxs_hald4.tiff ^
  -hald-clut ^
  -alpha off ^
  -strip ^
  -set colorspace sRGB ^
  -set profile %ICCPROFFWD%/sRGB-elle-V4-srgbtrc.icc ^
  sxs_srgb.tiff
%IM7DEV%magick ^
  sxs_src.tiff ^
  sxs_hald4.tiff ^
  -hald-clut ^
  -alpha off ^
  -strip ^
  -set colorspace sRGB ^
  -set profile %ICCPROFFWD%/sRGB-elle-V4-srgbtrc.icc ^
  sxs_srgb2.tiff

As a check, all pixel values should be in the range 0 to 100%.

%IM7DEV%magick ^
  sxs_srgb.tiff ^
  -verbose ^
  info: 
 Image:
  Filename: sxs_srgb.tiff
  Format: TIFF (Tagged Image File Format)
  Mime type: image/tiff
  Class: DirectClass
  Geometry: 4924x7378+0+0
  Resolution: 300x300
  Print size: 16.4133x24.5933
  Units: PixelsPerInch
  Colorspace: sRGB
  Type: TrueColor
  Endianness: Undefined
  Depth: 32-bit
  Channels: 3.0
  Channel depth:
    Red: 32-bit
    Green: 32-bit
    Blue: 32-bit
  Channel statistics:
    Pixels: 36329272
    Red:
      min: -7.0414546e+09  (-1.6394664)
      max: 2.7220413e+09 (0.63377464)
      mean: 88182815 (0.020531662)
      median: 7951049.5 (0.001851248)
      standard deviation: 5.5761269e+08 (0.12982932)
      kurtosis: 14.516477
      skewness: 0.97806606
      entropy: 0.61986251
    Green:
      min: -1.3132924e+09  (-0.30577472)
      max: 2.4574259e+09 (0.57216406)
      mean: 2.0908802e+08 (0.048682099)
      median: 22778076 (0.0053034341)
      standard deviation: 5.2403698e+08 (0.12201187)
      kurtosis: 9.3330579
      skewness: 3.2287864
      entropy: 0.74443715
    Blue:
      min: -62755620  (-0.014611431)
      max: 2.399358e+09 (0.55864406)
      mean: 2.3756478e+08 (0.055312361)
      median: 13388174 (0.0031171772)
      standard deviation: 5.9520412e+08 (0.13858176)
      kurtosis: 6.3408956
      skewness: 2.7877829
      entropy: 0.70687993
  Image statistics:
    Overall:
      min: -7.0414546e+09  (-1.6394664)
      max: 2.7220413e+09 (0.63377464)
      mean: 1.7827854e+08 (0.041508707)
      median: 14705766 (0.0034239531)
      standard deviation: 5.5895126e+08 (0.13014098)
      kurtosis: 10.063477
      skewness: 2.3315451
      entropy: 0.6903932
  Rendering intent: Perceptual
  Gamma: 0.454545
  Chromaticity:
    red primary: (0.64,0.33,0.03)
    green primary: (0.3,0.6,0.1)
    blue primary: (0.15,0.06,0.79)
    white point: (0.3127,0.329,0.3583)
  Matte color: grey74
  Background color: white
  Border color: srgb(223,223,223)
  Transparent color: black
  Interlace: None
  Intensity: Undefined
  Compose: Over
  Page geometry: 4924x7378+0+0
  Dispose: Undefined
  Iterations: 0
  Compression: None
  Orientation: TopLeft
  Profiles:
    Profile-icc: 1256 bytes
  Properties:
    date:create: 2024-06-16T11:18:46+00:00
    date:modify: 2024-06-16T11:18:46+00:00
    date:timestamp: 2024-06-16T11:19:12+00:00
    icc:copyright: Copyright 2016, Elle Stone (http://ninedegreesbelow.com/), CC-BY-SA 3.0 Unported (https://creativecommons.org/licenses/by-sa/3.0/legalcode).
    icc:description: sRGB-elle-V4-srgbtrc.icc
    icc:manufacturer: sRGB chromaticities from A Standard Default Color Space for the Internet - sRGB, http://www.w3.org/Graphics/Color/sRGB; also see http://www.color.org/specification/ICC1v43_2010-12.pdf
    quantum:format: floating-point
    signature: a9d556fae803f90ea5c511330cb9e9533e1c55ecf002a355b236d050ce15914c
    tiff:alpha: unspecified
    tiff:endian: lsb
    tiff:photometric: RGB
    tiff:rows-per-strip: 16
  Tainted: False
  Filesize: 415.76066MiB
  Number pixels: 36329272
  Pixel cache type: Memory
  Pixels per second: 376.05061MP
  Time-to-live: 0:0:0:0  2024-06-16T11:19:12Z
  User time: 0.093u
  Elapsed time: 0:01.096
  Version: ImageMagick 7.1.1-27 (Beta) Q32-HDRI x86_64 570a9a048:20240108 https://imagemagick.org
%IM7DEV%magick ^
  sxs_srgb2.tiff ^
  -verbose ^
  info: 
 Image:
  Filename: sxs_srgb2.tiff
  Format: TIFF (Tagged Image File Format)
  Mime type: image/tiff
  Class: DirectClass
  Geometry: 4924x7378+0+0
  Resolution: 300x300
  Print size: 16.4133x24.5933
  Units: PixelsPerInch
  Colorspace: sRGB
  Type: TrueColor
  Endianness: Undefined
  Depth: 16/32-bit
  Channels: 3.0
  Channel depth:
    Red: 32-bit
    Green: 32-bit
    Blue: 32-bit
  Channel statistics:
    Pixels: 36329272
    Red:
      min: 0  (0)
      max: 54680 (0.83436332)
      mean: 3062.9009 (0.046736873)
      median: 0 (0)
      standard deviation: 10442.777 (0.15934656)
      kurtosis: 8.7935526
      skewness: 3.2438942
      entropy: 0.086646456
    Green:
      min: 0  (0)
      max: 42397 (0.64693675)
      mean: 3994.6254 (0.060954076)
      median: 344 (0.0052491035)
      standard deviation: 9380.2863 (0.14313399)
      kurtosis: 6.0483498
      skewness: 2.7224373
      entropy: 0.74323548
    Blue:
      min: 0  (0)
      max: 65535 (1)
      mean: 4660.77 (0.071118791)
      median: 1779 (0.0271458)
      standard deviation: 7860.2871 (0.11994029)
      kurtosis: 12.946593
      skewness: 3.3303195
      entropy: 0.80853491
  Image statistics:
    Overall:
      min: 0  (0)
      max: 65535 (1)
      mean: 3906.0988 (0.059603247)
      median: 707.66667 (0.010798301)
      standard deviation: 9227.7834 (0.14080695)
      kurtosis: 9.2628317
      skewness: 3.0988837
      entropy: 0.54613895
  Rendering intent: Perceptual
  Gamma: 0.454545
  Chromaticity:
    red primary: (0.64,0.33,0.03)
    green primary: (0.3,0.6,0.1)
    blue primary: (0.15,0.06,0.79)
    white point: (0.3127,0.329,0.3583)
  Matte color: grey74
  Background color: white
  Border color: srgb(223,223,223)
  Transparent color: black
  Interlace: None
  Intensity: Undefined
  Compose: Over
  Page geometry: 4924x7378+0+0
  Dispose: Undefined
  Iterations: 0
  Compression: None
  Orientation: TopLeft
  Profiles:
    Profile-icc: 1256 bytes
  Properties:
    date:create: 2024-06-16T11:19:12+00:00
    date:modify: 2024-06-16T11:19:12+00:00
    date:timestamp: 2024-06-16T11:19:28+00:00
    icc:copyright: Copyright 2016, Elle Stone (http://ninedegreesbelow.com/), CC-BY-SA 3.0 Unported (https://creativecommons.org/licenses/by-sa/3.0/legalcode).
    icc:description: sRGB-elle-V4-srgbtrc.icc
    icc:manufacturer: sRGB chromaticities from A Standard Default Color Space for the Internet - sRGB, http://www.w3.org/Graphics/Color/sRGB; also see http://www.color.org/specification/ICC1v43_2010-12.pdf
    signature: 56989dbd4fbdd407c71c9e3a8f1fb7c67b3b98a686f05d82fc8d23d1813e96be
    tiff:alpha: unspecified
    tiff:endian: lsb
    tiff:photometric: RGB
    tiff:rows-per-strip: 32
  Tainted: False
  Filesize: 207.88107MiB
  Number pixels: 36329272
  Pixel cache type: Memory
  Pixels per second: 387.21403MP
  Time-to-live: 0:0:0:0  2024-06-16T11:19:28Z
  User time: 0.093u
  Elapsed time: 0:01.093
  Version: ImageMagick 7.1.1-27 (Beta) Q32-HDRI x86_64 570a9a048:20240108 https://imagemagick.org

Future

There is scope for squishing that is adaptive to hue. For example, this image has higher chroma in green and blue than in red, so an adaptive mechanism might reduce chroma less for red colours.

For video and similar work, the script is too automatic. We could take an aggregate xyY of all the frames, calculate the required squishing, and apply the same squishing to all frames.

The process isn't fast. For practical use, it may be better to use it to make HALD cluts to cover some typical situations, and automatically choose the best of these based on image statistics.

Scripts

For convenience, .bat scripts are also available in a single zip file. See Zipped BAT files.

sqshxyY.bat

rem %1 input image in xyY space
rem %2 output image in xyY space
rem %3 Primaries and WP: "horseshoe", "sRGB", etc.
rem %4 white point
rem %5 method: clamp, filmic, divide or null

@rem
@rem Also uses:
@rem   sqxCLUTDIR Directory for cluts. [%ICCPROF%]
@rem   sqxHiLimit hiLimit for oogbox [0.7]
@rem   sqxMAXSTAT_FILE name for maximum statistic [none]
@rem   sqxFORCE_MAX assume this maximum instead of statistic [no force]
@rem
@rem Updated:
@rem   14-June-2024 Remove channel option from oogbox.

@rem FIXME: also check image has embedded XY profile.


@if "%1"=="" findstr /B "rem @rem" %~f0 & exit /B 1

@setlocal enabledelayedexpansion

@call echoOffSave

call %PICTBAT%setInOut %1 sqx

if not "%2"=="" if not "%2"=="." set OUTFILE=%2

set SHAPE=%~3
if "%SHAPE%"=="." set SHAPE=
if "%SHAPE%"=="" set SHAPE=sRGB

set WhtPnt=%~4
if "%WhtPnt%"=="." set WhtPnt=

set METHOD=%5
if "%METHOD%"=="." set METHOD=
if "%METHOD%"=="" set METHOD=filmic

if /I not %METHOD%==clamp if /I not %METHOD%==divide if /I not %METHOD%==filmic if /I not %METHOD%==null (
  echo %0: Bad method [%METHOD%]
  exit /B 1
)

if "%sqxCLUTDIR%"=="" set sqxCLUTDIR=%ICCPROF%

if "%sqxHiLimit%"=="" set sqxHiLimit=0.7

set TMPDIR=\temp

set CLUTDIM=4096

rem FIXME: CLUT_NAME also needs WP, including for horseshoe.

set CLUT_NAME=%SHAPE: =_%

set MultiParam=1
if %CLUT_NAME%==%SHAPE% set MultiParam=0

set WPx=0.31
set WPy=0.32

:: But what if horseshoe?
:: How does user declare WP for horseshoe?
if "%SHAPE%"=="horseshoe" (
  call :getbary "outPrim sRGB"
  if ERRORLEVEL 1 exit /B 1
) else if %MultiParam%==0 (
  call :getbary "outPrim %SHAPE%"
  if ERRORLEVEL 1 exit /B 1
) else (
  call :getbary "%SHAPE%"
  if ERRORLEVEL 1 exit /B 1
)

if ERRORLEVEL 1 exit /B 1

if not "%WhtPnt%"=="" (
  call :getwp "%WhtPnt%"
)

set CLUT_NAME=%CLUTDIM%_%CLUT_NAME%_%WPx:.=_%_%WPy:.=_%

echo %0: CLUT_NAME=%CLUT_NAME%

set CLUT_FILE=%sqxCLUTDIR%\sqx_clut_%CLUT_NAME%.tiff

echo %0: CLUT_FILE=%CLUT_FILE%

if exist %CLUT_FILE% goto clut_ok


if %MultiParam%==1 goto MultiName
if /I not %SHAPE%==horseshoe goto notHorseshoe

echo %0: Making %CLUT_FILE%

set HSESHOE=m 426.8 358.8 c-106.3 -106.3 -177.2 -174.2 -222.7 -219.7 -39.2 -39.2 -94 -73.4 -114.3 -73.4 -28.7 0 -39.5 45.1 -39.5 98.5 0 59.3 18.4 227.4 66 306.4 0 0 10.3 15.6 21.7 20 l 288.8 -131.8 z

set DR_HSESHOE=translate -47.8,19.2 path '%HSESHOE%'

%IM7DEV%magick ^
  -size %CLUTDIM%x%CLUTDIM% xc:Black ^
  -fill White -stroke None ^
  -draw "scale %%[fx:w/512],%%[fx:h/512] stroke-width %%[fx:512/w] %DR_HSESHOE%" ^
  -rotate 180 ^
  -virtual-pixel Black ^
  -distort depolar %%[fx:hypot(w,h)/sqrt(2)],0,%%[fx:w-w*%WPx%],%%[fx:h*%WPy%] ^
  -scale "x1^!" ^
  -flop ^
  -alpha off ^
  -define quantum:format=floating-point ^
  -depth 32 ^
  %CLUT_FILE%

goto clut_ok

:MultiName

call :drTri "%SHAPE%"
if ERRORLEVEL 1 exit /B 1

goto clut_ok

:notHorseshoe

call :drTri "outPrim %SHAPE%"
if ERRORLEVEL 1 exit /B 1

goto clut_ok


:clut_ok

echo %0: Applying method %METHOD% forwards:

set wrMAXSTAT=
if not "%sqxMAXSTAT_FILE%"=="" (
  set wrMAXSTAT=-format %%[fx:maxima] +write info:%sqxMAXSTAT_FILE%
)

set sTWEAK=
if /I %METHOD%==clamp (
  set sTWEAK=-clamp
) else if /I %METHOD%==divide (
  if "%sqxFORCE_MAX%"=="" (
    set sTWEAK=-evaluate Divide %%[fx:maxima]
  ) else (
    set sTWEAK=-evaluate Divide %sqxFORCE_MAX%
  )
) else if /I %METHOD%==filmic (
  if "%sqxFORCE_MAX%"=="" (
    set sTWEAK=-process 'oogbox loLimit 0 hiLimit %sqxHiLimit% v'
  ) else (
    set sTWEAK=-process 'oogbox forceMax %sqxFORCE_MAX% loLimit 0 hiLimit %sqxHiLimit% v'
  )
) else if /I %METHOD%==null (
  set sTWEAK=
) else (
  echo %0: Bad method [%METHOD%]
  exit /B 1
)

%IM7DEV%magick ^
  %INFILE% ^
  -set colorspace sRGB ^
  -strip ^
  +profile "*" ^
  -define quantum:format=floating-point ^
  -depth 32 ^
  -process 'rhotheta offset %WPx%,%WPy%' ^
  -channel RGB -separate +channel ^
  ( -clone 1 ^
    -write %TMPDIR%\sxs_theta.miff ^
    %CLUT_FILE% ^
    -clut ^
    -colorspace Gray ^
    -write %TMPDIR%\sxs_rho0.miff ^
    +delete ) ^
  -delete 1 ^
  ( -clone 0 ^
    %TMPDIR%\sxs_rho0.miff ^
    -define compose:clamp=off ^
    -compose DivideSrc -composite ^
    -colorspace Gray ^
    %wrMAXSTAT% ^
    %sTWEAK% ^
    -write %TMPDIR%\sxs_rho_rel2.miff ^
    +delete ) ^
  -delete 0 ^
  %TMPDIR%\sxs_Y.miff

echo %0: Reverse:

if /I not %OUTFILE%==NULL: %IM7DEV%magick ^
  %TMPDIR%\sxs_rho_rel2.miff ^
  %TMPDIR%\sxs_rho0.miff ^
  -define compose:clamp=off ^
  -compose Multiply -composite ^
  -define quantum:format=floating-point ^
  -depth 32 ^
  %TMPDIR%\sxs_theta.miff ^
  %TMPDIR%\sxs_Y.miff ^
  -combine ^
  -set colorspace xyY ^
  -process 'rhotheta offset %WPx%,%WPy% inverse' ^
  %OUTFILE%


call echoRestore

@endlocal

@exit /B 0

::---------------------------------------
:: Subroutines

:drTri

call :getbary %1
if ERRORLEVEL 1 exit /B 1

set i=0
for %%A in (%NUMS%) do (
  if "!prev!"=="" (
    set prev=%%A
  ) else (
    set /A rad=8
    echo ow !prev! and %%A !rad!
    set sDRAW[!i!]=%%[fx:w*!prev!],%%[fx:h-h*%%A]
    set prev=
    set /A i+=1
  )
)

set sDRAW

%IM7DEV%magick ^
  -size %CLUTDIM%x%CLUTDIM% xc:Black ^
  -fill White -stroke None ^
  -draw "polygon %sDRAW[0]%,%sDRAW[1]%,%sDRAW[2]%" ^
  -rotate 180 ^
  -virtual-pixel Black ^
  -distort depolar %%[fx:hypot(w,h)/sqrt(2)],0,%%[fx:w-w*%WPx%],%%[fx:h*%WPy%] ^
  -scale "x1^!" ^
  -flop ^
  -alpha off ^
  -define quantum:format=floating-point ^
  -depth 32 ^
  %CLUT_FILE%

exit /B 0

::---------------------------------------

:getbary

set ARGS=%~1
echo %0: ARGS=%ARGS%

set NUMS=
set WP=
for /F "usebackq tokens=1,2" %%A in (`%IM7DEV%magick ^
  xc: ^
  -process 'barymap inChannels xyY outChannels xyY %ARGS% f stdout v' ^
  NULL: 2^>^&1`) do (
  echo %0: [%%A][%%B]
  if %%A==outPrim set NUMS=%%B
  if %%A==outWP set WP=%%B
)

rem FIXME: no longer outPrim etc.

echo %0: WP=%WP%

if "%NUMS%"=="" (
  echo %0: barymap failed ARGS=%ARGS%
  exit /B 1
)

for /F "tokens=1,2 delims=," %%A in ("%WP%") do (
  set WPx=%%A
  set WPy=%%B
)

exit /B 0


::---------------------------------------

:getwp

set ARGS=%~1
echo %0: ARGS=%ARGS%

set NUMS=
set WP=
for /F "usebackq tokens=1,2" %%A in (`%IM7DEV%magick ^
  xc: ^
  -process 'barymap inChannels xyY outChannels xyY  outWP %ARGS% f stdout v' ^
  NULL: 2^>^&1`) do (
  if %%A==outWP set WP=%%B
)

echo %0: WP=%WP%

if "%WP%"=="" (
  echo %0: barymap failed ARGS=%ARGS%
  exit /B 1
)

for /F "tokens=1,2 delims=," %%A in ("%WP%") do (
  set WPx=%%A
  set WPy=%%B
)

exit /B 0

xyyHorse.bat

rem %1 input image in xyY space.
rem %2 output black/white histogram with horseshoe.
rem Also creates small sRGB version of input image.

%IM7DEV%magick ^
  %1 ^
  -strip ^
  ( +clone ^
    -set colorspace xyY ^
    -resize 512x512 ^
    -colorspace sRGB ^
    +write %~n1_sm.jpg ^
    +delete ^
  ) ^
  -process 'plotrg dim 512 norm verbose' ^
  -set colorspace sRGB ^
  -flip ^
+write %~n1_prop.png ^
  -fill White +opaque Black ^
  -set colorspace sRGB ^
  %2

if ERRORLEVEL 1 exit /B 1

call %PICTBAT%cieHorseshoe %2 %2

if ERRORLEVEL 1 exit /B 1

cieHorseshoe.bat

set INFILE=%~1
set OUTFILE=%2
set COLS=%~3

if "%INFILE%"=="." set INFILE=
if "%INFILE%"=="" set INFILE=-size 512x512 xc:

if "%OUTFILE%"=="." set OUTFILE=
if "%OUTFILE%"=="" set OUTFILE=ch.png

if "%COLS%"=="." set COLS=
if "%COLS%"=="" set COLS=-fill None -stroke Red

set HSESHOE=m 426.8 358.8 c-106.3 -106.3 -177.2 -174.2 -222.7 -219.7 -39.2 -39.2 -94 -73.4 -114.3 -73.4 -28.7 0 -39.5 45.1 -39.5 98.5 0 59.3 18.4 227.4 66 306.4 0 0 10.3 15.6 21.7 20 l 288.8 -131.8 z

set DR_HSESHOE=translate -47.8,19.2 path '%HSESHOE%'

:: We may need to use same Q-num.

%IM7DEV%magick ^
  %INFILE% ^
  %COLS% ^
  -draw "scale %%[fx:w/512],%%[fx:h/512] stroke-width %%[fx:512/w] %DR_HSESHOE%" ^
  -define quantum:format=floating-point -depth 32 ^
  %OUTFILE%

rhoStats.bat

@rem %1 A grayscale file containing rho (chroma) at each pixel.

@%IM7DEV%magick ^
  %1 ^
  -format "MIN=%%[fx:minima] MEAN=%%[fx:mean] MAX=%%[fx:maxima]\n" ^
  +write info: ^
  ( +clone ^
    -clamp ^
  ) ^
  -define compose:clamp=off ^
  -compose Difference -composite ^
  -fill White +opaque Black ^
  -scale "1x1^!" ^
  -format PROPOUT=%%[fx:mean] ^
  +write info: ^
  NULL:
  

All images on this page were created by the commands shown, using:

%IMG7%magick -version
Version: ImageMagick 7.1.1-20 Q16-HDRI x86 98bb1d4:20231008 https://imagemagick.org
Copyright: (C) 1999 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI OpenCL OpenMP(2.0) 
Delegates (built-in): bzlib cairo freetype gslib heic jng jp2 jpeg jxl lcms lqr lzma openexr pangocairo png ps raqm raw rsvg tiff webp xml zip zlib
Compiler: Visual Studio 2022 (193532217)

To improve internet download speeds, some images may have been automatically converted (by ImageMagick, of course) from PNG or TIFF or MIFF to JPG.

Source file for this web page is sqshxyysh.h1. To re-create this web page, execute "procH1 sqshxyysh".


This page, including the images, is my copyright. Anyone is permitted to use or adapt any of the code, scripts or images for any purpose, including commercial use.

Anyone is permitted to re-publish this page, but only for non-commercial use.

Anyone is permitted to link to this page, including for commercial use.


Page version v1.0 17-December-2018.

Page created 16-Jun-2024 11:19:42.

Copyright © 2024 Alan Gibson.