snibgo's ImageMagick pages

Linear camera raw

How linear is my camera raw data? How can I make it linear?

• What is linear, and why does it matter?
• The method
• Correcting linearity
• Scene-referred images
• Saturated sensels
• Scripts

Commands of this page use utility programs such as cSelect.exe. I don't publish the source or binary of these.

What is linear, and why does it matter?

A digital camera measures light received from a subject. In particular, it measures the energy emitted within particular parts of the spectrum. For some purposes, we want to ensure that the numerical values at the pixels are proportional to the energy received (technically, proportional to the object luminance). This ensures that certain operations such as blurring will replicate physical processes, and help transformations to output colorspaces to be technically correct, and helps images from multiple sources (photos, videos, film scans, paintings, CGI) to have a common standard ("scene-referred") so they can be composited together easily and accurately. If we have multiple photos at different exposures of the same scene, we can multiply each by a number to make the same object the same tone and color on all the photos, so they can be merged for High Dynamic Range.

%IMG7%magick ^
  xc:rgb(20%%,30%%,30%%) ^
  -set colorspace RGB ^
  -colorspace xyY ^
  txt: 

# ImageMagick pixel enumeration: 1,1,0,65535,xyy
0,0: (18863,21558,18267)  #49AF5436475B  xyy(73.398,83.882,71.0769)

%IMG7%magick ^
  xc:rgb(20%%,30%%,30%%) ^
  -set colorspace RGB ^
  -evaluate Multiply 0.5 ^
  -colorspace xyY ^
  txt: 

# ImageMagick pixel enumeration: 1,1,0,65535,xyy
0,0: (18863,21558,9133)  #49AF543623AD  xyy(73.398,83.882,35.5384)

%IMG7%magick ^
  xc:rgb(20%%,30%%,30%%) ^
  -set colorspace RGB ^
  -evaluate Multiply 3 ^
  -colorspace ^
  xyY txt: 

# ImageMagick pixel enumeration: 1,1,0,65535,xyy
0,0: (18863,21558,54800)  #49AF5436D610  xyy(73.398,83.882,213.231)

%IMG7%magick ^
  xc:rgb(20%%,30%%,30%%) ^
  -set colorspace RGB ^
  -evaluate Subtract 20%% ^
  -colorspace xyY ^
  txt: 

# ImageMagick pixel enumeration: 1,1,0,65535,xyy
0,0: (14723,21545,5160)  #398354291428  xyy(57.2863,83.8339,20.0761)

(Accurate processing also requires us to know the precise spectra that the camera records as pure red, green, blue and white -- the three primaries and the white point -- but that is beyond the scope of this web page.)

The real world has a wide range of possible luminances, from zero to infinity. A camera captures only a narrow range of luminances, and we want linearity within that range. There is no upper limit to luminance, it is unbounded, so linear vales are often recorded as floating point to allow for a higher dynamic range than a camera can record. Even the lower limit has problems; we need to go deep underground to experience true darkness. A camera might record zero even when there is light. Conversely, even when the shutter hasn't been opened, a camera might record non-zero, due to noise. Floating point also helps in the shadows, because the bottom few stops will have integer values 1, 2, 4, 8... which doesn't allow for much detail to be recorded between the stops. Merely converting from integer to floating-point doesn't help the shadows, but we can combine data from multiple exposures to supply shadow detail.

We can measure the linearity of the camera. If it isn't linear, we can derive a transformation that will make the values linear, and the different channels may need different transformations.

The process only applies to raw camera files. In-camera JPEG images are certainly not linear due to the sRGB tonal transformation. That is reversible, but the camera has probably also made local variations, boosting contrast and saturation, increasing sharpness and so on, to make the image more readable and attractive to humans. A JPEG or other processed image can often be made approximately linear, but attempts at accurate linearity are doomed.

The method

We apply the method to a Nikon D800 camera. The same method can be applied to any camera that makes raw images that dcraw can read.

We want to know if the raw sensor values are proportional to object luminance. We could photograph a step wedge on a light box and measure the value at each step.

Another method is to photograph a plain gray-ish card at different (but known) manual exposures. If we open the aperture by one stop or double the exposure time, the average value should double. The value should double for every extra stop. We repeat at a range of exposures, until a sensor records zero or 100%. At or beyond those limits, the average will be misleading, so we ignore those readings.

The JPEG version of a typical photo looks like this:

set NEFDIR=\pictures\20180406\ %IMG7%magick ^ %NEFDIR%AGA_3536.JPG ^ -resize 400x400 ^ lcr_samp.jpg

The camera records the difference between the manual exposure and what it thinks is the correct exposure, shown as "ExpDiff" in tables below.

We assume the lens apertures and shutter speeds are accurate.

We use dcraw to de-Bayer the image. (Instead, we could separate the four channels without de-Bayering. As the subject is very flat (low contrast), this shouldn't make much difference.)

The script RawLevelsRng.bat writes the statistics into a CSV file, which csv2tab.bat converts to an HTML table for this web page.

call %PICTBAT%RawLevelsRng %NEFDIR%AGA_NN.NEF 3534 3553 lcr_shots.csv

call csv2tab lcr_shots

Some ExpDiff values are duplicated because my photography was a bit careless.

Something strange is happening in the blue at high exposures. The sensor is saturated at +4.7 and +5.6, but then records lower values at +6.3 and above. It seems the electronic sensor exhibits solarisation, like film does. The same is true, to a lesser degree, in the other channels.

We eliminate rows that have min at zero or max at one; remove columns other than the ExpDiff and mean values; sort on ExpDiff; and group rows that have the same ExpDiff values, calculating the mean values of the means.

(I don't publish source or binaries of utility programs cSelect, chStrs, cProject or cSort.)

set TMPCSV=\temp\lcr.csv
cSelect  /ilcr_shots.csv /o%TMPCSV% /h /kminR /w0 /x
cSelect  /i%TMPCSV% /h /kminG /w0 /x
cSelect  /i%TMPCSV% /h /kminB /w0 /x
cSelect  /i%TMPCSV% /h /kmaxR /w1 /x
cSelect  /i%TMPCSV% /h /kmaxG /w1 /x
cSelect  /i%TMPCSV% /h /kmaxB /w1 /x
chStrs   /i%TMPCSV% /f"+" /t\0
cProject /i%TMPCSV% /h /kExpDiff,meanR,meanG,meanB
rem cSort    /i%TMPCSV% /olcr_shots2.csv /h /kExpDiff
cGroup   /i%TMPCSV% /olcr_shots2.csv /h /kExpDiff /u /emeanR,meanG,meanB
cPrefix  /ilcr_shots2.csv /h /tExpDiff,meanR,meanG,meanB
call csv2tab lcr_shots2

Some observations:

• The subject is roughly gray, but green values are about double the red and blue values because we haven't applied white-balancing.
• The subject is fairly flat, but the camera meter doesn't automatically "expose to the right".
• We can see that values roughly double with each exposure increase by one stop.
• In each channel, if we take log (base 2) of the highest mean and subtract log (base 2) of the lowest mean, we get the difference between the ExpDiff values.

Each unit of ExpDiff is a photographic stop, a doubling of exposure. So we plot the log (base 2) of the mean values against ExpDiff:

gnuplot-base -c %PICTBAT%plotExp.gp ^ lcr_shots2.csv lcr_shots2.svg ^ . . "2 3 4" ExpDiff LogMeanVal

We can see the log values are very close to a straight line with unity slope, and the channels are very nearly parallel. This is true within the experimental error of the camera recording ExpDiff to a resolution of 0.1 stops.

As each channel is a straight line with unity slope, each channel records values that are proportional to intensity.

As the channels are parallel, there is a constant difference in LogMeanVal, which means there is a constant factor in MeanVal, which means the colour balance doesn't change though the measured range.

I conclude this camera is sufficiently linear that no transformation is needed.

Correcting linearity

If the linearity was worse, we could correct it. One obvious method is to construct a clut where the input values are from the previous table, and the outputs are the ideal values. We might assume the ExpDiff values are correct, and the means at the largest ExdDiff are correct. The ideal values for each ExpDiff are then easily calculated. We would also add entries in the clut for input values zero and one, with outputs zero and one.

Scene-referred images

Real life doesn't have a maximum brightness, or maximum possible intensity of red, green, blue or any other colour. Computers are not good at representing all values between zero and infinity, but floating-point is reasonably close.

We use dcraw to process the image, using the camera white-balance, no auto-brighten, no gamma correction, converting to XYZ colorspace, writing to a TIFF which will include dcraw's XYZ profile.

set SRC_NEF=%PICTLIB%20130713\AGA_1372.NEF

%DCRAW% -v -W -w -4 -o 5 -T -O lcr_xyz.tiff %SRC_NEF%

We use IM's "-profile" to convert to any linear colorspace we want. In this simple example, we convert to linear sRGB. We also divide by the maximum value so the largest value is QuantumRange. This is only for arithmetical convenience; in scene-referred encoding, QuantumRange has no significance. For some processing, such as panorama stitching and HDR stacking, we wouldn't divide by anything at this stage.

%IMG7%magick ^
  lcr_xyz.tiff ^
  -crop 267x233+3033+4189 +repage ^
  -set colorspace XYZ ^
  -profile %ICCPROF%\sRGB-elle-V4-g10.icc ^
  -evaluate Divide %%[fx:maxima] ^
  -set gamma 1.0 ^
  -define quantum:format=floating-point -depth 32 ^
  lcr_xyz_toes.tiff

The image in lcr_xyz_toes.tiff is linear and scene-referred, with sRGB primaries. It has an embedded ICC profile. To show the image on the web (or output device), we convert to sRGB (or whatever the device needs):

%IMG7%magick ^ lcr_xyz_toes.tiff ^ -profile %ICCPROF%\sRGB-elle-V4-srgbtrc.icc ^ lcr_xyz_toes_s.png

This image is darker than the usual sRGB toes.png image because toes.png has been slightly prettified.

For linear sRGB, an alternative method is to use dcraw options -W -4, and then assign the linear sRGB profile.

For editing, we might prefer the linear Rec2020 colorspace:

%IMG7%magick ^
  -verbose ^
  lcr_xyz.tiff ^
  -crop 267x233+3033+4189 +repage ^
  -set colorspace XYZ ^
  -profile %ICCPROF%\Rec2020-elle-V4-g10.icc ^
  -evaluate Divide %%[fx:maxima] ^
  -define quantum:format=floating-point -depth 32 ^
  lcr_rec22_toes.tiff

The image in lcr_rec22_toes.tiff is linear and scene-referred, with Rec2020 primaries. After editing in linear space, we can convert to sRGB for display:

%IMG7%magick ^ lcr_rec22_toes.tiff ^ -profile %ICCPROF%\sRGB-elle-V4-srgbtrc.icc ^ lcr_rec22_toes_s.png

Saturated sensels

For some purposes, such as High dynamic range stacks, we want to know which pixels have incorrect colours because sensor elements were saturated.

We might examine the white-balanced demosaiced image, and declare that any pixel that has a value in any channel beyond a threshold has saturated.

For a more precise result, we examine raw pixel values before demosaicing. A Nikon D800 camera records sensel values in 14 bits. dcraw records these 14-bit numbers in 16-bit integers. The maximum 14-bit number is 2¹⁴-1 = 16383. A sensel with this value might happen to be correct, but is is more likely to be saturated, so we assume it has saturated. Moreover, we assume that a saturation in any of the eight surrounding sensels will pollute this demosaiced sensel, so we declare this pixel to be saturated when any of the 3x3 window of sensels is 16383.

set SRC_NEF=%PICTLIB%20130713\AGA_1372.NEF

%DCRAW% -v -D -W -w -g 1 1 -6 -T -O lcr_sens.tiff %SRC_NEF%

for /F "usebackq" %%L in (`%IMG7%magick ^
  lcr_sens.tiff ^
  -format "MIN=%%[minima]\nMAX=%%[maxima]\n" +write info: ^
  -statistic Maximum 3x3 ^
  -white-threshold 16382 ^
  -fill Black +opaque White ^
  -format "SAT_PC=%%[fx:mean*100]\n" +write info: ^
  lcr_sens2.tiff`) do set %%L

echo MIN=%MIN% MAX=%MAX% SAT_PC=%SAT_PC% 

MIN=5 MAX=16383 SAT_PC=0.000346828 

Note that "-white-threshold" sets to white all pixels that are above the given value.

The result lcr_sens2.tiff is a mask that is white where sensor saturation occurred; otherwise black.

Under our definition, about 0.0003% of pixels are saturated. In this 35MP image, this is about 100 pixels. Ordinary photos will often include specular highlights, so to entirely avoid saturated sensels we would need heavy "underexposure" in the camera, resulting in noise.

Scripts

For convenience, .bat scripts are also available in a single zip file. See Zipped BAT files.

RawLevelsRng.bat

rem Given %1 is a template name of raw files that includes NN,
rem %2 is a start number,
rem %3 is an end number,
rem %4 is an output for CSV,
rem writes data for each raw file.


set INTMPLT=%1
set START=%2
set END=%3
set CSVFILE=%4

if "%INTMPLT%"=="." set INTMPLT=
if "%INTMPLT%"=="" set INTMPLT=abc_NN.NEF

if "%START%"=="." set START=
if "%START%"=="" set START=0

if "%END%"=="." set END=
if "%END%"=="" set END=%START%

if "%CSVFILE%"=="." set CSVFILE=
if "%CSVFILE%"=="" set CSVFILE=rlr.csv

set TMPTIFF=\temp\rlr.tiff

echo file, ExpDiff, minR, meanR, maxR,  minG, meanG, maxG,  minB, meanB, maxB >%CSVFILE%

for /l %%N in (%START%, 1, %END%) do (
  echo %%N

  set LZ=0000%%N
  set LZ=!LZ:~-4!
  echo LZ=!LZ!

  set NEFFILE=%INTMPLT:NN=!LZ!%

  echo !NEFFILE!

  call %PICTBAT%RawLevelsOne !NEFFILE! %TMPTIFF% %CSVFILE%

  if ERRORLEVEL 1 exit /B 1
)

RawLevelsOne.bat

@rem
@rem Updated:
@rem   8-August-2022 for IM v7.
@rem

set NEFFILE=%1
set TMPTIFF=%2
set CSVFILE=%3

set NEF_SHT=%~n1

%DCRAW% -v -4 -r 1 1 1 1 -g 1 1 -o 0 -T -R 1 -O %TMPTIFF% %NEFFILE%

if ERRORLEVEL 1 exit /B 1

set EDIF=
for /F "usebackq tokens=3 delims=: " %%A in (`exiftool ^
  -ExposureDifference ^
  %NEFFILE%`) do set EDIF=%%A

if "%EDIF%"=="" exit /B 1

set sVALS=^
%NEF_SHT%, ^
%EDIF%,  ^
%%[fx:minima.r], %%[fx:mean.r], %%[fx:maxima.r],  ^
%%[fx:minima.g], %%[fx:mean.g], %%[fx:maxima.g],  ^
%%[fx:minima.b], %%[fx:mean.b], %%[fx:maxima.b]\n

%IMG7%magick ^
  %TMPTIFF% ^
  -gravity Center ^
  -crop 33x33%%+0+0 +repage ^
  -format "%sVALS%" ^
  info: >>%CSVFILE%

csv2tab.bat

rem From %1.csv, creates partial %1.htm with a table.
rem %2 is parameters to table, eg "border".

set c2tTAB_PARAMS=%2
if "%c2tTAB_PARAMS%"=="." set c2tTAB_PARAMS=

chSep   /p0 /i%1.csv /o%1.htm /f"," /t\(/td\)\(td\) /T\(/th\)\(th\) /n
cPrefix /p0 /i%1.htm /l\(tr\)\(td\) /r\(/td\)\(/tr\) /t"\(table %c2tTAB_PARAMS%\)" /b\(/table\) /X
chStrs  /p0 /i%1.htm /m1 /f\(td\) /t\(th\)
chStrs  /p0 /i%1.htm /m1 /f\(/td\) /t\(/th\)

%IMG7%magick -version

Version: ImageMagick 7.1.0-42 Q16-HDRI x64 396d87c:20220709 https://imagemagick.org
Copyright: (C) 1999 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC HDRI OpenCL 
Delegates (built-in): bzlib cairo freetype gslib heic jng jp2 jpeg jxl lcms lqr lzma openexr pangocairo png ps raqm raw rsvg tiff webp xml zip zlib
Compiler: Visual Studio 2022 (193231332)

Page version v1.0 29-March-2018.

Page created 13-Aug-2022 23:05:25.

Copyright © 2022 Alan Gibson.