snibgo's ImageMagick pages

Linear camera raw

How linear is my camera raw data? How can I make it linear?

Commands of this page use utility programs such as cSelect.exe. I don't publish the source or binary of these.

What is linear, and why does it matter?

A digital camera measures light received from a subject. In particular, it measures the energy emitted within particular parts of the spectrum. For some purposes, we want to ensure that the numerical values at the pixels are proportional to the energy received (technically, proportional to the object luminance). This ensures that certain operations such as blurring will replicate physical processes, and help transformations to output colorspaces to be technically correct, and helps images from multiple sources (photos, videos, film scans, paintings, CGI) to have a common standard ("scene-referred") so they can be composited together easily and accurately. If we have multiple photos at different exposures of the same scene, we can multiply each by a number to make the same object the same tone and color on all the photos, so they can be merged for High Dynamic Range.

Beware that some writers use "linear" to mean pixel values are proportional to the object luminance plus or minus some constant, so a doubling of luminance does not result in a doubling of pixel values. This page uses "linear" to mean values are strictly proportional to the object luminance.

If a pixel is linear rgb(20%,30%,30%) then we can multiply all channels by 0.5 or 3 to make results rgb(10%,15%,15%) or rgb(60%,90%,90%) and the three pixels have the same chromaticity, the same x and y values in xyY space:

%IMG7%magick ^
  xc:rgb(20%%,30%%,30%%) ^
  -set colorspace RGB ^
  -colorspace xyY ^
  txt: 
# ImageMagick pixel enumeration: 1,1,65535,xyy
0,0: (18863.3,21557.7,18266.8)  #49AF5436475B  xyy(28.7835%,32.8949%,27.8733%)
%IMG7%magick ^
  xc:rgb(20%%,30%%,30%%) ^
  -set colorspace RGB ^
  -evaluate Multiply 0.5 ^
  -colorspace xyY ^
  txt: 
# ImageMagick pixel enumeration: 1,1,65535,xyy
0,0: (18863.3,21557.7,9133.38)  #49AF543623AD  xyy(28.7835%,32.8949%,13.9366%)
%IMG7%magick ^
  xc:rgb(20%%,30%%,30%%) ^
  -set colorspace RGB ^
  -evaluate Multiply 3 ^
  -colorspace ^
  xyY txt: 
# ImageMagick pixel enumeration: 1,1,65535,xyy
0,0: (18863.3,21557.7,54800.3)  #49AF5436D610  xyy(28.7835%,32.8949%,83.6198%)

But if we subtract 20% from all channels we get rgb(0,10%,10%) which has different chromaticity:

%IMG7%magick ^
  xc:rgb(20%%,30%%,30%%) ^
  -set colorspace RGB ^
  -evaluate Subtract 20%% ^
  -colorspace xyY ^
  txt: 
# ImageMagick pixel enumeration: 1,1,65535,xyy
0,0: (14722.6,21545.3,5159.55)  #398354291428  xyy(22.4652%,32.876%,7.87297%)

Note that some cameras write raw values with a black level offset, and this should be subtracted to make the values linear.

(Accurate processing also requires us to know the precise spectra that the camera records as pure red, green, blue and white -- the three primaries and the white point -- but that is beyond the scope of this web page.)

The real world has a wide range of possible luminances, from zero to infinity. A camera captures only a narrow range of luminances, and we want linearity within that range. There is no upper limit to luminance, it is unbounded, so linear vales are often recorded as floating point to allow for a higher dynamic range than a camera can record. Even the lower limit has problems; we need to go deep underground to experience true darkness. A camera might record zero even when there is light. Conversely, even when the shutter hasn't been opened, a camera might record non-zero, due to noise. Floating point also helps in the shadows, because the bottom few stops will have integer values 1, 2, 4, 8... which doesn't allow for much detail to be recorded between the stops. Merely converting from integer to floating-point doesn't help the shadows, but we can combine data from multiple exposures to supply shadow detail.

We can measure the linearity of the camera. If it isn't linear, we can derive a transformation that will make the values linear, and the different channels may need different transformations.

The process only applies to raw camera files. In-camera JPEG images are certainly not linear due to the sRGB tonal transformation. That is reversible, but the camera has probably also made local variations, boosting contrast and saturation, increasing sharpness and so on, to make the image more readable and attractive to humans. A JPEG or other processed image can often be made approximately linear, but attempts at accurate linearity are doomed.

The method

We apply the method to a Nikon D800 camera. The same method can be applied to any camera that makes raw images that dcraw can read.

We want to know if the raw sensor values are proportional to object luminance. We could photograph a step wedge on a light box and measure the value at each step.

Another method is to photograph a plain gray-ish card at different (but known) manual exposures. If we open the aperture by one stop or double the exposure time, the average value should double. The value should double for every extra stop. We repeat at a range of exposures, until a sensor records zero or 100%. At or beyond those limits, the average will be misleading, so we ignore those readings.

The JPEG verson of a typical photo looks like this:

set NEFDIR=\pictures\20180406\

%IM%convert ^
  %NEFDIR%AGA_3536.JPG ^
  -resize 400x400 ^
  lcr_samp.jpg
lcr_samp.jpg

The camera records the difference between the manual exposure and what it thinks is the correct exposure, shown as "ExpDiff" in tables below.

We assume the lens apertures and shutter speeds are accurate.

We use dcraw to de-Bayer the image. (Instead, we could separate the four channels without de-Bayering. As the subject is very flat (low contrast), this shouldn't make much difference.)

The script RawLevelsRng.bat writes the statistics into a CSV file, which csv2tab.bat converts to an HTML table for this web page.

call %PICTBAT%RawLevelsRng %NEFDIR%AGA_NN.NEF 3534 3553 lcr_shots.csv

call csv2tab lcr_shots
file ExpDiff minR meanR maxR minG meanG maxG minB meanB maxB
AGA_3534 +1.9 0.0971237 0.115321 0.134493 0.209354 0.236018 0.255497 0.151476 0.172334 0.191684
AGA_3535 +0.9 0.0440375 0.0555668 0.0661479 0.0994888 0.113619 0.125368 0.0705272 0.0829434 0.0942702
AGA_3536 0 0.0215152 0.0284767 0.0378424 0.0496834 0.0582615 0.0661631 0.0353552 0.0426908 0.0503548
AGA_3537 -1.1 0.00875868 0.013573 0.0216068 0.0225223 0.0276618 0.0324102 0.0154269 0.0202093 0.0246586
AGA_3538 -2.1 0.0037995 0.00669889 0.0166018 0.0107424 0.0137834 0.0170901 0.00677501 0.0100658 0.0138247
AGA_3539 -1.8 0.00515755 0.00846438 0.0183719 0.0139773 0.0173787 0.0213016 0.00906386 0.0126837 0.0162814
AGA_3540 -3.1 0.00146487 0.00345158 0.0130617 0.00482185 0.00700873 0.00939956 0.00282292 0.00509911 0.00747692
AGA_3541 -3.8 0.000427253 0.00226522 0.0123293 0.00299077 0.00472025 0.0065919 0.00164797 0.00343114 0.00534066
AGA_3542 +1.9 0.0964828 0.114794 0.132082 0.210452 0.23423 0.256779 0.150652 0.170609 0.189731
AGA_3543 +2.9 0.189487 0.231595 0.260441 0.418112 0.472265 0.511803 0.30135 0.345266 0.379232
AGA_3544 +3.9 0.396185 0.473593 0.51284 0.851362 0.942682 0.965194 0.627527 0.703038 0.755413
AGA_3545 +4.7 0.72221 0.826153 0.890242 0.960983 0.963096 0.965255 0.997253 0.999971 1
AGA_3546 +4.7 0.675517 0.761467 0.814069 0.960983 0.963096 0.965255 0.997253 0.999971 1
AGA_3547 +5.6 0.997314 0.999989 1 0.963058 0.965202 0.96733 0.996567 0.99925 1
AGA_3548 +6.3 0.997253 0.999989 1 0.969589 0.971792 0.973922 0.99382 0.996567 0.999313
AGA_3549 +7.4 0.997253 0.999989 1 0.969589 0.971792 0.973922 0.99382 0.996567 0.999313
AGA_3550 +6.4 0.997253 0.999989 1 0.969589 0.971792 0.973922 0.99382 0.996567 0.999313
AGA_3551 +7.4 0.997314 0.999988 1 0.9738 0.976005 0.978134 0.991714 0.994371 0.997055
AGA_3552 +9.1 0.997314 0.999988 1 0.967269 0.969413 0.971542 0.980209 0.982896 0.98558
AGA_3553 +8.1 0.997314 0.999988 1 0.967269 0.969413 0.971542 0.980209 0.982896 0.98558

Some ExpDiff values are duplicated because my photography was a bit careless.

Something strange is happening in the blue at high exposures. The sensor is saturated at +4.7 and +5.6, but then records lower values at +6.3 and above. It seems the electronic sensor exhibits solarisation, like film does. The same is true, to a lesser degree, in the other channels.

We eliminate rows that have min at zero or max at one; remove columns other than the ExpDiff and mean values; sort on ExpDiff; and group rows that have the same ExpDiff values, calculating the mean values of the means.

(I don't publish source or binaries of utility programs cSelect, chStrs, cProject or cSort.)

set TMPCSV=\temp\lcr.csv
cSelect  /ilcr_shots.csv /o%TMPCSV% /h /kminR /w0 /x
cSelect  /i%TMPCSV% /h /kminG /w0 /x
cSelect  /i%TMPCSV% /h /kminB /w0 /x
cSelect  /i%TMPCSV% /h /kmaxR /w1 /x
cSelect  /i%TMPCSV% /h /kmaxG /w1 /x
cSelect  /i%TMPCSV% /h /kmaxB /w1 /x
chStrs   /i%TMPCSV% /f"+" /t\0
cProject /i%TMPCSV% /h /kExpDiff,meanR,meanG,meanB
rem cSort    /i%TMPCSV% /olcr_shots2.csv /h /kExpDiff
cGroup   /i%TMPCSV% /olcr_shots2.csv /h /kExpDiff /u /emeanR,meanG,meanB
cPrefix  /ilcr_shots2.csv /h /tExpDiff,meanR,meanG,meanB
call csv2tab lcr_shots2
ExpDiff meanR meanG meanB
-3.8 0.0022652200000000001 0.0047202499999999996 0.0034311400000000001
-3.1 0.0034515800000000001 0.0070087300000000003 0.0050991099999999996
-2.1 0.0066988899999999999 0.013783399999999999 0.0100658
-1.8 0.0084643800000000005 0.0173787 0.012683699999999999
-1.1 0.013573 0.0276618 0.020209299999999999
0 0.028476700000000001 0.058261500000000001 0.042690800000000001
0.9 0.0555668 0.113619 0.0829434
1.9 0.11505750000000001 0.235124 0.1714715
2.9 0.231595 0.47226499999999999 0.34526600000000002
3.9 0.47359299999999999 0.94268200000000002 0.70303800000000005

Some observations:

Each unit of ExpDiff is a photographic stop, a doubling of exposure. So we plot the log (base 2) of the mean values against ExpDiff:

gnuplot -c %PICTBAT%plotExp.gp ^
  lcr_shots2.csv lcr_shots2.svg ^
  . . "2 3 4" ExpDiff LogMeanVal
lcr_shots2.png

We can see the log values are very close to a straight line with unity slope, and the channels are very nearly parallel. This is true within the experimental error of the camera recording ExpDiff to a resolution of 0.1 stops.

As each channel is a straight line with unity slope, each channel records values that are proportional to intensity.

As the channels are parallel, there is a constant difference in LogMeanVal, which means there is a constant factor in MeanVal, which means the colour balance doesn't change though the measured range.

I conclude this camera is sufficiently linear that no transformation is needed.

Correcting linearity

If the linearity was worse, we could correct it. One obvious method is to construct a clut where the input values are from the previous table, and the outputs are the ideal values. We might assume the ExpDiff values are correct, and the means at the largest ExdDiff are correct. The ideal values for each ExpDiff are then easily calculated. We would also add entries in the clut for input values zero and one, with outputs zero and one.

Scene-referred images

Real life doesn't have a maximum brightness, or maximum possible intensity of red, green, blue or any other colour. Computers are not good at representing all values between zero and infinity, but floating-point is reasonably close.

We use dcraw to process the image, using the camera white-balance, no auto-brighten, no gamma correction, converting to XYZ colorspace, writing to a TIFF which will include dcraw's XYZ profile.

set SRC_NEF=%PICTLIB%20130713\AGA_1372.NEF

%DCRAW% -v -W -w -4 -o 5 -T -O lcr_xyz.tiff %SRC_NEF%

We use IM's "-profile" to convert to any linear colorspace we want. In this simple example, we convert to linear sRGB. We also divide by the maximum value so the largest value is QuantumRange. This is only for arithmetical convenience; in scene-referred encoding, QuantumRange has no significance. For some processing, such as panorama stitching and HDR stacking, we wouldn't divide by anything at this stage.

%IMG7%magick ^
  lcr_xyz.tiff ^
  -crop 267x233+3033+4189 +repage ^
  -set colorspace XYZ ^
  -profile %ICCPROF%\sRGB-elle-V4-g10.icc ^
  -evaluate Divide %%[fx:maxima] ^
  -set gamma 1.0 ^
  -define quantum:format=floating-point -depth 32 ^
  lcr_xyz_toes.tiff

The image in lcr_xyz_toes.tiff is linear and scene-referred, with sRGB primaries. To show the image on the web (or output device), we convert to sRGB (or whatever the device needs):

%IMG7%magick ^
  lcr_xyz_toes.tiff ^
  -profile %ICCPROF%\sRGB-elle-V4-srgbtrc.icc ^
  lcr_xyz_toes_s.png
lcr_xyz_toes_s.pngjpg

This image is darker than the usual sRGB toes.png image because toes.png has been slightly prettified.

For linear sRGB, an alternative method is to use dcraw options -W -4, and then assign the linear sRGB profile.

For editing, we might prefer the linear Rec2020 colorspace:

%IMG7%magick ^
  -verbose ^
  lcr_xyz.tiff ^
  -crop 267x233+3033+4189 +repage ^
  -set colorspace XYZ ^
  -profile %ICCPROF%\Rec2020-elle-V4-g10.icc ^
  -evaluate Divide %%[fx:maxima] ^
  -define quantum:format=floating-point -depth 32 ^
  lcr_rec22_toes.tiff

The image in lcr_rec22_toes.tiff is linear and scene-referred, with Rec2020 primaries. After editing in linear space, we can convert to sRGB for display:

%IMG7%magick ^
  lcr_rec22_toes.tiff ^
  -profile %ICCPROF%\sRGB-elle-V4-srgbtrc.icc ^
  lcr_rec22_toes_s.png
lcr_rec22_toes_s.pngjpg

Saturated sensels

For some purposes, such as High dynamic range stacks, we want to know which pixels have incorrect colours because sensor elements were saturated.

We might examine the white-balanced demosaiced image, and declare that any pixel that has a value in any channel beyond a threshold has saturated.

For a more precise result, we examine raw pixel values before demosaicing. A Nikon D800 camera records sensel values in 14 bits. dcraw records these 14-bit numbers in 16-bit integers. The maximum 14-bit number is 214-1 = 16383. A sensel with this value might happen to be correct, but is is more likely to be saturated, so we assume it has saturated. Moreover, we assume that a saturation in any of the eight surrounding sensels will pollute this demosaiced sensel, so we declare this pixel to be saturated when any of the 3x3 window of sensels is 16383.

set SRC_NEF=%PICTLIB%20130713\AGA_1372.NEF

%DCRAW% -v -D -W -w -g 1 1 -6 -T -O lcr_sens.tiff %SRC_NEF%

for /F "usebackq" %%L in (`%IMG7%magick ^
  lcr_sens.tiff ^
  -format "MIN=%%[minima]\nMAX=%%[maxima]\n" +write info: ^
  -statistic Maximum 3x3 ^
  -white-threshold 16382 ^
  -fill Black +opaque White ^
  -format "SAT_PC=%%[fx:mean*100]\n" +write info: ^
  lcr_sens2.tiff`) do set %%L

echo MIN=%MIN% MAX=%MAX% SAT_PC=%SAT_PC% 
MIN=5 MAX=16383 SAT_PC=0.000346828 

Note that "-white-threshold" sets to white all pixels that are above the given value.

The result lcr_sens2.tiff is a mask that is white where sensor saturation occurred; otherwise black.

Under our definition, about 0.0003% of pixels are saturated. In this 35MP image, this is about 100 pixels. Ordinary photos will often include specular highlights, so to entirely avoid saturated sensels we would need heavy "underexposure" in the camera, resulting in noise.

Scripts

For convenience, .bat scripts are also available in a single zip file. See Zipped BAT files.

RawLevelsRng.bat

rem Given %1 is a template name of raw files that includes NN,
rem %2 is a start number,
rem %3 is an end number,
rem %4 is an output for CSV,
rem writes data for each raw file.


set INTMPLT=%1
set START=%2
set END=%3
set CSVFILE=%4

if "%INTMPLT%"=="." set INTMPLT=
if "%INTMPLT%"=="" set INTMPLT=abc_NN.NEF

if "%START%"=="." set START=
if "%START%"=="" set START=0

if "%END%"=="." set END=
if "%END%"=="" set END=%START%

if "%CSVFILE%"=="." set CSVFILE=
if "%CSVFILE%"=="" set CSVFILE=rlr.csv

set TMPTIFF=\temp\rlr.tiff

echo file, ExpDiff, minR, meanR, maxR,  minG, meanG, maxG,  minB, meanB, maxB >%CSVFILE%

for /l %%N in (%START%, 1, %END%) do (
  echo %%N

  set LZ=0000%%N
  set LZ=!LZ:~-4!
  echo LZ=!LZ!

  set NEFFILE=%INTMPLT:NN=!LZ!%

  echo !NEFFILE!

  call %PICTBAT%RawLevelsOne !NEFFILE! %TMPTIFF% %CSVFILE%

  if ERRORLEVEL 1 exit /B 1
)

RawLevelsOne.bat

set NEFFILE=%1
set TMPTIFF=%2
set CSVFILE=%3

set NEF_SHT=%~n1

%DCRAW% -v -4 -r 1 1 1 1 -g 1 1 -o 0 -T -R 1 -O %TMPTIFF% %NEFFILE%

if ERRORLEVEL 1 exit /B 1

set EDIF=
for /F "usebackq tokens=3 delims=: " %%A in (`exiftool ^
  -ExposureDifference ^
  %NEFFILE%`) do set EDIF=%%A

if "%EDIF%"=="" exit /B 1

set sVALS=^
%NEF_SHT%, ^
%EDIF%,  ^
%%[fx:minima.r], %%[fx:mean.r], %%[fx:maxima.r],  ^
%%[fx:minima.g], %%[fx:mean.g], %%[fx:maxima.g],  ^
%%[fx:minima.b], %%[fx:mean.b], %%[fx:maxima.b]\n

%IMDEV%convert ^
  %TMPTIFF% ^
  -gravity Center ^
  -crop 33x33%%+0+0 +repage ^
  -format "%sVALS%" ^
  info: >>%CSVFILE%

csv2tab.bat

rem From %1.csv, creates partial %1.htm with a table.
rem %2 is parameters to table, eg "border".

set c2tTAB_PARAMS=%2
if "%c2tTAB_PARAMS%"=="." set c2tTAB_PARAMS=

chSep   /p0 /i%1.csv /o%1.htm /f"," /t\(/td\)\(td\) /T\(/th\)\(th\) /n
cPrefix /p0 /i%1.htm /l\(tr\)\(td\) /r\(/td\)\(/tr\) /t"\(table %c2tTAB_PARAMS%\)" /b\(/table\) /X
chStrs  /p0 /i%1.htm /m1 /f\(td\) /t\(th\)
chStrs  /p0 /i%1.htm /m1 /f\(/td\) /t\(/th\)

All images on this page were created by the commands shown, using:

%IM%identify -version
Version: ImageMagick 6.9.9-50 Q16 x64 2018-06-02 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2015 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Visual C++: 180040629
Features: Cipher DPC Modules OpenMP 
Delegates (built-in): bzlib cairo flif freetype gslib heic jng jp2 jpeg lcms lqr lzma openexr pangocairo png ps raw rsvg tiff webp xml zlib

To improve internet download speeds, some images may have been automatically converted (by ImageMagick, of course) from PNG or TIFF or MIFF to JPG.

Source file for this web page is lincamraw.h1. To re-create this web page, execute "procH1 lincamraw".


This page, including the images, is my copyright. Anyone is permitted to use or adapt any of the code, scripts or images for any purpose, including commercial use.

Anyone is permitted to re-publish this page, but only for non-commercial use.

Anyone is permitted to link to this page, including for commercial use.


Page version v1.0 29-March-2018.

Page created 13-Feb-2020 21:39:42.

Copyright © 2020 Alan Gibson.