snibgo's ImageMagick pages

dcraw and WB

... (white balance) and clipping and contrast.

dcraw, Dave Coffin's raw image converter, is a splendid piece of work. It must count as (probably) the greatest and most influential little program of the last 20 years. It isn't a library, and it's difficult to pull apart to tweak it in the directions I want, or even to understand. But what it does, it does well.

This page shows how we can use dcraw to get accurately white-balanced images, with good contrast and brightness, without clipping, automatically.

Scripts on this page assume that the version of ImageMagick in %IMDEV% has been built with various process modules. See Process modules.

Install dcraw

Some versions of IM come with a pre-built binary of dcraw, or you can build it yourself.

  1. Get the source. There is only one source file.
    wget http://cybercom.net/~dcoffin/dcraw/dcraw.c
    Or you can download my slightly hacked version, dcrawimg.c. Or download the official source, and apply the differences file dcrawag.diff between it and my hacked version.
  2. Patch and hack it as you want. As written by Dave Coffin, it doesn't have an option to name the output file. The ImageMagick suite includes a patch for this, adding a "-O <file>" option.
  3. Compile and link, and install. The Jasper library is only used for a type of camera I don't have, and my computer doesn't have the library. These are the Cygwin bash commands I use:
    @%IMSRC%\buildDcraw.sh
  4. Rename the executable to whatever you want, and copy it to wherever you want. My patch changes the behaviour of the program, so I don't (currently) call it dcraw.exe. I run it indirectly, via an environment variable.
  5. echo %DCRAW% 
    @dcr_dcraw.lis
  6. Check that running the program with no parameters gives something like this:
    %DCRAW% 
    @dcr_dcraw_hlp.lis,h
    Note the -O and -B options.

Auto brighten

dcraw has a "automatically brighten the image" operation. This stretches the pixel values by multiplying channel values. The idea is that if the maximum value is 75%, multiplying by 1.3333 will use more of the range between 0 and 100%. (dcraw won't also stretch the bottom, so if the minimum value was 10% it might become 13.333%. Why won't it stretch both ends? I don't know.) This happens by default; if we don't want it, we can disable it with "-W", and we can then use IM to "-auto-level" or whatever we want.

But this auto-brighten feature has a problem: it doesn't merely stretch to use values up to 100%, but over-stretches, clipping the top 1% of pixels. This is a lot of pixels. An area one-tenth of the image width and one-tenth of the height amounts to 1% of the image area. The whole point of using raw images (for me) is to get good pixels from the camera, giving me more flexibility than the in-camera JPEG provides, and hence better quality. Wrecking 1% of the pixels before I even begin the creative stuff is a bad start.

So I've also created a patch for dcraw. This adds a variable, BrightThr initialised to 0.0001, and an option "-B <num>" to set the variable. The variable is used in a line that used to read...

perc = width * height * 0.01;

... but I have changed to ...

perc = width * height * BrightThr;

So, by default "auto-brighten" will now clip 0.0001 of the pixels, which is 0.01% of the pixels. If that is too much, -B 0 will auto-brighten without clipping any pixels. As before, "-W" will not auto-brighten at all.

White balance

dcraw applies a white balance by simple channel multiplication in linear colorspace, before de-Bayering and conversion to sRGB or whatever. It can get the multipliers from metadata in the raw file ("camera" white balance), or from averaging a rectangle of pixels that should be neutral gray, or from the "-r" command line option. Multiplying channels can result in clipping.

dcraw's -b option, to set brightness, also multiplies in linear colorspace. See Brightness below.

We will show three methods for white balancing:

  1. Use camera WB and default dcraw processing;
  2. Usu camera WB and dcraw -R n;
  3. Calculate -r n n n n multipliers from gray patches.

[Embed linear images with a profile from Elle.]

First, some basic statistics about the raw Bayer data. "-D" give us "Document", which isn't de-Bayered or scaled. We don't auto-brighten or white-balance or adjust gamma.

:skip

set SRCNEF=%PICTLIB%20171029\AGA_3437.nef

set WEBDIMS=600x600
set WEBSIZE=-resize %WEBDIMS%

rem goto skip2

%DCRAW% -v -6 -W -D -r 1 1 1 1 -g 1 1 -o 0 -T -O dcr_basic.tiff %SRCNEF%

What are the minimum, mean and maximum Bayer values?

%IM%convert dcr_basic.tiff ^
  -format "%%[fx:minima*QuantumRange] %%[fx:mean*QuantumRange] %%[fx:maxima*QuantumRange]\n" ^
  info: 
0 1148.18 16383

We are using IM Q16 but the camera records only 14 bits per pixel, so the maximum value possible is 2^14-1 = 16383.

How many Bayer values are at the extremes?

call %PICTBAT%propIntClip dcr_basic.tiff 
 f:\prose\PICTURES>rem From image dcr_basic.tiff, 

f:\prose\PICTURES>rem finds proportion that have 0 or 100 in any channel. 

f:\prose\PICTURES>c:\im\ImageMagick-6.9.5-3-Q16\convert   dcr_basic.tiff   -separate   -fill White -opaque Black   -fill Black +opaque White   -background Black   -compose Lighten -layers Merge   -format "clipped=%[fx:mean]\n"   info: 
clipped=1.54145e-006

f:\prose\PICTURES>c:\im\ImageMagick-6.9.5-3-Q16\convert   dcr_basic.tiff   -separate   -fill White +opaque Black +write x.tiff   -negate +write x2.tiff   -background Black   -compose Lighten -layers Merge +write x3.tiff   -format "shadowClipped=%[fx:mean]\n"   info: 
shadowClipped=1.54145e-006

A very small number of sensor pixels recorded no signal at all. A larger number recorded the maximum signal, so the sensors were saturated (the amount of light was at or beyond the capacity of the sensors). (The camera has 35 M pixels, so the proportion of pixels that are clipped, 0.05%, isn't horribly bad.) We want to avoid increasing the numbers at either end.

WB and default dcraw

We tell dcraw to make a 16-bit TIFF sRGB image, using the camera WB, with an auto brighten that doesn't clip.

First, we use -H 0. This normalises the WB channel multipliers so the smallest multiplier is 1.0, so values in other channels can increase, so this can cause clipping in those other channels. However, this clipping means that pixels recorded at or near 100% in all three channels will become exactly or nearly white after the balancing, which is normally the correct result.

%DCRAW% -v -6 -w -o 1 -H 0 -B 0 -T -O dcr_wbdef0.tiff %SRCNEF%

What are the minimum, mean and maximum channel values?

%IM%convert dcr_wbdef0.tiff ^
  -format "%%[fx:minima] %%[fx:mean] %%[fx:maxima]\n" ^
  info: 
0 0.186094 1

How many pixel channel values are at the extremes?

call %PICTBAT%propIntClipRGB dcr_wbdef0.tiff 
shadowClipped.0=11489
shadowClipped.1=24
shadowClipped.2=1448
highlightClipped.0=882807
highlightClipped.1=13256
highlightClipped.2=1.37619e+006

Many pixels (about 3%) have clipped in the blue channel.

The image looks like this:

%IM%convert ^
  dcr_wbdef0.tiff ^
  %WEBSIZE% ^
  dcr_wbdef0.jpg
dcr_wbdef0.jpg

Using a GUI editor, we can see that most of the sky has burnt out in one or more channels.

We repeat, using -H 1. Modes from this upwards normalise the WB channel multipliers so the largest is 1.0. It also does other processing that I don't understand.

%DCRAW% -v -6 -w -o 1 -H 1 -B 0 -T -O dcr_wbdef1.tiff %SRCNEF%

What are the minimum, mean and maximum channel values?

%IM%convert dcr_wbdef1.tiff ^
  -format "%%[fx:minima] %%[fx:mean] %%[fx:maxima]\n" ^
  info: 
0 0.119695 1

How many pixel channel values are at the extremes?

call %PICTBAT%propIntClipRGB dcr_wbdef1.tiff 
shadowClipped.0=11580
shadowClipped.1=23
shadowClipped.2=1538
highlightClipped.0=195181
highlightClipped.1=1
highlightClipped.2=1

That's better. Far fewer pixels have clipped.

The image looks like this:

%IM%convert ^
  dcr_wbdef1.tiff ^
  %WEBSIZE% ^
  dcr_wbdef1.jpg
dcr_wbdef1.jpg

Eek! The sky has gone stupid. Other parts of the image are darker than the previous version, and this is expected as the multiplier normalisation has reduced instead of increasing values.

We repeat, using -H 2:

%DCRAW% -v -6 -w -o 1 -H 2 -B 0 -T -O dcr_wbdef2.tiff %SRCNEF%

What are the minimum, mean and maximum channel values?

%IM%convert dcr_wbdef2.tiff ^
  -format "%%[fx:minima] %%[fx:mean] %%[fx:maxima]\n" ^
  info: 
0 0.14441 1

How many pixel channel values are at the extremes?

call %PICTBAT%propIntClipRGB dcr_wbdef2.tiff 
shadowClipped.0=11422
shadowClipped.1=23
shadowClipped.2=1538
highlightClipped.0=70589
highlightClipped.1=70498
highlightClipped.2=70498

That's better. Far fewer pixels have clipped, about 0.2%.

The image looks like this:

%IM%convert ^
  dcr_wbdef2.tiff ^
  %WEBSIZE% ^
  dcr_wbdef2.jpg
dcr_wbdef2.jpg

That looks better.

Settings of -H 3 and above look bad.

WB and dcraw -R n

%DCRAW% -v -4 -o 0 -T -H 1 -O x.tiff %SRCNEF%

%IM%identify -format %[fx:maxima] x.tiff

%DCRAW% -v -4 -o 0 -T -H 0 -R 0.25745 -O x.tiff %SRCNEF%

%IM%convert x.tiff -set colorspace RGB -colorspace sRGB s.tiff
:skip2

set RNum=2

%DCRAW% -v -4 -w -W -o 0 -H 0 -R %RNum% -T -O dcr_wbrn0.tiff %SRCNEF%

What are the minimum, mean and maximum channel values?

%IM%convert dcr_wbrn0.tiff ^
  -format "%%[fx:minima] %%[fx:mean] %%[fx:maxima]\n" ^
  info: 
0 0.0451321 0.958984

Calculate the maximum multiplied by the RNum:

for /F "usebackq" %%L in (`%IM%convert dcr_wbrn0.tiff ^
  -format "newRNum=%%[fx:maxima*%RNum%]\n" ^
  info:`) do set %%L

echo newRNum=%newRNum% 
newRNum=1.91797 

Re-run dcraw, using newRNum:

%DCRAW% -v -4 -w -W -o 0 -H 0 -R %newRNum% -T -O dcr_wbrn0.tiff %SRCNEF%

How many pixel channel values are at the extremes?

call %PICTBAT%propIntClipRGB dcr_wbrn0.tiff 
shadowClipped.0=62
shadowClipped.1=2
shadowClipped.2=12
highlightClipped.0=70446
highlightClipped.1=5
highlightClipped.2=1

Blah.

The image looks like this:

%IM%convert ^
  dcr_wbrn0.tiff ^
  %WEBSIZE% ^
  dcr_wbrn0.jpg

goto :eof
dcr_wbrn0.jpg

When using -R n, you should usually also use -W or -6.

WB and dcraw -r from gray patches

This method calculates white balance from gray patches on a colour card. During the photography session, we include a standard colour card in one of the photos. We automatically find the patches, and the lightest patch should be a neutral gray, so this gives us the channel multipliers.

First, we process the raw image in a "flat" manner, with no white balancing or conversion to sRGB. [[ The "-h" option reduces the time to one-third. ]] We allow a small proportion of pixels to clip in "auto brighten".

In the -r option, the four numbers are multipliers for the Bayer RGBG channels. Whatever numbers we give, the standard version of dcraw will normalise to make the lowest or the highest multiplier 1.0. If "-H 0" is used (the default), the lowest multiplier is made 1.0, so this will never lower values in any channels (and it can cause clipping). For higher "-H" settings, the highest multiplier is made 1.0, so this will never raise values in any channels (and it can't cause clipping). In either case, we can't use -r to change the overall lightness of the image.

(My patched version of dcraw has an extra option, -R n, which prevents normalisation of the -r white balance multipliers.)

set SRCNEF=%PICTLIB%20171029\AGA_3431.nef

set WEBSIZE=-resize 600x600

set dcrParams=-v -B 0.0001 -6 -T

%DCRAW% %dcrParams% -r 1 1 1 1 -o 0 -g 1 1 -O dcr_flat.tiff %SRCNEF%

Out of interest, we take a look at the image.
We pretend the image is sRGB, so IM won't convert it.
For the web, we show just the relevant part.

%IM%convert ^
  dcr_flat.tiff ^
  -set colorspace sRGB ^
  -gravity SouthWest -crop 35x35%%+0+0 ^
  %WEBSIZE% ^
  -quality 40 ^
  dcr_flat.jpg
dcr_flat.jpg

The image looks dark because the pixels values are linear but we are displaying as if they were sRGB.

Next, extract the colour card. See Finding and analysing colour charts for more detail.

call %PICTBAT%24card dcr_flat.tiff dcr_flat_XX.png

This gives a 6x4 pixel image, and a scaled-up version for viewing:

dcr_flat_scl.png

dcr_flat_scl.png

The bottom row of the 6x4 image should be neutral shades of gray. We could choose any pixel on that row to give us the multipliers for red and blue, compared to green, but the lightest patch will show a colour-cast most easily, so we choose it. We are not concerned with the absolute values in the channels (eg whether the patch is pure white or 90% white), but only with their relative values.

set FMT=^
r=%%[fx:mean.r]\n^
g=%%[fx:mean.g]\n^
b=%%[fx:mean.b]\n^
mr=%%[fx:mean.g/mean.r]\n^
mb=%%[fx:mean.g/mean.b]\n

for /F "usebackq" %%L in (`%IM%convert ^
  -precision 16 ^
  dcr_flat_mat.png ^
  -gravity SouthWest -crop 1x1+0+0 +repage ^
  -format "%FMT%" ^
  info:`) do set %%L

echo r=%r% g=%g% b=%b% 
echo mr=%mr% mb=%mb% 
@dcr_flat_white.lis

For convenience, we put this in a script, whiteMult.bat.

Re-run dcraw with those multipliers. This time, we include the conversion to sRGB.

%DCRAW% %dcrParams% -r %mr% 1 %mb% 1 -o 1 -O dcr_real.tiff %SRCNEF%

Out of interest, we take a look at the image.
For the web, we show just the relevant part.

%IM%convert ^
  dcr_real.tiff ^
  -gravity SouthWest -crop 35x35%%+0+0 ^
  %WEBSIZE% ^
  -quality 40 ^
  dcr_real.jpg
dcr_real.jpg

The white balance of the result looks good. We can check it by finding the card and examining the bottom row. (If we hadn't used "-h" for the first dcraw, we could re-use the coordinates from the first 24card.bat.)

call %PICTBAT%24card dcr_real.tiff dcr_real_XX.png

This gives a 6x4 pixel image, and a scaled-up version for viewing:

dcr_real_scl.png

dcr_real_scl.png

Rather than looking at numbers, we will graph the gray patches:

%IM%convert ^
  dcr_real_mat.png ^
  -gravity South ^
    -crop x1+0+0 +repage ^
  -scale "600x1^!" ^
  dcr_real_mat_gr.png

call %PICTBAT%graphLineCol ^
  dcr_real_mat_gr.png
dcr_real_mat_gr_glc.png

As expected, the lightest patch is neutral. The others are fairly close, though not exact. We have ignored the camera's WB metadata, and used dcraw features to balance a stepped gray card. We can use the same settings to balance all photos (or video frames) taken under the same conditions.

If we want a closer match to gray throughout the range, we can find the polynomials that most closely transform each channel to match the grayed patches, and hence to match each other. See Colour checker charts for more detail.

set mypolyr=
for /F "usebackq tokens=1,2 delims==" %%A in (`%IMDEV%convert ^
  dcr_real_mat.png ^
  -gravity South -crop x1+0+0 +repage ^
  ^( +clone ^
     -colorspace Gray ^
  ^) ^
  -precision 16 ^
  -process 'cols2mat method NoCrossPoly noTrans f stdout' ^
  NULL:`) do (
  if "%%A"=="PolyRed" set mypolyr=%%B
  if "%%A"=="PolyGreen" set mypolyg=%%B
  if "%%A"=="PolyBlue" set mypolyb=%%B
)
if "%mypolyr%"=="" goto error

echo mypolyr=%mypolyr% 
echo mypolyg=%mypolyg% 
echo mypolyb=%mypolyb% 
@dcr_poly.lis

For convenience, we put this in a script, grayPoly.bat.

We can apply these polynomials directly to the _mat image, and graph it.

%IM%convert ^
  dcr_real_mat.png ^
  -gravity South ^
    -crop x1+0+0 +repage ^
  -channel R -function Polynomial %mypolyr% ^
  -channel G -function Polynomial %mypolyg% ^
  -channel B -function Polynomial %mypolyb% ^
  +channel ^
  -scale "600x1^!" ^
  dcr_real_mat_gr2.png

call %PICTBAT%graphLineCol ^
  dcr_real_mat_gr2.png
dcr_real_mat_gr2_glc.png

The graph is greatly improved (though the difference in the patches is not visible).

Apply the polynomials to the output from dcraw, and also ensure the standard deviation is at least 0.166667. For the setmnsd module, see Set mean and stddev.

%IMDEV%convert ^
  dcr_real.tiff ^
  -channel R -function Polynomial %mypolyr% ^
  -channel G -function Polynomial %mypolyg% ^
  -channel B -function Polynomial %mypolyb% ^
  +channel ^
  -auto-level ^
  -depth 32 ^
  -define "quantum:format=floating-point" ^
  +write dcr_rp.tiff ^
  ( +clone ^
    %WEBSIZE% ^
    -quality 40 ^
    +write dcr_rp_sm.jpg ^
    +delete ^
  ) ^
  -process 'setmnsd sd 0.166667 d incOnly v' ^
  +write dcr_rp_sd.tiff ^
  %WEBSIZE% ^
  -quality 40 ^
  dcr_rp_sd_sm.jpg 

We show the result before and after setting the SD.

dcr_rp_sm.jpg dcr_rp_sd_sm.jpg

The setmnsd module has sent this to stdout:

@dcr_rp_sd_sm.lis

Developing without clipping

A purist (like me) might say we shouldn't allow dcraw to clip at all. What happens then? We repeat the above with auto-brighten but without any clipping.

set dcrParams=-v -B 0 -6 -T

%DCRAW% %dcrParams% -r 1 1 1 1 -o 0 -g 1 1 -O dcr_flat_nc.tiff %SRCNEF%

call %PICTBAT%24card dcr_flat_nc.tiff dcr_flat_nc_XX.png

call %PICTBAT%whiteMult dcr_flat_nc_mat.png dcr_wm

set dcr_wm 
@dcr_wm.lis
%DCRAW% %dcrParams% -r %dcr_wm_mr% 1 %dcr_wm_mb% 1 -o 1 -O dcr_real_nc.tiff %SRCNEF%

call %PICTBAT%24card dcr_real_nc.tiff dcr_real_nc_XX.png

call %PICTBAT%grayPoly dcr_real_nc_mat.png dcr_gp

set dcr_gp 
@dcr_gp.lis
%IMDEV%convert ^
  dcr_real_nc.tiff ^
  -channel R -function Polynomial %dcr_gp_polyr% ^
  -channel G -function Polynomial %dcr_gp_polyg% ^
  -channel B -function Polynomial %dcr_gp_polyb% ^
  +channel ^
  -auto-level ^
  -process 'setmnsd sd 0.166667 d incOnly v' ^
  +write dcr_rpnc_sd.tiff ^
  %WEBSIZE% ^
  -quality 40 ^
  dcr_rpnc_sd_sm.jpg 
@dcr_rpnc_sd_sm.lis

Here is the result:

dcr_rpnc_sd_sm.jpg

dcr_rpnc_sd_sm.jpg

Because we haven't allowed dcraw to clip, the output from dcraw is darker, with lower contrast (ie lower standard deviation). So setmnsd needed a steeper curve to make the SD 0.166667, which has visibly increased the saturation. In addition, setmnsd applies a sigmoidal, "S", curve instead of the dcraw linear multiplication, so this increases the contrast around the mid-point and decreases it at the extremes. In my opinion, both results are satisfactory.

Downsizing for the web needs some sharpening. I like the halo minimization technique, although this also slightly increases saturation.

call %PICTBAT%resampHM ^
  dcr_rpnc_sd.tiff ^
  %WEBDIMS% d 100 . . ^
  dcr_web.png
dcr_web.pngjpg

Brightness

dcraw has an option, -b, for setting the brightness. This multiples all the channels by the same number, after de-Bayering, but still in linear RGB space. If we have auto-brightened, some pixel values will already be at 100%, and -b will increase them further, causing more clipping.

I don't usually want clipping, so I don't use -b.

For basic "developing" of raw images by dcraw, I do the equivalent of IM "-auto-level" but no other tonal adjustment.

Clipping

As an indicator of clipping, the script propIntClipRGB.bat counts how many pixels in each channel are at the lowest value (shadow clipping) or highest value (highlight clipping). As the script does an "-auto-level" to each channel independently, we expect a value of at least one for each clipped.

call %PICTBAT%propIntClipRGB dcr_flat.tiff 
@dcr_flat_clp.lis
call %PICTBAT%propIntClipRGB dcr_real.tiff 
@dcr_real_clp.lis

Why do we have over a million pixels shadow-clipped? Experiments show that dcraw's "o 1" creates this.

call %PICTBAT%propIntClipRGB dcr_rp.tiff 
@dcr_rp_clp.lis
call %PICTBAT%propIntClipRGB dcr_rp_sd.tiff 
@dcr_rp_sd_clp.lis
call %PICTBAT%propIntClipRGB dcr_flat_nc.tiff 
@dcr_flat_nc_clp.lis
call %PICTBAT%propIntClipRGB dcr_real_nc.tiff 
@dcr_real_nc_clp.lis
call %PICTBAT%propIntClipRGB dcr_rpnc_sd.tiff 
@dcr_rpnc_sd_clp.lis

Use IM for sRGB conversion

The above development works well, except for the shadow clipping. This seems to be caused by dcraw's conversion to sRGB. So, we can do that in IM instead. This changes the process from the second dcraw onwards, but we show the complete process for clarity.

set dcrParams=-v -B 0 -6 -T -o 0 -g 1 1

%DCRAW% %dcrParams% -r 1 1 1 1 -O dcr_flat_nc2.tiff %SRCNEF%

call %PICTBAT%24card dcr_flat_nc2.tiff dcr_flat_nc2_XX.png

call %PICTBAT%whiteMult dcr_flat_nc2_mat.png dcr_wm2

set dcr_wm2 
@dcr_wm2.lis
%DCRAW% %dcrParams% -r %dcr_wm2_mr% 1 %dcr_wm2_mb% 1 -O dcr_real_nc2.tiff %SRCNEF%

call %PICTBAT%24card dcr_real_nc2.tiff dcr_real_nc2_XX.png

call %PICTBAT%grayPoly dcr_real_nc2_mat.png dcr_gp2

set dcr_gp2 
@dcr_gp2.lis
%IMDEV%convert ^
  dcr_real_nc2.tiff ^
  -channel R -function Polynomial %dcr_gp2_polyr% ^
  -channel G -function Polynomial %dcr_gp2_polyg% ^
  -channel B -function Polynomial %dcr_gp2_polyb% ^
  +channel ^
  -set colorspace RGB ^
  -colorspace sRGB ^
  -auto-level ^
  -process 'setmnsd sd 0.166667 d incOnly v' ^
  +write dcr_rpnc2_sd.tiff ^
  %WEBSIZE% ^
  -quality 40 ^
  dcr_rpnc2_sd_sm.jpg 
@dcr_rpnc2_sd_sm.lis

Here is the result:

dcr_rpnc2_sd_sm.jpg

dcr_rpnc2_sd_sm.jpg
call %PICTBAT%propIntClipRGB dcr_flat_nc2.tiff 
@dcr_flat_nc2_clp.lis
call %PICTBAT%propIntClipRGB dcr_real_nc2.tiff 
@dcr_real_nc2_clp.lis
call %PICTBAT%propIntClipRGB dcr_rpnc2_sd.tiff 
@dcr_rpnc2_sd_clp.lis

Future

The "-B 0" option works well, stretching values upwards to use the full 16-bit range without clipping. Or we can allow any proportion of pixels to clip. But what proportion is reasonable? I think 1% is far too high, but how can we determine the right amount? One possibility: any pixel that is already at maximum in at least one channel, and has high values in the other channels, can be safely clipped.

Aside: dcraw contains code, in function colorcheck(), that does some processing with pixels of a 24-colour card in an image. The coordinates of each patch (w, h, x, y) should be put in the array cut[24][4], but the only way of doing this is to modify the dcraw source code. Weird.

Scripts

For convenience, .bat scripts are also available in a single zip file. See Zipped BAT files.

whiteMult.bat

rem From bottom-left pixel of image %1,
rem calculates multipliers green/red and green/blue.
rem Sets environment variables prefixed by %2.

@if "%1"=="" findstr /B "rem @rem" %~f0 & exit /B 1

@setlocal enabledelayedexpansion

@call echoOffSave

call %PICTBAT%setInOut %1 wm

set PREF=%2
if "%PREF%"=="." set PREF=
if "%PREF%"=="" set PREF=whiteMult


set r=

set FMT=^
r=%%[fx:mean.r]\n^
g=%%[fx:mean.g]\n^
b=%%[fx:mean.b]\n^
mr=%%[fx:mean.g/mean.r]\n^
mb=%%[fx:mean.g/mean.b]\n

for /F "usebackq" %%L in (`%IM%convert ^
  -precision 16 ^
  %INFILE% ^
  -gravity SouthWest -crop 1x1+0+0 +repage ^
  -format "%FMT%" ^
  info:`) do set %%L

if "%r%"=="" exit /B 1

echo %PREF%_r=%r% %PREF%_g=%g% %PREF%_b=%b% 
echo %PREF%_mr=%mr% %PREF%_mb=%mb%

call echoRestore

@endlocal & set %PREF%_r=%r%&set %PREF%_g=%g%&set %PREF%_b=%b%& set %PREF%_mr=%mr%&set %PREF%_mb=%mb%

grayPoly.bat

rem From botom row of image %1, assuming pixels should be neutral grays,
rem calculates channel polynomials.
rem Sets environment variables prefixed by %2.

@if "%1"=="" findstr /B "rem @rem" %~f0 & exit /B 1

@setlocal enabledelayedexpansion

@call echoOffSave

call %PICTBAT%setInOut %1 gp

set PREF=%2
if "%PREF%"=="." set PREF=
if "%PREF%"=="" set PREF=grayPoly


set polyr=
for /F "usebackq tokens=1,2 delims==" %%A in (`%IMDEV%convert ^
  %INFILE% ^
  -gravity South -crop x1+0+0 +repage ^
  ^( +clone ^
     -colorspace Gray ^
  ^) ^
  -precision 16 ^
  -process 'cols2mat method NoCrossPoly noTrans f stdout' ^
  NULL:`) do (
  if "%%A"=="PolyRed" set polyr=%%B
  if "%%A"=="PolyGreen" set polyg=%%B
  if "%%A"=="PolyBlue" set polyb=%%B
)
if "%mypolyr%"=="" exit /B 1

echo %PREF%_polyr=%polyr% 
echo %PREF%_polyg=%polyg% 
echo %PREF%_polyb=%polyb% 

call echoRestore

@endlocal & set %PREF%_polyr=%polyr%&set %PREF%_polyg=%polyg%&set %PREF%_polyb=%polyb%

propIntClipRGB.bat

@rem From image %1,
@rem after auto-levelling each channel independently,
@rem finds number of pixels that have 0 or 100% in each channel.

@%IM%convert ^
  %1 ^
  -channel RGB ^
  -auto-level ^
  -separate ^
  +channel ^
  ( -clone 0-2 ^
    -fill White +opaque Black ^
    -negate ^
    -format "shadowClipped.%%s=%%[fx:int(mean*w*h+0.5)]\n" ^
    +write info: ^
    -delete 0-2 ^
  ) ^
  -fill Black +opaque White ^
  -format "highlightClipped.%%s=%%[fx:int(mean*w*h+0.5)]\n" ^
  info:

dcrawag.diff

--- dcraw.c	2017-11-06 23:02:40.011105100 +0000
+++ dcrawimg.c	2018-03-17 06:12:12.234571600 +0000
@@ -2,6 +2,8 @@
    dcraw.c -- Dave Coffin's raw photo decoder
    Copyright 1997-2016 by Dave Coffin, dcoffin a cybercom o net
 
+   [This file has been slightly hacked by Alan Gibson.]
+
    This is a command-line ANSI C program to convert raw photos from
    any digital camera on any computer running any operating system.
 
@@ -21,6 +23,9 @@
 
    $Revision: 1.477 $
    $Date: 2016/05/10 21:30:43 $
+
+   Updates (by Alan Gibson):
+     17-March-2018 Added "-R" option.
  */
 
 #define DCRAW_VERSION "9.27"
@@ -28,6 +33,8 @@
 #ifndef _GNU_SOURCE
 #define _GNU_SOURCE
 #endif
+#define WIN32_LEAN_AND_MEAN
+#define VC_EXTRALEAN
 #define _USE_MATH_DEFINES
 #include <ctype.h>
 #include <errno.h>
@@ -42,7 +49,7 @@
 #include <time.h>
 #include <sys/types.h>
 
-#if defined(DJGPP) || defined(__MINGW32__)
+#if defined(DJGPP) || defined(__MINGW32__) || defined(WIN32) || defined(WIN64)
 #define fseeko fseek
 #define ftello ftell
 #else
@@ -68,6 +75,10 @@
 typedef unsigned long long UINT64;
 #endif
 
+#if !defined(M_PI)
+#  define M_PI 3.14159265358979323846
+#endif
+
 #ifdef NODEPS
 #define NO_JASPER
 #define NO_JPEG
@@ -104,6 +115,7 @@
 FILE *ifp, *ofp;
 short order;
 const char *ifname;
+const char *outfile = NULL;
 char *meta_data, xtrans[6][6], xtrans_abs[6][6];
 char cdesc[5], desc[512], make[64], model[64], model2[64], artist[64];
 float flash_used, canon_ev, iso_speed, shutter, aperture, focal_len;
@@ -140,6 +152,9 @@
 void (*write_thumb)(), (*write_fun)();
 void (*load_raw)(), (*thumb_load_raw)();
 jmp_buf failure;
+double BrightThr=0.0001;
+int noNormWBM=0;
+float wbmDiv=1.0;
 
 struct decode {
   struct decode *branch[2];
@@ -4247,6 +4262,13 @@
 	dmax = pre_mul[c];
   }
   if (!highlight) dmax = dmin;
+  if (noNormWBM) dmax = wbmDiv;
+  if (verbose) {
+    fprintf (stderr,
+      _("Multipliers pre-norm: ") );
+    FORC4 fprintf (stderr, " %f", pre_mul[c]);
+    fputc ('\n', stderr);
+  }
   FORC4 scale_mul[c] = (pre_mul[c] /= dmax) * 65535.0 / maximum;
   if (verbose) {
     fprintf (stderr,
@@ -9538,6 +9560,21 @@
 	for (out_cam[i][j] = k=0; k < 3; k++)
 	  out_cam[i][j] += out_rgb[output_color-1][i][k] * rgb_cam[k][j];
   }
+
+  if (verbose) {
+    fprintf (stderr, "rgb_cam: ");
+    for (i=0; i < 3; i++)
+      for (j=0; j < colors; j++)
+        fprintf (stderr, " %g", rgb_cam[i][j]);
+    fprintf (stderr, "\n");
+
+    fprintf (stderr, "out_cam: ");
+    for (i=0; i < 3; i++)
+      for (j=0; j < colors; j++)
+        fprintf (stderr, " %g", out_cam[i][j]);
+    fprintf (stderr, "\n");
+  }
+
   if (verbose)
     fprintf (stderr, raw_color ? _("Building histograms...\n") :
 	_("Converting to %s colorspace...\n"), name[output_color-1]);
@@ -9794,7 +9831,7 @@
   int c, row, col, soff, rstep, cstep;
   int perc, val, total, white=0x2000;
 
-  perc = width * height * 0.01;		/* 99th percentile white level */
+  perc = width * height * BrightThr;		/* eg 0.01 for 99th percentile white level */
   if (fuji_width) perc /= 2;
   if (!((highlight & ~2) || no_auto_bright))
     for (white=c=0; c < colors; c++) {
@@ -9802,6 +9839,12 @@
 	if ((total += histogram[c][val]) > perc) break;
       if (white < val) white = val;
     }
+
+  if (verbose)
+    fprintf (stderr,_("bright=%f, gamma_curve imax=%i\n"),
+      bright,
+      (int)floor((white << 3)/bright));
+
   gamma_curve (gamm[0], gamm[1], 2, (white << 3)/bright);
   iheight = height;
   iwidth  = width;
@@ -9862,6 +9905,7 @@
   if (argc == 1) {
     printf(_("\nRaw photo decoder \"dcraw\" v%s"), DCRAW_VERSION);
     printf(_("\nby Dave Coffin, dcoffin a cybercom o net\n"));
+    printf(_("slightly hacked by Alan Gibson v2.0\n"));
     printf(_("\nUsage:  %s [OPTION]... [FILE]...\n\n"), argv[0]);
     puts(_("-v        Print verbose messages"));
     puts(_("-c        Write image data to standard output"));
@@ -9873,8 +9917,10 @@
     puts(_("-a        Average the whole image for white balance"));
     puts(_("-A <x y w h> Average a grey box for white balance"));
     puts(_("-r <r g b g> Set custom white balance"));
+    puts(_("-R n      Divisor for white balance multipliers"));
     puts(_("+M/-M     Use/don't use an embedded color matrix"));
     puts(_("-C <r b>  Correct chromatic aberration"));
+    puts(_("-O <file> Write output to this file"));
     puts(_("-P <file> Fix the dead pixels listed in this file"));
     puts(_("-K <file> Subtract dark frame (16-bit raw PGM)"));
     puts(_("-k <num>  Set the darkness level"));
@@ -9891,6 +9937,7 @@
     puts(_("-D        Document mode without scaling (totally raw)"));
     puts(_("-j        Don't stretch or rotate raw pixels"));
     puts(_("-W        Don't automatically brighten the image"));
+    puts(_("-B <num>  Clipping threshold for auto-brighten (default = 0.0001)"));
     puts(_("-b <num>  Adjust brightness (default = 1.0)"));
     puts(_("-g <p ts> Set custom gamma curve (default = 2.222 4.5)"));
     puts(_("-q [0-3]  Set the interpolation quality"));
@@ -9918,6 +9965,10 @@
       case 'b':  bright      = atof(argv[arg++]);  break;
       case 'r':
 	   FORC4 user_mul[c] = atof(argv[arg++]);  break;
+      case 'R':
+           noNormWBM = 1;
+           wbmDiv = atof(argv[arg++]);
+           break;
       case 'C':  aber[0] = 1 / atof(argv[arg++]);
 		 aber[2] = 1 / atof(argv[arg++]);  break;
       case 'g':  gamm[0] =     atof(argv[arg++]);
@@ -9943,6 +9994,7 @@
 #endif
 	break;
       case 'P':  bpfile     = argv[arg++];  break;
+      case 'O':  outfile    = argv[arg++];  break;
       case 'K':  dark_frame = argv[arg++];  break;
       case 'z':  timestamp_only    = 1;  break;
       case 'e':  thumbnail_only    = 1;  break;
@@ -9965,6 +10017,7 @@
       case '4':  gamm[0] = gamm[1] =
 		 no_auto_bright    = 1;
       case '6':  output_bps       = 16;  break;
+      case 'B':  BrightThr        = atof(argv[arg++]);  break;
       default:
 	fprintf (stderr,_("Unknown option \"-%c\".\n"), opt);
 	return 1;
@@ -10226,7 +10279,10 @@
     if (write_to_stdout)
       strcpy (ofname,_("standard output"));
     else {
-      strcpy (ofname, ifname);
+      if (outfile)
+        strcpy (ofname,outfile);
+      else {
+        strcpy (ofname, ifname);
       if ((cp = strrchr (ofname, '.'))) *cp = 0;
       if (multi_out)
 	sprintf (ofname+strlen(ofname), "_%0*d",
@@ -10234,6 +10290,7 @@
       if (thumbnail_only)
 	strcat (ofname, ".thumb");
       strcat (ofname, write_ext);
+      }
       ofp = fopen (ofname, "wb");
       if (!ofp) {
 	status = 1;

All images on this page were created by the commands shown, using:

%IM%identify -version
Version: ImageMagick 6.9.5-3 Q16 x86 2016-07-22 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2015 ImageMagick Studio LLC
License: http://www.imagemagick.org/script/license.php
Visual C++: 180040629
Features: Cipher DPC Modules OpenMP 
Delegates (built-in): bzlib cairo flif freetype jng jp2 jpeg lcms lqr openexr pangocairo png ps rsvg tiff webp xml zlib
%DCRAW% | head -n 4 
 Raw photo decoder "dcraw" v9.27
by Dave Coffin, dcoffin a cybercom o net
slightly hacked by Alan Gibson

To improve internet download speeds, some images may have been automatically converted (by ImageMagick, of course) from PNG to JPG.

Source file for this web page is dcrawwb.h1. To re-create this web page, execute "procH1 dcrawwb".


This page, including the images, is my copyright. Anyone is permitted to use or adapt any of the code, scripts or images for any purpose, including commercial use.

Anyone is permitted to re-publish this page, but only for non-commercial use.

Anyone is permitted to link to this page, including for commercial use.


Page version v1.0 9-November-2017.

Page created 17-Jul-2018 11:36:41.

Copyright © 2018 Alan Gibson.