From: John Sheehy on
"David J. Littleboy" <davidjl(a)gol.com> wrote in
news:esq8u8$9tm$1(a)nnrp.gol.com:

> In the 5D and 1Dmk2, the ISO 100 read noise looks to me to be
> dominated by quantization errors;

The standard deviation of an ISO 100 blackframe is about 1.26 ADU on a
mkII, and 1.97 on a 5D (it's 0.9 ADU on the Pentax K10D). These are all
quite a bit more than what is possible with quantization, and the ranges
near black are where quantization is potentially most harmful.

With the 14-bit 1DmkIII, the blackframe read noise at ISO 100 is 4.88
ADU; not an improvement over the 1.26 of the 12-bit successor, and a hint
that the mkII's 1.26 has nothing much to do with posterization (and how
do you posterize black, anyway?).

> the bit depth of the A/D converter
> is two bits shy of what's needed, maybe three.

It's definitely shy of what is usable within the sensor wells. We really
can't tell at what point in the signal chain the high read noise (in
electrons) comes from, but it is really there, so even if sensor wells
are only limited by shot noise (and dark current noise, when applicable),
the *camera* is still subjected to high read noise. The blackframe read
noise at ISO 100 is generally in the 20 to 30 electron range. That means
that two neighboring pixels could be as much as 100 electrons off, as far
as their recorded and real differences are concerned.

> The dynamic range at
> ISO 100 to 400 is simply consistent with a 12-bit A/D converter.

There really aren't that many samples of unique camera arrangements where
you can observe one effect in isolation. We are victims of all kinds of
coincidences in our observations.

Anyway, regardless whether its the ADC, or the first amplifiers, the
camera, somewhere, is picking up tremendous read noise, as measured in
relative signal electrons, at the lowest ISOs electrons. You don't need
more than 12 bits to get more DR out of these cameras; you need less read
noise. You want just enough read noise to dither away any posterization.
There is *far* more than what is needed for that in current cameras.

> It's
> only at ISO 800 and above that other noise sources intrude.
>
> That's why the D200 has the same ISO 100 dynamic range as the 5D.

Does it? Most of the tests I've seen are about the RAW conversions,
which tend to equalize things a bit; not RAW comparisons. The D200 may
have less shot noise, but I think it also has more read noise, IIRC, and
stronger banding noise (horizontal, not vertical as in those high-
contrast edge problems), hidden by even stronger 2D random read noise.
Nikon also clips their RAW data at the blackpoint; not a very good idea
if you're really interested in clean deep shadows.

>> Shot noise is
>> the least of our digital imaging problems, IMO, especially with large
>> sensors.
>
> I'm not seeing pattern noise in pused images. Here's a 5D ISO 3200
> file pushed 3 stops. Straight from Lightroom with noise reduction (and
> sharpening) turned off.
>
> http://www.pbase.com/davidjl/image/75374090/original

There's a strong horizontal banding component there. Load it into
photoshop, and open the motion blur tool. Set it to 0 degrees, and 50
pixels. Then, change it to 90 degrees. See the difference? The
horizontal blur creates much stronger horizontal streaks than the
vertical does vertical, because there is horizontal streaking in the
image, the pattern of which is masked by the more random 2D noise, but
the strength of which is not.

--

<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<>
John P Sheehy <JPS(a)no.komm>
><<> <>>< <>>< ><<> <>>< ><<> ><<> <>><
From: acl on
On Mar 9, 5:30 am, John Sheehy <J...(a)no.komm> wrote:

> which tend to equalize things a bit; not RAW comparisons. The D200 may
> have less shot noise, but I think it also has more read noise, IIRC, and
> stronger banding noise (horizontal, not vertical as in those high-
> contrast edge problems), hidden by even stronger 2D random read noise.

Where did you see this horizontal banding? I've never seen anything
like that in my raw files, only vertical bands, eg here:
http://www.pbase.com/al599/image/74719850
I once took a black frame, dumped the data using dcraw, and then
looked at the fourier transform of parts of the frame, and saw a
smoothed spike indicating patterns periodic in the horizontal
direction (ie vertical banding). But that was what I was looking for
so maybe I missed something. Or maybe it is dcraw. I don't know. So,
how did you see horizontal banding?

> Nikon also clips their RAW data at the blackpoint; not a very good idea
> if you're really interested in clean deep shadows.
>

From: John Sheehy on
"acl" <achilleaslazarides(a)yahoo.co.uk> wrote in
news:1173408339.946676.128380(a)v33g2000cwv.googlegroups.com:

> Where did you see this horizontal banding?

It was not readily visible in the D200, for the reason I mentioned. The
ratio of 2D noise to 1D noise determines how well you can see the 1D noise.
I extracted the 1D components from the image, from an unilluminated area,
and the intensity was actually greater than with my Canon 20D, which has a
reputation for banding at high ISOs. I subsequently did the same for
several cameras by various manufacturers, and found that they all had
banding, and all had stronger banding than the 20D, but their 2D random
noise was so much more so that the banding was masked.

By banding, I mean offsets in the RAW data on a line-by-line basis, visible
or not. Even when banding is not visible as "banding", it's removal still
makes the remaining noise look much more natural (except, of course, for
the inherent pixel grid of both the signal and noise), despite the fact
that the standard deviation may drop by less than 1%. Any patterned or 1D
noise, is much more destructive per unit of standard deviation, than 2D
random noise. Even binning and downsampling fail to reduce 1D noise at the
rate it reduces 2D noise; same for viewing full-res images from a distance,
or printed small; the lines do not fade away, whether they are perceived as
lines or not.

I've taken blackframes from my Canons and binned them down to ridiculous
levels, and all that remains is horizontal and vertical lines with no 2D
random noise left; the bands are resilient.

--

<>>< ><<> ><<> <>>< ><<> <>>< <>>< ><<>
John P Sheehy <JPS(a)no.komm>
><<> <>>< <>>< ><<> <>>< ><<> ><<> <>><
From: Roger N. Clark (change username to rnclark) on
David J. Littleboy wrote:
> "Paul Rubin" <http://phr.cx(a)NOSPAM.invalid> wrote:
>
>>"David J. Littleboy" <davidjl(a)gol.com> writes:
>>
>>>I'm not seeing pattern noise in pushed images. Here's a 5D ISO 3200 file
>>>pushed 3 stops. Straight from Lightroom with noise reduction (and
>>>sharpening) turned off.
>>>
>>>http://www.pbase.com/davidjl/image/75374090/original
>>
>>You mean this is at 25,600?
>
>
> Exactly!

David,
Pretty cool demonstration.

Here is pushing some other limits with a Canon 1D Mark II:

Night and Low Light Photography with Digital Cameras
http://www.clarkvision.com/photoinfo/night.and.low.light.photography

Figures 3a, 3b are 623 second exposures at ISO 1600.

Figure 4, set 2 equivalent to ISO 66,450!
Figure 4, set 3 is an average of 6 frames at ISO 374,000!

Figure 5, set 5 is a single frame at ISO 3,883,000 !!!

Figure 8: night scene at equivalent ISO 16,000.
Figure 9: night scene at equivalent ISO 64,000 (no dark subtraction).
Figure 12: night scene at equivalent ISO320,000 (with dark subtraction).

Figure 13: night scene: 64 frames averaged at ISO 320,000.

Try that with film: you would get nothing.

For Ray:
Electronic sensors have quantum efficiencies in the
30% range, and film a percent or so, so right there, one
sees that electronic sensors are much more sensitive.
Next combine the fact that there is a threshold with film
requiring a fair number of photons before a grain will
record the light, then add the fact of reciprocity failure
and there are significant technical reasons why film does not do
well in low light situations.

Add the above facts and it is no wonder why amateur astronomers
had pretty much abandoned film for digital cameras.
Amateur astronomers are getting much better astrophotos
with DSLRs than they every did with film, even when using
DSLRs on smaller telescopes in light polluted skies!
The results are truly astounding. For example:
a 27-minute exposure from light polluted Denver of the Pleiades:
http://www.clarkvision.com/galleries/gallery.astrophoto-1/web/m45-700MM-8534-8561_C16B-add27-v3-800.html
is better than I ever did with the best long exposure professional
astronomical film (103aF) from dark skies and longer exposures
with faster lenses.

Action: DSLRs produce better images at high ISO than any film
I ever used. E.g., here is a lion eating a zebra before
sunrise on the Serengeti a few weeks ago (1/250 sec at ISO 800):
http://www.clarkvision.com/galleries/gallery.africa/web/lion.c01.23.2007.JZ3F0240b-700.html
I couldn't have gotten much of an image with film pushed
to ISO 800. Other examples are on my web site.

Roger
From: Paul Rubin on
"Roger N. Clark (change username to rnclark)" <username(a)qwest.net> writes:
> Here is pushing some other limits with a Canon 1D Mark II:
>
> Night and Low Light Photography with Digital Cameras
> http://www.clarkvision.com/photoinfo/night.and.low.light.photography

Wow, neat. The super-high-ISO examples have very visible horizontal
banding--what happens if you take that out with a notch filter?

Is there a feasible way to remove the Bayer filter from a DSLR sensor?

What about shorter exposures at super ISO's?