Jim, I like your description of the test procedure. It is quite a bit more
sophisticated than mine, but based on the same intuitions. We want to
subject the competing cameras to the same source, as well as the same
optics. In theory, one could go further by using a calibrated light
source, but most of us don;t quite need that level of sophistication.

Again, if it will serve the community, I'll provide resources to do camera
benchmarking in my lab.

Badri Roysam
Associate Professor
Advanced Imaging Systems Laboratory
Department of Electrical, Computer and Systems Engineering
Rensselaer Polytechnic Institute
110 8th Street, Troy, New York 12180-3590.
Office(JEC 6046): 518-276-8067, Lab(JEC 6308): 518-276-8207, Fax: 518-276-6261
Email: [log in to unmask], Web: http://www.rpi.edu/~roysab

On Wed, 28 Feb 2001, James Pawley wrote:

> I would like to echo Badri's point:
>
> Everyone is in favor of more "sensitivity" but no one knows what it is.
>
> I teach that it is a mixture of quantum efficiency (the fraction of
> the photons arriving that actually contribute to the signal) and read
> out noise. The former should be high, the latter low (a few
> electrons/pixel/sec on any reasonable CCD. Readout noise should NOT
> to be confused with dark current: only the square root of the dark
> current (also in electrons.pixel/sec) is a source of noise) .  QE and
> Read Noise are two numbers that should be available on any
> quasi-scientific camera.
>
> If not, a way to test one camera against another just requires a
> standard set up.
>
> I suggest that you do NOT use a microscope but a putting the camera
> on a tripod, and add a c-mount lens with a calibrated aperture (f/
> numbers 2.2.8, 4, 5.6, etc, each of which represents a factor of 2
> change in light throughput.) and probably a ND 3 (1000:1) filter.
>
> Set the camera to make an image of some part of your lab lit only by
> normal, artificial room lighting (you don't want sunlight to change
> the readings).
>
> If it is a video rate camera: connect to a monitor with the contrast
> all the way up and the  brightness set where you can see as much as
> possible (up to you). If it is a digital camera: connect to the
> computer: set the contrast as high as helps you see some large
> object. (You can use a video-rate image processor if you plan to use
> one on the  microscope. The "no-signal" light level will be lower
> than in an unprocessed image, but just as reproducible.)
>
> Close the aperture down (bigger numbers) until you can't see any
> image (Add the ND 3 filter if needed to get NO signal).
>
> Note the setting of the aperture.
>
> Place camera number 2 in the same set-up and note the "no-signal" aperture.
>
> The camera capable of seeing something with the smallest aperture is
> the most sensitive and the factor of improvement is 2 to the power of
> the number of click stops on the aperture between the two settings.
>
> This test measures what is called the "noise-equivalent light level"
> i.e., when the signal from the sensor (which depends on QE) equals
> the noise level of the statistical variations in the dark current and
> of the circuitry needed to read it out. It is as good a measure of
> the camera "sensitivity" as you will get.
>
> But you will have to do it yourself, the manufacturers aren't clued
> into it (or don't want to be). Fortunately it is easy to do.
> Remember, when comparing digital and video cameras, you need to make
> the digital exposure times equivalent to video rate. (about 33 ms/
> image)
>
> Of course, sensitivity isn't the only thing. Resolution is also
> important.  I have made a test object on my laser printer in the form
> of a set of sets of nested, "l-shaped" lines. Each set of "Ls" used a
> different line width and a line spacing equal to twice the line
> width. (i.e., black and white bars of varying period)
>
> Image such a pattern through the lens as above and note the aperture
> at which the finer lines become indistinguishable. You will notice
> that vertical lines usually disappear before horizontal lines on
> video cameras.
>
> Just a thought.
>
> Jim P.
>
>
>
> >Dear Colleague:
> >
> >I have been actively searching for a sensitive frame rate imager for a
> >considerable period of time, and here are some things that I have learned:
> >
> >1. There are two common technologies - Charge Coupled Devices (CCD), and
> >Complementary Metal Oxide .. (CMOS). The latter is suggested as the way of
> >the future, and set to upstage CCDs soon.
> >
> >2. There is no common way of describing camera sensitivity!!!!!!!
> >
> >Perhaps this is a project for Barbara Foster's group. There is a very
> >real need for a common benchmark measurement. I am willing to
> >help!! I'll be happy to provide lab space, manpower, and a web server to
> >the community. It is important enough to warrant community involvement.
> >
> >  Each manufacturer has its own way of stating the sensitivity. What's
> >worse, a manufacturer will just as likely state sensitivities for two
> >different products of their own differently!!!!! What's even worse,
> >is that several manufacturers don't even bother to give you these numbers.
> >
> >Bottom line: You have to try out these cameras. DO NOT BUY ONE WITHOUT
> >TRYING. THE SPEC NUMBERS ARE NEARLY USELESS. DO NOT GO BY THE SALES PITCH.
> >THEY CAN BE VERY MISLEADING FOR A VARIETY OF REASONS (EVEN WITH GOOD
> >PEOPLE!).
> >
> >3. Here are some basic things to look for before trying out a camera:
> >
> >a. Spectral response curve (useful only as a relative measure, the units
> >for response are all different, so beware). Make sure the camera has
> >adequate sensitivity in the wavelengths of interest.
> >
> >b. The effective quantum efficiency (QE). Again, there is room for
> >confusion. The real number of interest is the product of the camera's fill
> >factor, and the QE of the imaging surface. This gives you a good starting
> >point when selecting cameras to try out. Expect numbers in the 30 - 70%
> >range.
> >
> >c. The number of bits (8/10/12) is a bit of a phantom figure. Usually, the
> >achievable dynamic range is lower, and many of the lower bits are just
> >noise.
> >
> >4. The business of recording image sequences to disk is still relatively
> >immature. The best technology that I've come across thus far (having tried
> >it!) is Boulder Imaging Inc (http://www.cjvision.com).
> >
> >Badri Roysam
> >Associate Professor
> >Advanced Imaging Systems Laboratory
> >Department of Electrical, Computer and Systems Engineering
> >Rensselaer Polytechnic Institute
> >110 8th Street, Troy, New York 12180-3590.
> >Office(JEC 6046): 518-276-8067, Lab(JEC 6308): 518-276-8207, Fax: 518-276-6261
> >Email: [log in to unmask], Web: http://www.rpi.edu/~roysab
>
> --
>                ****************************************
> Prof. James B. Pawley,                             Ph.  608-263-3147
> Room 223, Zoology Research Building,               FAX  608-265-5315
> 1117 Johnson Ave., Madison, WI, 53706  [log in to unmask]
> "A scientist is not one who can answer questions
> but one who can question answers."
>                 Theodore Schick Jr., Skeptical Enquirer, 21-2:39
>