CCD Smear Reduction
Introduction
CCD imaging devices comprise an array of photo-sensitive sites (typically silicon photodiodes) which can convert light into electrical charge which is then stored within the site (often in the photo-diode capacitance). The CCD then holds a "charge image" which represents the total light exposure at each position since the start. When exposure is complete, the charge stored in each detector site can be "dragged" from site to site my a moving pattern of electric fileds until it reaches an output charge amplifier where it can be "read out" - usually by conversion to a voltage. The charge transfer across the device is highly efficient, and scientific-grade CCDs (especially if cooled) may offer the user extremely low noise levels (only a few electrons equivalent per readout).Scientific grade CCDs typically transfer the exposed image charge from photo-site to photosite rather than using a separate system of charge-transfer registers (which is more typical of video-rate CCDs). The whole charge-image is typically shifted down the array of photosites one row at a time, and the bottom row is read out by a horizontal shift into an output amplifier. The photosites remain light-sensitive whilst this transfer is taking place, and it is usual to cut off the image illumination whilst the charge transfer operation is going on, perhaps by cutting off a light source or by using a shutter. These measures prevent additional exposure causing problems.
Image Smear
If the image is not removed during charge transfer (perhaps because interposing a shutter is not practical, and the image source cannot be turned off), then problems can occur. Imagine the situation at a particular photosite part-way through the charge transfer process: the charge residing in the site at that instant relates to the amount of exposure received at a photosite some distance away across the device because the charge-image has been shifted. If there is additional exposure at the photosite under consideration from the light image (which does not shift!), then the extra charge generated will add to that currently residing in the photosite. The two packets of charge correspond to different places on the image, so the output signal is now confused. Because the charge transfer operation is usually linear, the result is "streaking" or "smearing" behind any bright section of the image.The same effect could be generated by pointing a camera at a scene with the shutter open for a while, and then moving it upwards or downwards with the shutter still open. The "static" part of the exposure would give a clear image, but the "moving" part of the exposure would give a blurred image superposed on the clear image.
Example
The
example image is that of a micro-well plate as used for biological
assays. In this case some of the wells are emitting a
bioluminescent glow as a result of the action of a drug on cells which
have expressed a bio-luminescent protein called Aequorin. The
drug has triggered the release of calcium ions and these in turn cause
the Aequorin to glow. There are four rows of uniform wells, and
then a set of four part-filled rows in which the luminescece reduces
from row to row. The exposure has been made over a good fraction of a second using a very high quality cooled CCD imager, but it has not been possible to interpose a shutter, and the cells cannot be "turned off". The rvertical eadout operation required a time similar to the exposure, and the resulting smear effect is quite serious: it is certainly enough to prevent any quantitative analysis of the image (for example to determine the efficacy of the drug dosage in each well).
Smear Removal
It is possiblle to derive a mathematical formula for the exposure process which accounts for the two phases of the exposure without a-priori knowledge of the form of the image. In one formulation, the charge at any point along a single column can be shown to be the sum of the exposure received at that point in the image plus a proportion of the line integral of the image exposure along the column below the current point. The signal is given by a Volterra integral equation. This type of equation can be solved (at least in principle) as long as the various exposure-related parameters are known or can be estimated from the image.A practical solution can be attempted by starting at the bottom of the column: the signal in the next image pixel upwards is given by the exposure at the next position plus a constant times the exposure in the bottom position. A similar argument can be made for the third row and indeed for all the rows up the column to the top.
The correct scaling constants can be supplied by knowing the exposure and readout times (and assuming that the image remains constant throughout both periods). Alternatively, it is also possible to estimate the optimum constants from teh image itself if some constratints can be applied (such as knowing the image exposure at the top and bottom of the column for example). This approach can accommodate a time-varying image brightness (very important for the bio-luminescent example!) and also a non-uniform sensitivity across the photosites of the CCD.
The
second image shows the result of applying the un-smearing algorithm to
the test image. It is immediately obvious that the bright
vertical streaks have gonce, and it is also possible to see clearly the
variation in brightenss within the lower rows of wells which had
prevuiously been "swamped" by the smear. This image can be used
for quantitative analysis.No Free Lunch!
The unsmearing algorithm is very successful
indeed, even with "real" rather than test images. It has been
deployed commercially and has also been patented.
It is not however without its limitations. Although the intensity
offsets associated with the smear have been wholly removed, the extra
light that was detected by each photosite during the charge shifting
part of the exposure carries with it extra noise (this is "shot" noise
- the CCD collects charge as finite numbers of discrete charge
carriers). The algorithm is not able to remove the noise associated with the smear - which is proportional to the square-root of the smear exposure. The corrected image has been contrast-enhanced to show the faint residual noise in each smeared column. The lesson is clear: all exposure incurs noise, including the "smeared" exposure, and noise cannot be avoided!
That said, the noise in this example is quite extreme because the four rows of bright wells are near full-scale exposure, and the camera has 16-bit-equivalent dynamic range. The method is entirely practical and can transform an un-usable image into a quantitative measurement.