Introduction to 2D Fourier Transforms for Sensor Analysis
Prepared 2017-01-29 by Bill Claff (revised 2017-01-30)
The Fourier Transform (FT) can be used a number of ways to
analyze and process images.
In this article we'll concentrate on the narrow topic of 2D FTs as applied to sensor analysis.
If you want to play with FT yourself a good place to start is with ImageJ.
Note that many authors, including myself when I slip up,
will use the acronym FFT rather than FT.
FFT stands for Fast Fourier Transform and is an algorithm commonly used to compute FTs.
I am going to present a number of synthetically generated
images that represent our understanding of what we ought to observe.
In each case the image will be presented on the left and the 2D FT for that image on the right.
The left-hand images are linked to PGM files so you can play with them yourself.
(You will want to use Save link as ... rather than Save image as ... to get the PGM otherwise you'll get a JPG.)
The style of PGM file provided can be opened in ImageJ as well as any text editor or even Excel.
Here's a diagram of the roadmap I will follow:
Now we add read noise:
Read noise arises principally from the electronics used to read the value of the pixel (hence the name).
This represents what we might reasonably expect to see from a well behaved sensor.
(Although non-DSNU FPN is not unusual and often presents as horizontal or vertical streaks.
Also, on some sensors, on chip Phase Detect Auto Focus (PDAF) may form a visible pattern.)
We are not expecting to see noise reduction in our raw
image; but it happens.
Noise reduction, hot pixel suppression, and other signal processing share the same general characteristic.
They all re-compute a pixel value based on neighboring pixels.
So to simulate this effect I added a small Gaussian Blur to
the typical black image:
I boosted the contrast on the 2D FT to make the effect more obvious.
This round "sphere-like" effect is an important feature to remember.
Black images are interesting but I also frequently analyze
evenly illuminated images.
So here's what Signal alone would look like:
Since this is just a signal and it's associated Photon Noise it's not too exciting.
Still not very exciting; but this is what we would expect from a well behaved sensor.
The manufacturing process can result in a small gradient on
We can also get a gradient if our image collection is imperfect.
Here's what the signal image with a gradient looks like:
Well, that's something new.
This cross pattern happens when the left/right or top/bottom edges of the image don't match.
Note the horizontal gradient is stronger than the vertical; and the cross is wider than it is high.
A more common problem with image collection is light falloff
Applied to the signal image it looks like:
This has the same cross effect as the gradient and also a small white center due to the falloff.
For completeness, let's combine them both:
The electronics used to read out the value of the pixels can
sometimes have a vertical or horizontal pattern.
Here I have rearranged the DSNU data from above into vertical bands:
This banding image has exactly the same statistical characteristics as the previous DSNU image shown here for comparison:
So statistically we can't distinguish banding from DSNU.
Banding acts like spatially correlated DSNU even though the origin is the external electronics and not the pixel.
When the pattern of banding remains constant from frame to frame then it is Fixed Pattern Noise (FPN).
We have seen a number of synthetically generated images and
their corresponding 2D FTs.
Now we have a good idea of how to interpret patterns in a 2D FT.
We know that a uniform FT with a white center is ideal.
We know that a cross is probably the result of a gradient, falloff, or banding.
We know that a slightly enlarged white center is probably from a strong falloff.
And we know that a large circular pattern is the signature of signal processing such as noise reduction.
There are other interesting case that I did not cover here
but you now have the tools to understand their 2D FT.
These cases includes visible bonds along the center of large sensors (which may comprise of two pieces of silicon that are bonded together), visible on chip PDAF, etc.