Archive

Archive for the ‘Image Processing’ Category

Ink caricature via edge detection and morphology

February 25, 2011 4 comments
Today,
I’m gonna teach you
how to turn …
 

← THIS

 

into THIS →

In this post, I’ll be showing an algorithm for generating an ink caricature out of a face photograph. Using a series of basic image processing techniques we will extract the features of the human face and then reconstruct them into a black-on-white sketch with details similar to brush (or pen) touches. This is useful for those who wish to avoid the work of manually producing such artworks. In addition, this set of methods can serve as initial groundwork for more intelligent and advanced algorithms in creating digital caricatures.

The overall algorithm can be summarized as follows:

Firstly, we choose a good threshold value to produce a satisfactory binary conversion of the raw image.  The parameter value should be sufficient to capture and preserve the important features of the face (i.e. eyes, brows, facial linings, teeth, etc.).

Applying edge detection using a spot filter would obtain the important linings of these facial aspects.

grayscaled image
binary equivalent
detected edges

Notice however that though we were able to capture the details of the face, due to the limitations of the photo (dark hair against dark background), we were not able to capture the details of the hair.  This is because the dark areas were not acquired/obtained by the chosen threshold value.  This can be resolved by applying value inversion, which changes the brightness of the pixels, but not the color.  We use the built-in method in Scilab, rgb2hsv(), to accomplish this and then revert it back its RGB representation.

img = rgb2hsv(im)
// img (:, :, 1) → HUE
// img (:, :, 2) → SATURATION
// img (:, :, 3) → VALUE

img_value = img (:,:,3);
img (:,:,3) = 1 – img_value;
im = hsv2rgb(img);

After applying the same methods performed above, the following pictures show the captured details of the photo.

grayscaled image
binary equivalent
detected edges

Next, we combine the edges detected for the face and the hair and obtain the overall edges for the face photo.

+
=

We remove the pits and holes by applying a simple morphology using a structuring element in a shape of a circle. Note that for a 5×5 matrix this would appear more like a diamond (like the one below), but it is the best shape to approximate the touch of a inkbrush or pen.

After morphology, we invert the processed image into its black-on-white equivalent and finally obtain our ink caricature.

structuring element
resulting image after morphology
inverted value (our ink caricature)

This set of methods also show to be effective for various facial expressions.

Raw images courtesy of DailyMail.

The limitations however are that (1) the methods sometimes need human supervision in choosing threshold values, so to make sure all the valuable features of the face are captured (aesthetic reasons, i.e. eyelashes, dimples, etc.).  (2) The effectiveness would depend on value contrasts, thus the need for value inversions. (3) The caricature detail would depend on the brush/pen size preferred and would greatly affect its accuracy.

———————————————————————————————–—————–————————————
References:
[1] Soriano, 2010. Binary operations. Applied Physics 186.
[2] Soriano, 2010. Fourier Transform model of image formation. Applied Physics 186.
[3] Wikipedia, 2010. Caricature.
[4] Portrait Workshop, 2010. Types of Caricatures.

Advertisements

14. Image compression

September 24, 2010 2 comments

Image compression has been the means of reducing the size of a graphics file for better storage convenience.  It is also a useful way of reducing the time requirement of sending large files over the Web.  Here we will explore a method in image compression by Principal Component Analysis (PCA).  This technique utilizes the idea that any image can be represented as a superposition of weighted base images.

Suppose we have the following image and its grayscale conversion:

Figure 1. A sample image (characteristically sharp)and its grayscale equivalent.  Courtesy of SuperStock.

We divide image into blocks of 10×10 dimensions and concatenate them.  These sub-blocks are arranged into an nxp matrix, where n is the number of blocks and p the number of elements in each block.

We apply PCA on the matrix with the pca() function in Scilab. which returns a set of eigenvalues, eigenvectors and principal components.

Figure 2. Plot of eigenvalues (A) and principal components (B) of the image.

This produces eigenimages which would be essential elements to the compressed image.

Figure 3. Eigenimages derived from the original image.

Eigenvalues tell how essential a particular set of eigenvectors is to making up the completeness of the image.  Based on these values expressed in percentages we choose the most important eigenvectors and reconstruct the image out of these.  Figure 4 shows the resulting images at 86.7%, 93.4%, 95.5%, and 97.5%.

Figure 4. Compressed reconstructions of the image at different numbers of eigenvectors. 1, 3, 5 and 10 respectively.

We can find out how much of the image has been compressed by counting how much of the eigenvector elements were used in the reconstruction and/or determining the file sizes. Our original image has the dimension 280×340 and is stored at 75.3KB (grayscale).  When compressed with only a certain number of eigenvectors (figure above), becomes reduced to 44.2KB, 51.3KB, 53.9KB, and 60.7KB respectively.

When circumstances do not really require high-definition images, it is often best to compress the images into a good size such that it’s quality is not compromised and information is well-kept.

For this activity, I would rate myself 10 for the job well done. 🙂

Credits: Jeff A. and Jonathan A.

———————————————————————————————–—————–————————————
References:
[1] Soriano, 2010. Image compression. Applied Physics 186.
[2] Mudrova, Prochazka. , 2005. Principal component analysis in image processing.
[3] TechTarget, 2010. What is image compression?

13. Color image segmentation

September 23, 2010 2 comments

Image segmentation is the process where a region of interest (ROI) is extracted from the image for further processing. Noting that segmentation is more often not feasible for grayscale images, one of the features this process utilizes is the ROI’s unique true color. This method has been widely used for practical applications such as remote sensing, microscopy and object recognitions.

Parametric vs. Non-Parametric Probability Distribution Estimation

Color-based segmentation can be performed by finding the probability wherein a pixel falls within a distribution of interest.  Parametric segmentation finds the Gaussian probability distribution function (PDF) in the R and G values of the ROI in order then to segment the whole image.  Non-parametric segmentation, on the other hand, finds the 2D histogram of the ROI and uses it to segment the image using histogram backprojection.

Parametric Segmentation

For instance, we have an image of a 3D object with a single color and crop a monochromatic region of interest in the picture (Figure 1).

Figure 1. A 3D object with a single color. A. a red mug, B. region of interest (ROI)

We transform the RGB color of the image into normalized chromaticity coordinates (NCC), such that:

It is important to note that r + g + b = 1, and b is dependent on r and g since b = 1 – r – g.  We implement this process in Scilab as follows:

R = ROI(:, :, 1);
G = ROI(:, :, 2);
B = ROI(:, :, 3);
I = R + G + B;

I (find(I==0)) = 1000000;

r = R ./ I;
g = G ./ I;

The probability p(r) tells how much a pixel with chromaticity r belongs to the ROI:

We find these necessary factors from our r and g values:

mu_r = mean(r);
mu_g = mean(g);
sigma_r = stdev(r);
sigma_g = stdev(g);

Using these values we apply segmentation to the entire image by calculating the probabilities for r and g.

im = imread('3D_object.png');

R = im(:, :, 1);
G = im(:, :, 2);
B = im(:, :, 3);

I = R + G + B;
I (find(I==0)) = 1000000;

r = R ./ I;
g = G ./ I;

Pr = (1/(sigma_r*sqrt(2*%pi)))*exp(-((r - mu_r).^2)/(2*(sigma_r^2)));
Pg = (1/(sigma_g*sqrt(2*%pi)))*exp(-((g - mu_g).^2)/(2*(sigma_g^2)));

The joint probability is taken as the product of p(r) and p(g).

joint_probability = Pr.*Pg;
imshow(joint_probability, []);

Figure 2 shows the extracted portions from the image via segmentation.

Figure 2. Extracted region of interest (ROI) after applying parametric segmentation.

Non-parametric segmentation

Here we will use the 2D histogram of the r and g values of the image to do the segmentation.  After normalizing all pixels, we tag their membership by calculating finding the histogram of the r and g values in a matrix.

BINS = 32;
rint = round(r*(BINS-1) + 1);
gint = round(g*(BINS-1) + 1);

colors = gint(:) + (rint(:)-1)*BINS;
hist = zeros(BINS, BINS);
for row = 1:BINS
   for col = 1:(BINS-row+1)
      hist(row, col) = length(find(colors==( ((col + (row-1)*BINS)))));
   end;
end;

The resulting matrix when rotated such that the origin is at the lower left corner, produces the 2D histogram for the ROI membership, which can be interpreted from the normalized chromaticity space or NCS (Figure 3).

Figure 3. A. 2D histogram for pixel membership; B. the Normalized Chromaticity Space (NCS)

Using this histogram, we can now apply segmentation to the image by means of backprojection. After finding the r and g values:

arsize = size(r);
rows = arsize(1);
cols = arsize(2);

backproj = zeros(rows, cols);

for i=1:rows
   for j=1:cols
      r_val = r(i, j);
      g_val = g(i, j);
      r_new = round(r_val*(BINS-1) + 1);
      g_new = round(g_val*(BINS-1) + 1);
      backproj(i, j) = hist(r_new, g_new);
      // replace pixel with the value
   end;
end;

imshow(backproj, []);

Figure 4 shows the segmented image resulting from the non-parametric method.

Figure 4. Extracted region of interest (ROI) after applying non-parametric segmentation.

Comparing the segmented images obtained from the two algorithms, it can be observed that the non-parametric method has extracted a larger part of the region of interest (ROI).  The parametric method, on the other hand, has segmented sections that can be said to be sharper and smoother with respect to the objects surface (Gaussian approximation).  These two techniques, though different in their results, display good use for different scenes under different conditions.  Knowing the pros and cons of each, we would be able to wisely choose the right method for every need for segmentation.

For this section, I would rate myself 10 for being able to produce the outputs from both segmentation methods.

Credits: Dennis D., Tin C., and Marou R.

———————————————————————————————–—————–————————————
References:
[1] Soriano, 2010. Color image segmentation. Applied Physics 186.
[2] The Scilab Reference Manual. 2010.

12. Color camera processing

September 23, 2010 Leave a comment

A digital image is made of red, green and blue components overlaid at different levels. Each pixel is defined by a summation of the product of the spectral power distribution of the incident light source S(λ), the surface reflectance ρ(λ) and the spectral sensitivity of the camera η(λ).  If a camera has spectral sensitivity η(λ), η(λ), η(λ) for its RGB channels, the color of the pixel is defined as:

whereis the balancing constant that is equal to the inverse of the camera output for a white object ρ(λ).  The equations tell us why some pictures captured by cameras do not appear satisfactory.

White Balancing

Digital cameras have settings called WHITE BALANCE or BW, which allows us to select appropriate constants for specific conditions when capturing images.  Illumination conditions would depend whether it is sunny, cloudy or under the light of a bulbs.  Some cameras include an AWB or Automatic White Balance.  Learning how to set these in the camera allows obtaining an image wherein the white color is properly represented in the image and the other colors are also rendered properly.

Automatic White Balancing Algorithms

There are two popularly known algorithms for automatic white balancing.  The equations for RGB tell us the raw output of the camera divided by the camera output for a white object.  The White Patch Algorithm uses a white object and utilizes its RGB values as divider. The Gray World Algorithm, on the other hand, assumes that the average color of the world is gray.  It obtains balancing constants by taking the average RGB values of the image.

For instance we assemble a group of colored objects (with major hues represented) with a white background.  We choose a specific light source (here, fluorescent lamp) to capture them on image.

We locate a pixel belonging to a white object and with it set the hue values for white (Rw, Gw, Bw).

im = imread('A1. daylight.png');

R = im(:, :, 1);
G = im(:, :, 2);
B = im(:, :, 3);

//white object at (108, 213)
R_white = R(108, 213);
G_white = G(108, 213);
B_white = B(108, 213);

We implement the White Patch Algorithm by dividing all the RGB values of the pixels with their respective white balancing constants.

R_balanced = R / R_white;
G_balanced = G / G_white;
B_balanced = B / B_white;

To make sure that there are no pixels exceeding 1.0, we apply clipping on them to 1.0.

R_balanced(find(R_balanced > 1)) = 1;
G_balanced(find(G_balanced > 1)) = 1;
B_balanced(find(B_balanced > 1)) = 1;

im(:, :, 1) = R_balanced;
im(:, :, 2) = G_balanced;
im(:, :, 3) = B_balanced;

Same process will be performed for Gray World Algorithm, except that the white balancing constants for RGB should be the average value for the RGB planes.

R_white = mean(R);
G_white = mean(G);
B_white = mean(B);

The resulting images from these algorithms are shown in Figure 1 below.

White Balance White Patch Gray World
D
A
Y
L
I
G
H
T
S
H
A
D
E
T
U
N
G
S
T
E
N
F
L
U
O
R
E
S
Figure 1. Captured images at different white balance camera settings and resulting images after applying the White Patch and Gray World aAlgorithms.

From the images above, we can see that raw images captured at daylight and shade settings produce good approximations of the actual fluorescent setup.  However, the daylight image has little influences of blue, as well as tungsten which almost saturates the entire picture. The images corrected show fair results after applying the two algorithms.  The White Patch algorithm produced a good correction of the color values in a moderate way.  The output from the Gray World algorithm, however, put in too much brightness on the image, which we can suppose have resulted from several clipped pixels along the image.

We repeat these steps but now apply them to objects with similar hues and only put a small white object as reference.  Figure 2 shows the results after applying the two algorithms.

White Balance White Patch Gray World
D
A
Y
L
I
G
H
T
S
H
A
D
E
T
U
N
G
S
T
E
N
F
L
U
O
R
E
S
Figure 2. Captured images (of objects with similar hue) at different white balance camera settings and resulting images after applying the White Patch and Gray World Algorithms.

From the images above we can see the difference in accuracies of the resulting images upon performing the white balancing algorithms when the image is not dominated by white colors.  We can see that the images processed with the Gray World algorithm was altered significantly due to the lack of white background.  The White Patch algorithm, however, maintains good rendering of the colors and maintains a good representation of the objects.

From these outputs, the more favorable method to use (and recommend) is the White Patch algorithm for the reason that it is more accurate in representation and more safe for pictures under various settings.

For this segment, I would rate myself 10 for being able to capture and process the images as prescribed by the algorithms.

Credits: Dennis D., Alex C., and Tin C. (for lending me her camera).

———————————————————————————————–—————–————————————
References:
[1] Soriano, 2010. Color camera processing. Applied Physics 186.
[2] The Scilab Reference Manual. 2010.

10. Binary Operations

August 31, 2010 2 comments

Important applications, such as biomedical imaging, need the performance of accurate area detection and measurement, especially in disease examinations (e.g. malaria parasite detection, cancer cell detection). Oftentimes, noisy background obstructs the region of interest (ROI), which causes false interpretations from the image data.  Thresholding an image to binary is not able to make up for these errors due to the illegitimate features still present in the grayscaled specimen.

Methods in morphological image processing offer operators that are able to transform images and clean them from unwanted artifacts.  A good combination of the erosion and dilation techniques accomplish more sophisticated morphological operations that achieve image cleaning.  The morphological opening operator is an erosion followed by a dilation, using the same structuring element for both operations.  It reduces the size of areas that are of smaller size compared to the strel.  The closing operator, on the other hand, performs a reverse of this – it fills in small holes along areas similar to the strel.  A procedure doing opening followed by closing makes an intelligent workhorse for morphological noise removal.

For instance we have a specimen image similar to a microscope photo-capture of human cells (Figure 1).

Figure 1. An image of scattered punched paper digitized using a flatbed scanner. Suppose these circles are cells under the microscope. Courtesy of Soriano, M. [1].

We cut the big picture into sub-images of dimension 256 x 256 with slight overlap.  We perform binarization with a threshold that best captures the important parts of the image with minimal noise in the background.

Figure 2. Binarized sub-images of the cell image at optimal threshold.

For each of these images, we apply the opening and closing operations, using a structuring element shaped as a circle with a size larger than the background noise and about similar to the cells in the image.  This can then be implemented in Scilab as follows:

im = erode(im, se);    // opening, se being the cirular strel
im = dilate(im, se);
im = dilate(im, se);   // closing
im = erode(im, se);

The image, after this noise reduction method, and recombining the sub-images, produces a much excellent capture of the cells – our regions of interest (ROI) (Figure 3).

Figure 3. Cleaned image of the cell circles after undergoing morphological opening and closing operations.

In order to be able to calculate for the areas, we use the function bwlabel to distinguish connected areas in the image.

[L, n] = bwlabel(img);
area = [];
count = 1;

// scan regions for connected areas
for i = 1:n
   area(count) = length(find(L==i));
   count = count + 1;
end

Curiosity would raise a question on how to determine the area of the cells considering there are overlapping cells in the image. The best solution to this is to look into the histogram of the areas detected and find the range at which we are certain that overlaps are not involved.

scf(0); histplot(length(area), area);

This gives us a plot that tells us that most of the detected areas (which in our special case, are the unconnected ones) fall under the range 450-600 pixels (Figure 4A).  We can zoom into this range by creating a separate histogram for it (Figure 4B).

cells = find(area<600 & area>450);
scf(1); histplot(length(cells), area(cells));
Figure 4. Histogram of areas detected in the cleaned image. A. for all areas  B. for areas with no overlapping cells.

We can now calculate for the average area of the cells in this range, implemented by:

all_cells = area(cells);
average_area = sum(all_cells)/length(cells) //area
stdev_area = stdev(area(cells))             //error

Our calculations will result to an area A = 536.28 + 23.17.  To verify the integrity of this value, we attempt to measure the average area of the cells in a sub-image at which we are sure of no overlaps (Figure 5).

Figure 5. 10th sub-image of the cleaned binarized cells.

This gives us an area value A = 533.60 + 25.58, which is of no significant difference to our actual measurement.

Circles with cancer
Now that we know the area of the cells, we can now use these methods for cancer cell detection.  Suppose we have an image of cells now with the presence of enlarged blobs to portray abnormal cancer cells (Figure 6).

Figure 6. Another set of punched papers now with some “cells” bigger than the rest. Courtesy of Soriano, M. [1].

The technique here is to make an intelligent use of the morphological operations such that we can isolated the enlarged cells.  We instead use a structuring element with area greater than our previously measured area A.  We use a circular strel with radius R > sqrt[(μarea + σ) / π].  Oftentimes, R would need a larger value in order to eliminate clustered cells.  This will result to a binary image with the cancer cells isolated (Figure 7).

Figure 7. Isolated “cancer” cells from the set of punched papers.

Current intelligent systems already make more sophisticated applications of image isolation, wherein an image of a target property is captured separately based on a user-specified property identifier.  Nonetheless, it is the morphological operations that make up the fundamentals of these complex machines.

Credits to Dr. Soriano for the valuable tips and discussions.  I would rate myself 10 for doing an awesome job and enjoying the learning at the same time.

———————————————————————————————–—————–————————————
References:
[1] Soriano, 2010. Binary operations. Applied Physics 186.
[2] Mathworks, 2010. Morphology fundamentals: Dilation and erosion. Image processing toolbox.

Now that we know the area of the cells, we can now use this for cancer cell detection

9. Morphological Operations

August 8, 2010 3 comments

Morphological operations, such as erosion and dilation, transform images such that each output pixel is defined by the application of a rule on the input pixel and its neighbors.  The following table (adapted from Mathworks) summarizes the rules for these basic operations:

Operation Rule
Erosion The value of the output pixel is the minimum value of all the pixels in the input pixel’s neighborhood. In a binary image, if any of the pixels is set to 0, the output pixel is set to 0.
Dilation The value of the output pixel is the maximum value of all the pixels in the input pixel’s neighborhood. In a binary image, if any of the pixels is set to the value 1, the output pixel is set to 1.

The neighborhood of a certain pixel is defined by the structuring element used in the operation.  The function applies the rule to the input pixel and its neighbors based on the center point of the structuring element, and then assigns a value to the corresponding output pixel.  Figure 1 illustrates dilation with a 3-pixel horizontal line.

Figure 1. Morphological dilation operation with a horizontal line.  Courtesy of Mathworks.

Morphological erosion and dilation operations follow the Set Theory.  The erosion of A by structuring element B is expressed as:

This results to the reduction of the image by the shape of B.

On the other hand, the dilation of A by structuring element B is denoted by:

This results to the expansion or elongation of A in the shape of B.

———————————————————————————————–—————–————————————

In the succeeding sections, we will see the actual effect of these morphological operations on images with certain shapes when they are operated with particular strels (shortcut for structuring elements).  We have the following initial shapes:

  1. a 5×5 square
  2. a triangle (base = 4 boxes, height = 3 boxes)
  3. a 10×10 square hollow (2 boxes thick)
  4. a plus sign (one box thick, 5 boxes along each line)

The structuring elements with which we will apply the morphological rules with will be the following:

  1. 2×2 ones
  2. 2×1 ones
  3. 1×2 ones
  4. a cross (3 pixels long, one pixel thick)
  5. a diagonal line (2 pixels long, i.e. [0 1; 1, 0])

We apply the operations using Scilab’s erode() and dilate() functions, which are implemented through the following:

E = erode(Img, [SE, center])

where Img is the MxN binary image to be eroded, SE the arbitrary structuring element represented as a binary array, and center the origin of the structuring element.  Similar implementation is done with the dilate() function.

Note that the center of each structuring element should be assigned to a pixel of value 1 or “ON”.  The center for the cross strel will be pixel [2, 2].  For strels with even-numbered dimensions such as first three items listed, the center can be assigned to pixel [1, 1], while diagonal strel can have the pixel [2, 1] as center.

Erosion

The transformations resulting after erosion are shown in Figure 2, where the first row displays the initial shapes and the leftmost column displays the structuring elements.

Figure 2. Morphological erosion of basic shapes by certain structuring elements.

On binary images, erosion reduces the figures by the shape of the structuring element.  On grayscale images, the operation reduces the brightness of objects.  On certain practical applications, it is able to remove small irregularities from the image by subtracting objects that are of smaller radius compared to the structuring element.

Dilation

The results for morphological dilation is shown below (Figure 3).

Figure 3. Morphological dilation of basic shapes by certain structuring elements.

The dilation of binary images connects sections that are parted by spaces smaller compared to the structuring element.  On grayscale images, this function increases the brightness of objects.  Practical applications of this is usually to increase the size of objects and remove holes and broken areas.

Other operations such as skel and thin also impose different rules in processing images.

Using thin performs thinning of the image by means of border deletion using the Zhang-Suen technique.  The function gives an output similar to Figure 4, which was implemented simply by:

out = thin(img)
Figure 4. Morphological thinning of a stick figure.

The function skel, on the other hand, does a skeletonization of the image using the Medial Axis Transform.  Some optional parameters for processing include side [interior (default), exterior or both] and algorithm [fast euclidean (default) or exact euclidean].  It can be performed with the following script:

[skl,dt,lbl] = skel(img, 'interior', 'fast euclidean'])

Below are the figures resulting from varying the side parameter.

Figure 5. Morphological skeletonization of a stick figure using fast euclidean algorithm. Left to right: interior side, exterior side, both sides.

The fast and exact euclidean algorithm make little difference in their results.  It is only that the exact method runs relatively slower than the other.

These basic morphological operations are often used for further image processing and/or extracting information from the image.  An article following this post (soon) aims to make useful application of these methods.

I would rate myself 11 for this activity for completing the tasks and investigating for other morphological functions.

———————————————————————————————–—————–————————————
References:
[1] Soriano, 2010. Morphological operations. Applied Physics 186.
[2] Mathworks, 2010. Morphology fundamentals: Dilation and erosion. Image processing toolbox.
[3] IDL Astronomy User’s Library, 2007. Eroding and dilating image objects. NASA Goddard Space Flight Center.
[4] Wikipedia, 2010. Mathematical morphology.
[5] SIP, 2010. Scilab Image Processing Toolbox. Sourceforge Reference.

Categories: Image Processing Tags: ,

8b. Enhancement in the Frequency Domain

July 29, 2010 8 comments

Improving the quality of an image can often be done by manipulating the image’s frequency aspect.  Here we demonstrate basic applications of eliminating unwanted signals and patterns by means of the Fourier Transform.

Fingerprints: Ridge Enhancement
Clarity of patterns are very important in fingerprint recognition, especially in forensic science.  In cases when fingerprint data are unsatisfactorily represented, a filtering technique can be used to improve its ridges.

Figure 1. An example of an unprocessed fingerprint image.

An enhancement of the elevations and edges can be achieved by creating a filtering mask on the Fourier Transform of the image that allows only the frequencies of the fingerprint to pass through (Figure 2).

Figure 2. Creation of a filter mask for ridge enhancement. A. original FT of the fingerprint; B. Gaussian masks; C. filtered FT of the image.

The FT of the fingerprint is better viewed in the logarithmic scale, using Scilab’s log() function:

imshow(log(fftshift(abs(fingerprint_gray))), []);

The masking is performed by doing a bitwise multiplication on the FT of the fingerprint and the FT of the filter, and then calculating for product’s inverse Fourier Transform.

fingerprint = gray_imread('fingerprint.png');
filter = gray_imread('fft_fingerprint_filter.png');
newI = ifft(fftshift(filter).*fft2(fingerprint));
imshow(abs(newI), []);

The resulting image will be a fingerprint pattern composed of better defined ridges, with clearer elevations and improved contrast (Figure 6).  We particularly used a smooth mask (similar to a 2D Gaussian) to filter the FFT, instead a hard-edged mask (e.g. a circle), in order to avoid the effect of diffraction patterns (an airy disk, in the case of the circle).  This is part of considering the Fourier Transform of the circle, if we recall the properties of the 2D Fourier Transform.

Figure 3. Fingerprint ridge enhancement. A. a grayscale of the original fingerprint; B. an enhanced image of the fingerprint after filtering.

Note that filtering does not at all times need to be done using Gaussian masks.  The filter to be applied would depend on the initial Fourier Transform and is created by carefully identifying which of the frequencies belong to the desired parts of the image.

Lunar Landing Scanned Pictures: Line Removal
Pictures taken from space are sometimes developed on-board the spacecraft and then sent to Earth in their digitized form.  An example is the following image captured by the Lunar and Planetary Institute (LPI) during the Apollo 11 Mission.  The noticeable vertical and horizontal lines are said to have resulted from combining image “framelets” to form the composite image.

Figure 4. A composte image of the surface of the moon. Courtesy of USRA.

We will expect the frequencies of these lines to appear on the FT of the image along the x- and y-axis of the Fourier plane.  We can eliminate these unwanted patterns by creating a mask against these peaks.

Figure 5. Creation of filter mask for line removal. A. original FT of the image; B. small Gaussian masks on peaks along the frequency axes; C. filtered FT of the image.

Note that this time we used black shaded patterns on a white canvas.  This is to cancel to zero the occurrence of the lines and allowing only the frequencies of the desired parts of the image to pass through.  The result is shown in Figure 6.

Figure 6. Line removal technique in the frequency domain. A. a grayscale of the original lunar landing photo; B. an enhanced image after filtering.

Notice that the presence of the lines on the photo are now minimized (if not totally removed) in the resulting image.  The quality of the output image can be further improved by greater accuracies in filtering.

Canvas Weave Modeling and Removal
The original beauty of a painting or artwork can sometimes be obstructed by patterns from the surface of the canvas.  An example is the following image of the detail of an oil painting obtained from the UP Vargas Museum Collection [1].

Figure 7. Detail of an oil painting. Courtesy of the UP Vargas Museum.

The underlying weave patterns are characterized by diagonally-oriented lines, as shown in its Fourier Transform in Figure 8A.  We remove these lines by blocking their frequency peaks with our mask filter.

Figure 8. Creation of filter mask for weave removal. A. original FT of the image; B. small Gaussian masks on diagonal peaks; C. filtered FT of the image.

This process will result to the reduction of weave patterns from the canvas.  Better results can achieved by creating the mask such that it more accurately blocks the unwanted artifacts.

Figure 9. Weave pattern removal technique in the frequency domain. A. a grayscale of the original oil painting photo; B. an enhanced image after filtering.

The filters used for this case are made large enough in order to securely mask against the frequency peaks of the weave.  The ideal filter would most closely resemble smaller masks with highly accurate placements over the Fourier plane.  If we invert the color value of this mask filter and get its Fourier Transform, we will see the pattern of the canvas weave that we are trying to remove.

Figure 10. The weave pattern in the frequency domain. A. Color invert of the mask filter used; B. Fourier Transform of the filter.

These are only some of the basic applications of enhancing images in the Fourier plane.  Utilization of these methods does not limit to the examples presented above.  Fourier enhancement may also be used in to achieve other goals such as image sharpening, which is achieved by bandpass filtering process.

For this set of activities (8A and 8B), I would rate my 10 for the accomplishment.  I sincerely enjoyed this activity. 🙂
———————————————————————————————–—————–————————————
References:
[1] Soriano, 2010. Enhancement in the frequency domain. Applied Physics 186.
[2] Lie, 2010. Image enhancement in the frequency domain. CCU, Taiwan.

Weisstein