Finite Difference Operator

Method & Explanations

The input image is convolved with the following simple finite difference filters to obtain partial derivative images dx and dy representing changes in the vertical and horizontal direction.

The gradient magnitude of the image is then computed as the L2 Norm of dx and dy as follows.

Finally, the gradient magnitude is binarized using a certain threshold to enhance the edges.

Results

Derivative of Gaussian (DoG) Filter

Method & Explanations

The result of finite difference filter by itself is affected by noise, which can be reduced by the process of smoothing. Two methods are explored:

(1) Blurred Finite Difference: perform smoothing on input image with gaussian filter, then perform finite difference filtering as normal.
(2) Derivative of Gaussian: compute derivative of gaussian as a filter and convolve with input image.

The same binarization procedure (with same threshold and setup) is used for comparison.

Results

With smoothing, the resulting gradient magnitude is much clearer as the effect of noise is reduced. Additionally, the results between the two methods are mostly identical.

Image Sharpening

Method & Explanations

Input image is sharpened by enhancing (adding) high frequencies of the image. To obtain the high frequencies, convolution is performed on the image with gaussian kernel as a low pass filter to first obtain the low frequencies, and then high frequencies is just original image - low frequencies. Finally, the sharpened image is obtained from original image + alpha * high frequencies, where alpha is the degree of sharpening.

Unsharp Mask Filter: the above image sharpening procedure can be simplified down to just one convolution operation between the original image and a laplacian of gaussian kernel. The laplacian of gaussian kernel (approximation) is computed as unit impulse - gaussian, and the entire procedure follows the formula f + alpha * (f - conv(f,g)) = conv(f, (1 + alpha) * e - alpha * g) where f = original image, e = unit impulse, g = gaussian. Both implementations are tested and return identical results.

Results

Taj Mahal

Rural Road

NYC (Resharpen Blurred Image)

Input image is initially clear and sharp, it is blurred with gaussian filter first and then sharpened. In this case, the sharpened image is close to the original image but some details are still lost due to the blurring, resulting in overall worse quality, especially for very small and exact details like window frames on the buildings.

Hybrid Images

Method & Explanations

The goal is to create hybrid images that changes based on viewing distance, following the idea that high frequencies tend to dominate at close distances while low frequencies at farther distances.

The input requires a pair of images (well aligned). One image goes through a low pass filter to keep only lower frequencies, while the other image goes through a high pass filter to keep only higher frequencies. The low pass filter used is gaussian kernel, and the high pass filter used is simply original image - low frequencies. The cutoff frequency of each filter is related to the parameters of the gaussian kernel, which is obtained through experimentation. The final output is the average between the low frequencies of one image and the high frequencies of the other.

Results

Big Ben X Campanile

Time Square Over Time (Failure)

After trying different parameters like adjusting cutoff frequency or kernel sizes, I am unable to obtain a satisfactory result. I think this is due to the large difference in the images despite being aligned around the center tower. This results in a lack of resonance between the two that doesn't work well together in the hybrid.

Cat Loaf

Gaussian & Laplacian Stacks

Method & Explanations

Gaussian and Laplacian stacks for an image are structures with levels that correspond to different frequency ranges and are utilized for multi-resolution blending. Unlike image pyramids, there's no downsampling for stacks so each level is the same size.

Gaussian stack: the top level contains the entire frequency range (the original image), and gaussian filter is applied level by level as a low pass filter such that each level going down the pyramid is increasingly more blurred, or lower in frequency range. The recursive formula for constructing the Gaussian stack gs is: gs[level = 0] = input image, gs[level > 0] = conv(gaussian_filter, gs[level - 1]).

Laplacian stack: it's computed as a difference between adjacent levels of the Gaussian stack, so each level of the Laplacian stack represents a range of frequencies of the image similar to that of a staircase. The recursive formula for constructing the Laplacian stack ls using Gaussian stack gs with N levels is: ls[level = N - 1] = gs[-1], ls[level < N - 1] = gs[level] - gs[level + 1].

Results

Gaussian Stack: Apple

Laplacian Stack: Apple

Gaussian Stack: Orange

Laplacian Stack: Orange

Laplacian Stacks Blending Details

Note: to enhance the difference between levels, a modified gaussian stack implementation is used for which the sigma of the Gaussian filter is squared at each layer.

Multi-resolution Blending

Method & Explanations

The Gaussian and Laplacian stacks are used for multi-resolution blending based on the following algorithm:

For input image pair (A, B) and binary mask image M:
→ Step 1: Compute Laplacian stacks LA and LB for A and B
→ Step 2: Compute Gaussian stack GM for M
→ Step 3: Blend each level of LA and LB into L where for level = i, L[i] = GM[i] * LA[i] + (1 - GM[i]) * LB[i].
→ Step 4: Sum up all layers of L to obtain final blended image

Results

Rose Duet

Sun Cat Flowery

Latte Voyage

Otter-zilla Strikes

Key Takeaway

Images consist of a range of frequencies, and by manipulating the images at the frequency level using filters we can create very astounding effects that are hard to do from the pixel level. Low pass filters like Gaussian filters can remove the higher frequencies of the image, which are typically more sharp and exact details of the image. Simply by adding more of high frequencies, the image seems clearer as its details are more sharp.