Welcome to My CS180 Project 2 Portfolio!
Background of the project:
This projec is mainly about how can we do with different Filter and Frequencies include blurring, sharpening, creating hybrid image, and blending. This will require the students to understand how to deal with the high and low frequencies of the images and also how we can intergrate these different frequencies into the images we want.
1.1 Finite Difference Operator
We need to use the the following finite difference in the x, y directions:
On the following Camera Man Image:
Which what we are doing here is basically taking the partial derivative in x and y of the image by convolving the Camera man image here with the finite difference operators above. And we can use the convolv2d from scipy.signal library to achieve this. And we will now get a gradient mafnitude image. We can then turn the the image into an edge image by binarizing the gradient image. We need to try multiple times for this to get the threshold. Here is my process:
X Derivative:
Y Derivative:
Gradient Magnitude:
Edge Image (threshold: 0.34):
Summary and Notes (Asnswers to Questions):
- When we want to calculate the gradient magnitude, the only thing we need to do is to square every elements in x derivative and y derivative, add them (two 2D matrix) together, and take the square root. Which is similar like how you take the magnitude of a vector.
- You may need multiple tries to get the threshold for binarize. The general idea is that if a certain pixel value is greater than the threshold , then set it as the color white (1.0 or 255).
- There may be some negative pixel value after the convolution, so we need to deal with that. I simply just set it to color black (0) if it is negative.
- There is a liitle border in the original Cameraman image, so we can crop it to avoid the result getting influenced by the border
1.2 Derivative of Gaussian (DoG) Filter
The edge image that we got from the previous part was not good enough! What we can can do is first vlur the image by convolving with a Gaussian filter, and simply repeat what we did in the last part. We can use the getGaussianKernel() method from the cv2 library to create a 1D Gaussian. And we can use np.outer() with its own to get a 2D Gaussian Filter. Here is my blurred image of the camera man:
And now we get the blurred image, we can now repeat the same process using the blurred image to get the edge image
X Derivative:
Y Derivative:
Gradient Magnitude:
Edge Image (threshold: 0.35):
Alternatively, since we know that the convolution is commutative and associative, we can also covolve Gaussian with Dx, Dy first, and we will get the DoG (Derivative of Gaussian) filter. Which we then convolve the image with DoG filter and repeat the process for gradient magnitude and edge image. Note that it will look the same as what we just did by blurring the image first. Here is my process:
Original Gaussian Filter:
Derivative x of Gaussian:
Derivative y of Gaussian:
Note that the filter you see here is a lot smoother the filter printed out in the cell which look like the following (screenshot):
Original Gaussian Filter:
Derivative x of Gaussian:
Derivative y of Gaussian:
This is probally because I saved the image as .jpg file which is a lossy image format. Which will tend to smooth out the details and reduce high frequency noise.
X Derivative:
Y Derivative:
Gradient Magnitude:
Edge Image (threshold: 0.35):
Summary and Notes (Answer to Questions):
- When you call the method convolve2d(), remember to set the parameter "boundary" to "symm". In general, just put "symm" for every call of the method in this peoject.
- We can see that by blurring the image first, the gradient magnitude and edge image are smoother (than part 1.1).
- Note that no matter you blur the image first and take x, y derivative on it to get result, or concolve the Gaussian filter with x, y dervative first and then convolve the DoG on the image to get the result. The final image will look the same. This is because convolution is commutative and associative.
2.1 Image Sharpening
Like what we saw in part 1, the high frequency signal of an image is the structure and fine details where the color and brightness change quickly and obviously. So what we can do is to sharpen the image we want, which means enhance the structure of the image by adding more high frequency of the image to the original image. The process is easy that we can use the Gaussian filter to get the low frequency of the image. And subtract the low frequency of the image from the original image to get the high frequency part. And then add the high frequency back to the original image. Here is my process:
Original Image:
Low Frequency:
High Frequency:
sharppened with alpha = 0.1:
sharppened with alpha = 0.5
sharppened with alpha = 1
sharppened with alpha = 2
Maybe the effect is not that obvious, but if you look at the scaffold on the left, you can clearly see that it is becoming more and more clear. Also note that, instead of blurring and adding back the high frequency sapareatly, we are using the following the formula:
Here are some other Sharpening results:
Original Image:
Low Frequency:
High Frequency:
sharppened with alpha = 0.1:
sharppened with alpha = 0.5
sharppened with alpha = 1
sharppened with alpha = 2
Original Image:
Low Frequency:
High Frequency:
sharppened with alpha = 0.1:
sharppened with alpha = 0.5
sharppened with alpha = 1
sharppened with alpha = 2
Let's see what will happen if I blur and sharp a image that is already sharp
Original Image:
Blur Image:
Sharpenned with alpha = 2:
As we can see, after blurring, the sharpenning effect becomes very bad. The structure of the cat is still reall blurry even after sharpenning.
2.2 Hybrid Image
The eyes of human can't see high frequency signal from far away. Which means we can combine the high frequncy signal of a picture and add it to the low frequency signal of another picture to create an image where it looks differently from close and far. We call such an image as Hybrid Image. Here is my process:
High Freq Image (sigma = 12):
Low Freq Image (sigma = 20):
Hybrid Image:
If you look from close enough, the image is a cat, and if you look from far enough, it will become a man. Here are some other results:
High Freq (sigma = 2):
Low Freq (sigma = 5):
Hybrid Image of Ado and Suisei:
High Freq (sigma = 2):
Low Freq (sigma = 5):
Hybrid Image of Suisei and Miko:
My favorite result is definitely the hybrid image of Ado and Suisei, here is the frequency analysis of the result:
Original fft of Ado:
Low Freq fft of Ado:
High Freq fft of Ado:
Original fft of Suisei:
Low Freq fft of Suisei:
High Freq fft of Suisei:
Hybrid fft of Ado and Suisei:
Here is a failure that I tried at first for Ado and Suisei (using the same image as above). I set both of the sigma too high (s1 = 12, s2 = 20) that the high frequency of Ado is too strong and the low frequency of Suisei is too weak:
2.3 Gaussian and Laplacian Stacks
This part of the assignment is aim to blend 2 images seamlessly using Gaussian and Laplacian Stacks (or Pyramid). We first need to create the Gaussian Stacks and Laplacian Stacks for both image including the mask and inverse mask. After we apply the masks to the image, we combine the both Laplacian plyramid and collapse to get the final result. Here is my process:
Apple:
Orange:
Apple Laplacian Stacks:
level 0
level 1
level 2
level 3
level 4
level 5
level 6
level 7
Orange Laplacian Stacks:
level 0
level 1
level 2
level 3
level 4
level 5
level 6
level 7
Apple Laplacian Stacks after masking:
level 0
level 1
level 2
level 3
level 4
level 5
level 6
level 7
Orange Laplacian Stacks after masking:
level 0
level 1
level 2
level 3
level 4
level 5
level 6
level 7
Final result after combine the 2 masking Laplacian ,and collapsing into 1 image:
2.4 MultiResolution Blending
Now we can implement what we learned in the last section to create our own blending image:
Image 1:
Image 2:
Note that unlike the apple and ornage which has the same exact size, these 2 images are really different in size where I used the alignment code from previous part to get the correct size image. And this blending is using the same mask as above (I didn't use align for the apple and orange):
Mask :
reverse Mask:
Ado5 Laplacian Stacks:
level 0
level 1
level 2
level 3
level 4
level 5
level 6
level 7
Ado6 Laplacian Stacks:
level 0
level 1
level 2
level 3
level 4
level 5
level 6
level 7
Ado5 Laplacian Stacks after masking:
level 0
level 1
level 2
level 3
level 4
level 5
level 6
level 7
Ado6 Laplacian Stacks after masking:
level 0
level 1
level 2
level 3
level 4
level 5
level 6
level 7
Final result after combine the 2 masking Laplacian ,and collapsing into 1 image:
We can also try some irregular mask like the following:
Basketball :
Soccer:
Mask :
reverse Mask:
Final result after combine the 2 masking Laplacian ,and collapsing into 1 image:
Note that I create the irregular mask using photoshop
Final reflection on this project:
This is really a fun project to me, I spent a lot of time tweaking and find the best clicking point to blend 2 faces toegther so that there won't be 2 noses and chins. But I feel really motivated since I can play with the art that I like using code instead of drawing (I'm so bad at art).