jupyter notebook/python code
- Download Jupyter notebook through Anaconda Cloud.
- Implement the Python codes snippets given in the chapter below in a Jupyter Notebook .
- Submit your notebook through this workshop link
ATTACHED FILE(S)
22
Detecting Edges and Applying
Image Filters
In this chapter, we are going to see how to apply cool visual effects to images. We will learn
how to use fundamental image processing operators, discuss edge detection, and see how
we can use image filters to apply various effects to photos.
By the end of this chapter, you will know:
What 2D convolution is, and how to use it
How to blur an image
How to detect edges in an image
How to apply motion blur to an image
How to sharpen and emboss an image
How to erode and dilate an image
How to create a vignette filter
How to enhance image contrast
Detecting Edges and Applying Image Filters Chapter 2
[ 38 ]
2D convolution
Convolution is a fundamental operation in image processing. We basically apply a
mathematical operator to each pixel, and change its value in some way. To apply this
mathematical operator, we use another matrix called a kernel. The kernel is usually much
smaller in size than the input image. For each pixel in the image, we take the kernel and
place it on top so that the center of the kernel coincides with the pixel under consideration.
We then multiply each value in the kernel matrix with the corresponding values in the
image, and then sum it up. This is the new value that will be applied to this position in the
output image.
Here, the kernel is called the image filter and the process of applying this kernel to the
given image is called image filtering. The output obtained after applying the kernel to the
image is called the filtered image. Depending on the values in the kernel, it performs
different functions such as blurring, detecting edges, and so on. The following figure should
help you visualize the image filtering operation:
Let’s start with the simplest case, which is the identity kernel. This kernel doesn’t really
change the input image. If we consider a 3×3 identity kernel, it looks something like the
following:
Detecting Edges and Applying Image Filters Chapter 2
[ 39 ]
Blurring
Blurring refers to averaging the pixel values within a neighborhood. This is also called a
low pass filter. A low pass filter is a filter that allows low frequencies, and blocks higher
frequencies. Now, the next question that comes to our mind is: what does frequency mean
in an image? Well, in this context, frequency refers to the rate of change of pixel values. So
we can say that the sharp edges would be high-frequency content because the pixel values
change rapidly in that region. Going by that logic, plain areas would be low-frequency
content. Going by this definition, a low pass filter would try to smooth the edges.
A simple way to build a low pass filter is by uniformly averaging the values in the
neighborhood of a pixel. We can choose the size of the kernel depending on how much we
want to smooth the image, and it will correspondingly have different effects. If you choose a
bigger size, then you will be averaging over a larger area. This tends to increase the
smoothing effect. Let’s see what a 3×3 low pass filter kernel looks like:
We are dividing the matrix by 9 because we want the values to sum up to one. This is called
normalization, and it’s important because we don’t want to artificially increase the intensity
value at that pixel’s location. So, you should normalize the kernel before applying it to an
image. Normalization is a really important concept, and it is used in a variety of scenarios,
so you should read a couple of tutorials online to get a good grasp on it.
Here is the code to apply this low pass filter to an image:
Detecting Edges and Applying Image Filters Chapter 2
[ 40 ]
If you run the preceding code, you will see something like this:
Size of the kernel versus blurriness
In the preceding code, we are generating different kernels in the code, which are
, , and . We use the function, , to
apply these kernels to the input image. If you look at the images carefully, you can see that
they keep getting blurrier as we increase the kernel size. The reason for this is because when
we increase the kernel size, we are averaging over a larger area. This tends to have a larger
blurring effect.
An alternative way of doing this would be by using the OpenCV function, . If you
don’t want to generate the kernels yourself, you can just use this function directly. We can
call it using the following line of code:
Detecting Edges and Applying Image Filters Chapter 2
[ 41 ]
This will apply the 3×3 kernel to the input and give you the output directly.
Motion blur
When we apply the motion blurring effect, it will look like you captured the picture while
moving in a particular direction. For example, you can make an image look like it was
captured from a moving car.
The input and output images will look like the following ones:
Following is the code to achieve this motion blurring effect:
Detecting Edges and Applying Image Filters Chapter 2
[ 42 ]
Under the hood
We are reading the image as usual. We are then constructing a motion kernel. A
motion blur kernel averages the pixel values in a particular direction. It’s like a directional
low pass filter. A 3×3 horizontal motion-blurring kernel would look this:
This will blur the image in a horizontal direction. You can pick any direction and it will
work accordingly. The amount of blurring will depend on the size of the kernel. So, if you
want to make the image blurrier, just pick a bigger size for the kernel. To see the full effect,
we have taken a 15×15 kernel in the preceding code. We then use to apply this
kernel to the input image, to obtain the motion-blurred output.
Sharpening
Applying the sharpening filter will sharpen the edges in the image. This filter is very useful
when we want to enhance the edges of an image that’s not crisp enough. Here are some
images to give you an idea of what the image sharpening process looks like:
Detecting Edges and Applying Image Filters Chapter 2
[ 43 ]
As you can see in the preceding figure, the level of sharpening depends on the type of
kernel we use. We have a lot of freedom to customize the kernel here, and each kernel will
give you a different kind of sharpening. To just sharpen an image, as we are doing in the
top-right image in the preceding picture, we would use a kernel like this:
If we want to do excessive sharpening, as in the bottom-left image, we would use the
following kernel:
But the problem with these two kernels is that the output image looks artificially enhanced.
If we want our images to look more natural, we would use an edge enhancement filter. The
underlying concept remains the same, but we use an approximate Gaussian kernel to build
this filter. It will help us smooth the image when we enhance the edges, thus making the
image look more natural.
Here is the code to achieve the effects applied in the preceding screenshot:
Detecting Edges and Applying Image Filters Chapter 2
[ 44 ]
If you noticed, in the preceding code, we didn’t divide the first two kernels by a
normalizing factor. The reason for this is that the values inside the kernel already sum up to
one, so we are implicitly dividing the matrices by one.
Understanding the pattern
You must have noticed a common pattern in the image filtering code examples. We build a
kernel and then use to get the desired output. That’s exactly what’s happening in
this code example as well! You can play around with the values inside the kernel and see if
you can get different visual effects. Make sure that you normalize the kernel before
applying it, or else the image will look too bright because you are artificially increasing the
pixel values in the image.
Embossing
An embossing filter will take an image and convert it to an embossed image. We basically
take each pixel, and replace it with a shadow or a highlight. Let’s say we are dealing with a
relatively plain region in the image. Here, we need to replace it with a plain gray color
because there’s not much information there. If there is a lot of contrast in a particular region,
we will replace it with a white pixel (highlight), or a dark pixel (shadow), depending on the
direction in which we are embossing.
Detecting Edges and Applying Image Filters Chapter 2
[ 45 ]
This is what it will look like:
Let’s take a look at the code and see how to do this:
Detecting Edges and Applying Image Filters Chapter 2
[ 46 ]
If you run the preceding code, you will see that the output images are embossed. As we can
see from the preceding kernels, we are just replacing the current pixel value with the
difference of the neighboring pixel values in a particular direction. The embossing effect is
achieved by offsetting all the pixel values in the image by . This operation adds the
highlight/shadow effect to the picture.
Edge detection
The process of edge detection involves detecting sharp edges in the image, and producing a
binary image as the output. Typically, we draw white lines on a black background to
indicate those edges. We can think of edge detection as a high pass filtering operation. A
high pass filter allows high-frequency content to pass through and blocks the low-frequency
content. As we discussed earlier, edges are high-frequency content. In edge detection, we
want to retain these edges and discard everything else. Hence, we should build a kernel
that is the equivalent of a high pass filter.
Let’s start with a simple edge detection filter known as the filter. Since edges can
occur in both horizontal and vertical directions, the filter is composed of the
following two kernels:
Detecting Edges and Applying Image Filters Chapter 2
[ 47 ]
The kernel on the left detects horizontal edges and the kernel on the right detects vertical
edges. OpenCV provides a function to directly apply the filter to a given image. Here
is the code to use Sobel filters to detect edges:
cv2.CV_64F.
In the case of 8-bit input images, it will result in truncated derivatives, so depth
value can be used instead. In case edges are not that well-defined the value
of kernel can be adjusted, minor to obtain thinner edges and major for the opposite purpose.
The output will look something like the following:
In the preceding figure, the image in the middle is the output of a horizontal edge detector,
and the image on the right is the vertical edge detector. As we can see here, the filter
detects edges in either a horizontal or vertical direction and it doesn’t give us a holistic view
of all the edges. To overcome this, we can use the filter. The advantage of using
this filter is that it uses a double derivative in both directions. You can call the function
using the following line:
Detecting Edges and Applying Image Filters Chapter 2
[ 48 ]
The output will look something like the following screenshot:
Even though the kernel worked well in this case, it doesn’t always work well. It
gives rise to a lot of noise in the output, as shown in the following screenshot. This is where
the edge detector comes in handy:
Detecting Edges and Applying Image Filters Chapter 2
[ 49 ]
As we can see in the preceding images, the kernel gives rise to a noisy output,
which is not exactly useful. To overcome this problem, we use the edge detector. To
use the edge detector, we can use the following function:
As we can see, the quality of the edge detector is much better. It takes two numbers
as arguments to indicate the thresholds. The second argument is called the low threshold
value, and the third argument is called the high threshold value. If the gradient value is
beyond the high threshold value, it is marked as a strong edge. The edge detector
starts tracking the edge from this point and continues the process until the gradient value
falls below the low threshold value. As you increase these thresholds, the weaker edges will
be ignored. The output image will be cleaner and sparser. You can play around with the
thresholds and see what happens as you increase or decrease their values. The overall
formulation is quite deep. You can learn more about it at:
.
Erosion and dilation
Erosion and dilation are morphological image processing operations. Morphological image
processing basically deals with modifying geometric structures in the image. These
operations are primarily defined for binary images, but we can also use them on grayscale
images. Erosion basically strips out the outermost layer of pixels in a structure, whereas
dilation adds an extra layer of pixels to a structure.
Let’s see what these operations look like:
Following is the code to achieve this:
Detecting Edges and Applying Image Filters Chapter 2
[ 50 ]
Afterthought
OpenCV provides functions to directly erode and dilate an image. They are called erode
and dilate, respectively. The interesting thing to note is the third argument in these two
functions. The number of iterations will determine how much you want to erode/dilate a
given image. It basically applies the operation successively to the resultant image. You can
take a sample image and play around with this parameter to see what the results look like.
Creating a vignette filter
Using all the information we have, let’s see if we can create a nice vignette filter. The output
will look something like the following:
Detecting Edges and Applying Image Filters Chapter 2
[ 51 ]
Here is the code to achieve this effect:
What’s happening underneath?
The vignette filter basically focuses the brightness on a particular part of the image and the
other parts look faded. In order to achieve this, we need to filter out each channel in the
image using a Gaussian kernel. OpenCV provides a function to do this, which is called
. We need to build a 2D kernel whose size matches the size of the
image. The second parameter of the function, , is interesting. It is the
standard deviation of the Gaussian, and it controls the radius of the bright central region.
You can play around with this parameter and see how it affects the output.
Once we build the 2D kernel, we need to build a mask by normalizing this kernel and
scaling it up, as shown in the following line:
This is an important step because if you don’t scale it up, the image will look black. This
happens because all the pixel values will be close to zero after you superimpose the mask
on the input image. After this, we iterate through all the color channels and apply the mask
to each channel.
Detecting Edges and Applying Image Filters Chapter 2
[ 52 ]
How do we move the focus around?
We now know how to create a vignette filter that focuses on the center of the image. Let’s
say we want to achieve the same vignette effect, but we want to focus on a different region
in the image, as shown in the following figure:
All we need to do is build a bigger Gaussian kernel, and make sure that the peak coincides
with the region of interest. Following is the code to achieve this:
Detecting Edges and Applying Image Filters Chapter 2
[ 53 ]
Enhancing the contrast in an image
Whenever we capture images in low-light conditions, the images turn out to be dark. This
typically happens when you capture images in the evening, or in a dimly lit room. You
must have seen this happen many times! The reason this happens is because the pixel
values tend to concentrate near zero when we capture the images under such conditions.
When this happens, a lot of details in the image are not clearly visible to the human eye. The
human eye likes contrast, and so we need to adjust the contrast to make the image look nice
and pleasant. A lot of cameras and photo applications implicitly do this already. We use a
process called histogram equalization to achieve this.
To give an example, this is what it looks like before and after contrast enhancement:
As we can see here, the input image on the left is really dark. To rectify this, we need to
adjust the pixel values so that they are spread across the entire spectrum of values, that is,
between 0-255.
Detecting Edges and Applying Image Filters Chapter 2
[ 54 ]
Following is the code for adjusting the pixel values:
Histogram equalization is applicable to grayscale images. OpenCV provides a function,
, to achieve this effect. As we can see here, the code is pretty
straightforward, where we read the image and equalize its histogram to normalize the
brightness and increase the contrast of the image.
How do we handle color images?
Now that we know how to equalize the histogram of a grayscale image, you might be
wondering how to handle color images. The thing about histogram equalization is that it’s a
nonlinear process. So, we cannot just separate out the three channels in an RGB image,
equalize the histogram separately, and combine them later to form the output image. The
concept of histogram equalization is only applicable to the intensity values in the image. So,
we have to make sure not to modify the color information when we do this.
In order to handle the histogram equalization of color images, we need to convert it to a
color space, where intensity is separated from the color information. YUV is a good example
of such a color space, as the YUV model defines a color space in terms of one
Luminance (Y) and two Chrominance (UV) components. Once we convert it to YUV, we
just need to equalize the Y-channel and combine it with the other two channels to get the
output image.
Detecting Edges and Applying Image Filters Chapter 2
[ 55 ]
Following is an example of what it looks like:
Here is the code to achieve histogram equalization for color images:
Detecting Edges and Applying Image Filters Chapter 2
[ 56 ]
Summary
In this chapter, we learned how to use image filters to apply cool visual effects to images.
We discussed the fundamental image processing operators, and how we can use them to
build various things. We learned how to detect edges using various methods. We
understood the importance of 2D convolution and how we can use it in different scenarios.
We discussed how to smooth, motion-blur, sharpen, emboss, erode, and dilate an image.
We learned how to create a vignette filter, and how we can change the region of focus as
well. We discussed contrast enhancement and how we can use histogram equalization to
achieve it.
In the next chapter, we will discuss how to cartoonize a given image.
