Getting familiar with OpenCV 4.0

Introduction

Assignment One (15%)

Due Date: 5:00pm Sunday 23 August 2020

Objectives

• Getting familiar with OpenCV 4.0
• Reading, processing and displaying images using OpenCV 4.0
• Understanding color spaces

Task One

Many color spaces are available to represent pixel values of a color image. Some are perceptually uniform and others are not. This task is to read an image and display the original image in color and its components in a specified color space, such as CIE-XYZ, CIE-Lab, YCrCb and HSB, in gray-scale in a single viewing window as arranged below.

RGB color space is commonly used to represent a digitized images and each color component, also known as a color channel, is digitized into unsigned 8 bits, i.e. the possible values for each component ranges from 0 to 255. (See Appendix: Digital Representation of Images)Note that

255. a JPG image file is often decoded into a RGB image, RGB values of each pixel may be packed together with an alpha channel into 32 bits for efficient access by CPU. The value of alpha channel may be ignored or may be used for transparency as needed. When a JPG image is loaded by an OpenCV function, such as imread(…), please check OpenCV reference how the RGB values of each pixel is packed and stored and make sure you access these values correctly. (See Appendix: Digital Representation of Images)
256. For a color space other than RGB, check the range of its valid component values and you

may need to scale the color component values properly so as to display the components in gray-scale images whose pixel values are between 0 to 255, i.e. 8 bits;

All provided images may be used to test your code.

Task Two

A Chroma key is a technique used in film, television studio and photography to replace a portion of an image with a new image or to place a person, such as a newsreader, on a scenic background. Below shows an example:

The combined photo should be of the same size as the scenic photo, andYou are provided with a few photos of a single person in front of a green screen, referred to as green screen photos, and a few scenic photos. This task is to extract the person in a green screen photo using appropriate Chroma information, e.g. hue and/or other Chroma information, and place the person in a scenic photo according to the following:

1. The person should be aligned horizontally to the middle of the scenic photo

The program should display in a single viewing window the photo of a person in front of a green screen, person extracted from the green screen photo with white background, scenic photo, photo with the person being in the scenic in a single viewing window as illustrated below.

Requirements on implementation

The program should be named as “chromaKey” and shall take one or two image filenames as the input image(s) with/without an option –XYZ, -Lab,-YCrCb or -HSB for different tasks, e..

chromaKey –XYZ|-Lab|-YCrCb|-HSB imagefile # task one

chromaKey greenScreenImagefile scenicImageFile # task two

2. The size of the viewing window for both tasks should not be over the size of 1920×1080 (width x height pixels), but should not be smaller than
3. Aspect ratios of all images must be kept unchanged while they are being displayed in the viewing
4. No other third-party libraries should be used in the program except OpenCV 4.4.0. The code has to be in C/C++ or Python (with OpenCV 4.4.0, assuming numpy and matplotlib packages exist).
5. The code should be modularized with detail comments AND all source code should be placed in a single file “cpp” or “chromaKey.py”.

Marking Scheme

1. Zero marks may be graded if your code cannot run as specified in the
2. Program usability (2%)
3. Correct display of the raw image and color components as required (6%)
4. Extraction and display of the person from a green screen photo (4%)
5. Combination of the scenic photo with the extracted person and display of the combined photo (3%)

Submission

Zip the following source and executable files to your_login_name.zip. The zip file must be submitted via Moodle.

For C/C++ implementation: chromaKey.cpp, chromaKey.exe

For Python implementation: chromaKey.py

# Importing the required libraries
import numpy as np
import matplotlib.pyplot as plt
import cv2
import sys

# split method for Task 1
def split(color_space, Image):
# reading the image input by the user
image = cv2.imread(Image)
# converting image to rgb color space as OpenCV default color space is BGR
rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
# converting image to color space specified by the user using if-else statements
if color_space == ‘-XYZ’:
img = cv2.cvtColor(image, cv2.COLOR_BGR2XYZ)
elif color_space == ‘-Lab’:
img = cv2.cvtColor(image, cv2.COLOR_BGR2Lab)
elif color_space == ‘-YCrCb’:
img = cv2.cvtColor(image, cv2.COLOR_BGR2YCrCb)
elif color_space == ‘-HSB’:
img = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)
else:
print(“Wrong Argument”)
return

# After the image is coverted to specified color space, we split it into its 3 color components using cv2.split
x,y,z = cv2.split(img)

# Making a plot of 4 subplots to generate the required output
figure, axes = plt.subplots(nrows=2, ncols=2, figsize=(19.20,10.80))
axes[0, 0].imshow(rgb)
axes[0, 0].title.set_text(‘Original Image’)
axes[0, 0].axis(‘off’)
axes[0, 1].imshow(x, cmap=”gray”)
axes[0, 1].title.set_text(‘C1’)
axes[0, 1].axis(‘off’)
axes[1, 0].imshow(y, cmap=”gray”)
axes[1, 0].title.set_text(‘C2’)
axes[1, 0].axis(‘off’)
axes[1, 1].imshow(z, cmap=”gray”)
axes[1, 1].title.set_text(‘C3’)
axes[1,1].axis(‘off’)
# Saving the plot as an image in 1920×1080 size
plt.savefig(‘colorspaces.png’, dpi = 100)
# Reading the saved image using cv2
final = cv2.imread(‘colorspaces.png’)

# Display the final result in a new window
cv2.imshow(‘COLOR SPACES’,final)
cv2.waitKey(0)
cv2.destroyAllWindows()

# method to combine the two images (scenic and greenScreen image)
def combine(greenScreenImage, scenicImage):
# Read in the images
image = cv2.imread(greenScreenImage)
background = cv2.imread(scenicImage)
# get the height and width of scenic image as the output image should of the same size as the scenic image
height = background.shape[0]
width = background.shape[1]

# Making a copy of the image
image_copy = np.copy(image)
# Upcaling the image to size of scenic image
image_copy = cv2.resize(image, (width, height),interpolation = cv2.INTER_LINEAR)
# Changing color to RGB (from BGR)
image_copy = cv2.cvtColor(image_copy, cv2.COLOR_BGR2RGB)

# creating a colour threshold to remove the desired green region
lower_green = np.array([0, 100, 0])
upper_green = np.array([126, 255, 100])

# Defining the masked area
mask = cv2.inRange(image_copy, lower_green, upper_green)

# Masking the image to let the person show through
masked_image = np.copy(image_copy)
masked_image[mask != 0] = [0, 0, 0]

#converting background it to RGB
background = cv2.cvtColor(background, cv2.COLOR_BGR2RGB)
background_image = np.copy(background)

# Masking the background so that the pizza area is blocked
background_image[mask == 0] = [0, 0, 0]

# Adding the two images together to create a complete image
final_image = background_image + masked_image

# creating a white background image
white = np.zeros([height,width,3],dtype=np.uint8)
white.fill(255)
white[mask == 0] = [0, 0, 0]

# creating image of person with white background
white_background = white + masked_image

# plotting the 4 subplots to generate the required output
figure, axes = plt.subplots(nrows=2, ncols=2, figsize=(19.20,10.80))
axes[0, 0].imshow(image_copy)
axes[0, 0].title.set_text(‘Photo of a person with green screen’)
axes[0, 0].axis(‘off’)
axes[0, 1].imshow(white_background)
axes[0, 1].title.set_text(‘Photo of the same person with white background’)
axes[0, 1].axis(‘off’)
axes[1, 0].imshow(background)
axes[1, 0].title.set_text(‘Scenic photo’)
axes[1, 0].axis(‘off’)
axes[1, 1].imshow(final_image)
axes[1, 1].title.set_text(‘The same person in the scenic photo’)
axes[1,1].axis(‘off’)
# saving the plot to an 1920×1080 image
plt.savefig(‘chroma.png’, dpi = 100)

# reading the saved image
final = cv2.imread(‘chroma.png’)

# Display the final result
cv2.imshow(‘CHROMA KEY’,final)
cv2.waitKey(0)
cv2.destroyAllWindows()

# Parsing command line arguments and calling the appropriate methods
if __name__ == “__main__”:
arg1 = sys.argv[1]
arg2 = sys.argv[2]
if(arg1 == “-XYZ” or arg1 == “-Lab” or arg1 == “-YCrCb” or arg1 == “-HSB”):
split(arg1, arg2)
else:
combine(arg1, arg2)