CS 180 Programming Project 3: Face Morphing

Zhang Yuze

Part 1. Defining Correspondences

We need to select corresponding points on two facial images. I used ginput to select 46 points on the faces and used the Delaunay function from scipy.signal to obtain triangles. For all images, I need to ensure that the points are selected in the same order. To achieve higher-quality images, I select more prominent parts of the face.Below are the points I selected and the corresponding triangle template.

my face and pts

my face and tris

Part 2. Computing the "Mid-way Face"

In this task, I need to simultaneously transform both shape and color to generate the average face of two images. The parameter n is set as the blending ratio between the two images. For the shape transformation, I used the points selected during the 'defining correspondences' step and calculated the interrscte points of the two images in the (1-n): n ratio. Then, I used the Delaunay function to generate triangles. By solving matrix equations, I obtained the transformation matrix between the triangles. I applied this matrix to transform the pixel of each triangle, and finally, I stitched the triangles together to form the complete image. For the color transformation, I blended the color values of the two images in the (1-n): n ratio to get the interrscte color result.

mid way face

Part 3. The Morph Sequence

In the previous task, I already obtained the results of blending two facial images at different ratios. We can use warp_frac and dissolve_frac to control the blending of points and colors, and change them over time to create a video or GIF. Specifically, by adjusting the blending parameters for each frame, we can use the mimsave function from the imageio library to generate a 45-frame GIF, with each frame lasting 1/30 of a second.

morph sequence

Part 4. The "Mean face" of a population

The process of calculating the average face is similar to Part 2, but extends from two images to multiple images. I first calculated the average of all data points, then transformed the corresponding triangles of each image to the average triangle, and averaged the colors. By selecting a subset of images, such as male or female faces, or smiling faces, we can obtain a specific type of average face.

calm man

calm woman

smile man

smile woman

Part 5. Caricatures: Extrapolating from the mean

By calculating the average face, I can determine the differences between my face and the average face in both shape and color. By amplifying these differences, I create a new image where certain features of my face are exaggerated, known as a caricature. At first, I only amplified the shape transformation and ignored the color.

n=0.5

n=1

n=1.5

n=2

n=1

Bells and Whistles

Transition of Gender and Smile

By calculating the differences in shape and color between different types of average faces, face transformations can be achieved. I tried converting one type of face into another, such as transforming a male face into a female face, or a calm face into a smiling face.

original

Using PCA to Extract Feature

I applied PCA to 100 images, projecting them into an n-dimensional space. Using the coordinates instead of color and shape, I created caricatures. As n increases, the images become less clear but align more with my facial features.

original

PCA n=20

PCA n=40

PCA n=60