Final Project: Image Quilting

Randomly Sampled Texture

Objective: Generate a basic output texture using randomly sampled patches from the input texture to understand the foundational process of texture synthesis.

This step involves generating an output image by randomly sampling square patches from a given input texture sample. The patches are tiled sequentially starting from the upper-left corner until the output image size is reached. Any remaining space that cannot be filled with a full patch is left with black borders. First, I will match the element in the top-left corner of the image to the top-left corner of the output. Then, recursively in row and column order, each patch will be filled with random patches from the original image. In this step, we do not need to consider the overlap between patches.

Results

Overlapping Patches

Objective: Improve texture coherence by using overlapping regions between patches and selecting patches based on similarity.

In this step, overlapping patches are used to ensure better coherence between adjacent patches. Overlapping regions are evaluated based on their similarity using the Sum of Squared Differences (SSD). Patches are chosen randomly from those with SSD costs below a defined threshold (tol) To enhance the generative capability of the algorithm, we select the tol patches with the lowest cost and randomly choose one from these patches to fill in. When tol is large, the algorithm has stronger generative ability, but mismatched patterns may occur; when tol is small, the patterns are more uniform, but the ability to generate new content is limited.

Results

Seam Finding

Objective: Eliminate visible seams in overlapping patches by computing optimal seams for blending overlapping regions.

This step introduces seam finding to remove visible edge artifacts in overlapping patches. A minimum-cost path is computed through the overlapping region, blending pixels from both patches based on similarity. To avoid obvious edges in the stitching between patches, we find the path with the minimum cost and create a mask based on this path. The parts of the patch corresponding to the mask value of 1 are then stitched into the image, achieving a better visual effect.

Results

Additional Quilting Results

Objective: Demonstrate the versatility and effectiveness of the quilting algorithm by applying it to new input textures and producing high-quality results.

Apply the quilting algorithm (with seam finding) on different input textures to produce creative and realistic outputs. The output images should be larger than the input texture sample to demonstrate the algorithm's scalability.

Results

Texture Transfer

Objective: Apply a texture to a target image while preserving the target’s structure and incorporating the texture’s appearance.

Texture transfer involves applying the texture from a source sample onto a target image, preserving the target's structure while blending in the source texture's appearance. The implementation steps are similar to the previous section. The main difference lies in the calculation of the SSD cost image, which also includes the difference between the output image and the target image. For simplicity, set the output image to be the same size as the target image. During each step of generating the output image, use the new cost to calculate the seam and mask.

Results

Bells & Whistles: Iterative Texture Transfer

Gradually refine texture transfer results by iteratively improving the mapping between the source texture and the target image.

TThe texture transfer is performed iteratively on the target image, allowing the texture to match the target image at different scales. By adjusting the patch size and alpha value during the iterations, better results can be achieved.

Results

texture

texture

target

target

texture transfer

texture transfer

iterative texture transfer

iterative texture transfer