Automating image morphing using structural similarity on a halfway domain

Automating image morphing using structural similarity on a halfway domain
Jing Liao, Rodolfo Lima, Diego Nehab, Hugues Hoppe, Pedro Sander, Jinhui Yu.
ACM Trans. Graphics, 33(5), 2014. (Presented at SIGGRAPH 2014.)
Fast optimization to align intricate shapes using little interactive guidance.
Abstract: The main challenge in achieving good image morphs is to create a map that aligns corresponding image elements. Our aim is to help automate this often tedious task. We compute the map by optimizing the compatibility of corresponding warped image neighborhoods using an adaptation of structural similarity. The optimization is regularized by a thin-plate spline, and may be guided by a few user-drawn points. We parameterize the map over a halfway domain and show that this representation offers many benefits. The map is able to treat the image pair symmetrically, model simple occlusions continuously, span partially overlapping images, and define extrapolated correspondences. Moreover, it enables direct evaluation of the morph in a pixel shader without mesh rasterization. We improve the morphs by optimizing quadratic motion paths and by seamlessly extending content beyond the image boundaries. We parallelize the algorithm on a GPU to achieve a responsive interface and demonstrate challenging morphs obtained with little effort.
Hindsights: One exciting aspect is that just as in the earlier project Image-space bidirectional scene reprojection, the iterative implicit solver is able to evaluate the morph color directly in a pixel shader, without requiring any geometric tessellation, i.e. using only “gather” operations.

Errata: Referring to Figure 16, the text in Section 9 states “Note that the top two examples require no correspondence points” --- it should be just “the top example”.

Example results:
Input image 1 Input image 2

  (Caricature courtesy of Court Jones.)