Automating image morphing using structural similarity on a halfway domain

Automating image morphing using structural similarity on a halfway domain
Jing Liao, Rodolfo Lima, Diego Nehab, Hugues Hoppe, Pedro Sander, Jinhui Yu.
ACM Trans. Graphics, 33(5), 2014. (Presented at SIGGRAPH 2014.)
Fast optimization to align intricate shapes using little interactive guidance.
Abstract: The main challenge in achieving good image morphs is to create a map that aligns corresponding image elements. Our aim is to help automate this often tedious task. We compute the map by optimizing the compatibility of corresponding warped image neighborhoods using an adaptation of structural similarity. The optimization is regularized by a thin-plate spline, and may be guided by a few user-drawn points. We parameterize the map over a halfway domain and show that this representation offers many benefits. The map is able to treat the image pair symmetrically, model simple occlusions continuously, span partially overlapping images, and define extrapolated correspondences. Moreover, it enables direct evaluation of the morph in a pixel shader without mesh rasterization. We improve the morphs by optimizing quadratic motion paths and by seamlessly extending content beyond the image boundaries. We parallelize the algorithm on a GPU to achieve a responsive interface and demonstrate challenging morphs obtained with little effort.
Hindsights: One exciting aspect is that just as in the earlier project Image-space bidirectional scene reprojection, the iterative implicit solver is able to evaluate the morph color directly in a pixel shader, without requiring any geometric tessellation, i.e. using only “gather” operations.

Errata: Referring to Figure 16, the text in Section 9 states “Note that the top two examples require no correspondence points” --- it should be just “the top example”.


Input image 1 Input image 2
  (Caricature courtesy of Court Jones.)
ACM Copyright Notice
Copyright by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Publications Dept, ACM Inc., fax +1 (212) 869-0481, or The definitive version of this paper can be found at ACM's Digital Library