Semi-automatic adaptation of high-polygon wireframe face models through inverse perspective projection

Abstract

Precise registration of a generic 3D face model with a subject's face is a critical stage for model based analysis of facial expressions. In this study we propose a semi-automatic model fitting algorithm to fit a high-polygon wireframe model to a single image of a face. We manually mark important landmark points both on the wireframe model and the face image. We carry out an initial alignment by translating and scaling the wireframe model. We then translate the landmark vertices in the 3D wireframe model so that they coincide with inverse perspective projections of image landmark points. The vertices that are not manually labeled as landmark are translated with a weighted sum of vectorial displacement of k neighboring landmark vertices, inversely weighted by their 3D distances to the vertex under consideration. Our experiments indicate that we can fit a high-polygon model to the subject's face with modest computational complexity.