Remember the scene in the movie Face/Off where FBI special agent Sean Archer aka John Travolta undergoes an experimental surgical procedure to have his face swapped with the freelance terrorist Castor Troy, played by Nicolas Cage. That was quite a seamless fit but a very painful to watch procedure. Face Transplants have been a topic for research for quite sometime and quite a few have already benefited by some successful transplants. In the virtual world, face swapping has been crudely implemented, till now at least!
There have been quite a few apps that allow end-users to place props and masks over their faces. Apple’s new Photo Booth app allows you to have a bunch of birds hover over your head or even have some hearts pop up for the love stricken. These are fun but if you’re like me, you always wanted to address the world with an inspirational speech as none other than Che Guevara himself. Over the years as technology has rapidly developed, real time tracking has become accessible to the masses and now there are tools that allow you to have a metamorphosis of your facial features. What you end up doing is only limited by your imagination and end motives.
This is a technical demo for face substitution technique. The application works in real time and it’s developed using the opensource framework for creative coding openFrameworks: openFrameworks.cc. Most of the “magic” happens thanks to Jason Saragih’s c++ library for face tracking web.mac.com/jsaragih/FaceTracker/FaceTracker.html. The face tracking library returns a mesh that matches the contour of the eyes, nose, mouth and other facial features. That way the mesh obtained from a photo is matched to my own face in the video. Applying some color interpolation algorithms from Kevin Atkinson’s image clone code: methodart.blogspot.com/ gives it the blending effect that can be seen in the final footage.
Here is another video by Kyle McDonald that uses a different blending algorithm, which get rids of the artifacts and an results in an even creepier look:
FaceTracker, is an open source code developed by Jason Saragih that serves as an out-of-the-box tool for designing face alignment and face substitution apps. Non-rigid face alignment and tracking is a common problem in computer vision. It is the front-end to many algorithms that require registration, for example face and expression recognition. Even those directly working on face alignment and tracking often find implementing an algorithm from published work to be a daunting task, not least because baseline code against which performance claims can be assessed does not exist.
Source Code: http://web.mac.com/jsaragih/FaceTracker/FaceTracker.html
OfxFaceTracker Addon: github.com/kylemcdonald/ofxFaceTracker
OpenFrameworks: openframeworks.cc
