Jimmy Gunawan is experimenting with facial motion capture connected to Blender. This is his first demonstration; we have asked him for a more in-depth article on his setup.
In this video, I improvised a bit with the previous setup of Vincent head and facial control and connect map it to my own face via FaceOSC.
The rig is borrowed from Blender Cloud. It's really well done character rig for animation. Love it.
With this OSC setup, I supposed with powerful computer, there should be little or no lag. Overall I am quite happy with the result. Perhaps I should test it on other computer, maybe I can get access to better computer.
2 Comments
It's a cool concept, but the execution still needs work. It's cool to see something working, and I understand we need to see past the lag issues, but there's this huge disconnect between the input performance, and the output animation.
The pupils don't move at all, which takes a lot of life away. Not familiar with FaceOSC and its abilities, but eye tracking in your demo would be huge. The eyebrows are also way too sensitive, they shouldn't oscillate that much for how little you're actually moving your own eyes most of the time.
The mapping of the extracted parameters to the rig's parameters needs quite a bit of work. It seem too easy for expressions where the eyebrow is kind of up, or the mouth is moderately open, to be WAY over-exaggerated - even when the actor isn't making an exaggerated facial pose. On the other extreme, the CG character has got his mouth shut like a steel trap a lot of the time, even when the actor's mouth is loosely open. It has a habit of just moving the closed lips position around but not opening or loosening it.
A very nice progress!
Love it)