Denys Hsu is a 3D Artist & Indie Developer in Karlsruhe, Germany. He has created a wonderful plugin for Blender called BlendArMocap, which essentially uses a webcam to motion capture hands, head and body and easily transfer to a rig...The plugin is tiny, about a half megabyte, yet is powerful enough to capture intricate motion and save it to Blender layout.
The plugin incorporates Google Mediapipe research code which can capture hand gestures, face motion and body motion.. But there has never been a bridge between the Mediapipe research code and animation software....until now. Based on Google Mediapipe research code, BlendArMocap provides the link between Mediapipe and Blender rigged animation.
Using the plugin is a breeze. Just one button opens a webcam window and records the motion. The plugin uses Google Mediapipe code to grab the hand, face and body motion and transfers the animation to Blender. This plugin has the potential that has been obvious for many observers of the Google Mediapipe code, yet has never manifested as a usable program until now. This is a godsend program that democratizes the mocap element of animation. Without the expense of mocap gloves or markers, facial markers or face cams and mocap suits, this program uses a webcam to get the job done. While this plugin is in beta, the possibilities are very exciting. This marks the development of motion capture software utilized in Blender that has far-reaching consequences. As the Mediapipe code develops, the plugin can develop with it and be a true solution to easy and accurate mocap.
No tedious registration, the recording starts with the first pose. Rigging is provided with the Rigify plugin, which works well with human characters. once the rig is applied and properly weighted, the mocap is transfered to the rig, and results are instantly visible in the viewport.
Installation must include python code to run efficiently. At this time, administration privileges must run Blender. The plugin is meant to be used with a fully rigged character in Blender. The software transfers the motion to the rig very quickly. Previews can be played in the viewport. The small footprint of the area of capture eases the preparation of a large capture space. One can motion capture literally at the workstation. Of course, the resulting animation can be edited in the graph editor and pose editor for further refinement.
Tests I ran with the beta have achieved amazing results. Recording using subtle hand gestures was very easy: just move one's hands and the program records the movements. The same with arms, head and body. The Mediapipe software was meant to be simple to use: with a cellphone or webcam. The code Denys created builds on the simplicity and incorporates ease-of-use into the Blender plugin.
Denys Hsu has already created a very complete project, even in beta form. Hopefully, more people will recognize his potential in this project.
I presented live with Denys "Motion Capture with Your Webcam, BlendMocap addon and how to use it" at World Blender Meetup Day, March 19th, with the Los Angeles Blender Users Group (LA.Blend). I was scheduled from 10:30 to 11am Los Angeles time.