It looks like you're using an ad blocker! I really need the income to keep this site running.
If you enjoy BlenderNation and you think it’s a valuable resource to the Blender community, please take a moment to read how you can support BlenderNation.

Blender Live Shape - Real-Time Displacement Modifier

45

Sebastian Korczak likes to hook up gear to Blender. Check out his latest experiment: Blender Live Shape - a real-time displacement modifier (similar to normal map) based on live grayscale video from webcam. Read on for more of his work.

Sebastian writes:

For years I'm interested in Blender but this year I started to make some electronic project connected with Blender. Probably you like to see my concept of real-time 3D object in Blender controlled by webcam (I called it: Blender Live Shape :])

I use Processing for capturing webcam and generating greyscale images. There is subdivided plane in Blender with displacement modifier. Python script periodically replace modifier texture with new image. Virtual serial port pair make synchronization. At last we receive object with real-time generated mesh. It works in Blender standard mode and in Game Engine. All source code is on my project website. I think it's possible to use some python libraries for capture webcam direct to Blender.

Soon I'm going to make 3D scanner using webcam, Processing and Blender.

Links

About Author

Bart Veldhuizen

I have a LONG history with Blender - I wrote some of the earliest Blender tutorials, worked for Not a Number and helped run the crowdfunding campaign that open sourced Blender (the first one on the internet!). I founded BlenderNation in 2006 and have been editing it every single day since then ;-) I also run the Blender Artists forum and I'm Head of Community at Sketchfab.

45 Comments

  1. Wow, this is very impressive! Is it really possible to make a 3D scanner using this? If so, coupled with some retopo it would make a powerful tool.

  2. Nice! I see a high potential in this technique for other new future features ... e.g., transforming existing objects in real time with your fingers ... building your own worlds and objects not with a mouse or pen. Or something else :-)

  3. Question- is there any other equipment involved, just trying to understand how you get the webcam to talk to blender, I assume you only need blender and a webcam to replicate this amazing experiment?

  4. Genius idea! I would had never thought of doing that!

    It's the little side projects like this that make Blender so interesting to watch.

  5. Great experiment !

    A simple to use 3D scanner usable with Blender would be a great tool !

    By parenting a small cylinder, or any needle shaped object to the Displaced mesh and
    by using duplivert, you can make animated nails boards !

    Good work !

  6. Thanks friends! I'm very glad you like it. Replies for some comments:

    Yes, we can add all modifiers we want and have fun with it.

    Webcam, Processing and Blender, just it, see and try: http://en.myinventions.pl/018/BlenderLiveShapes.html

    Yes, there's some noise I can fix with smooth modifier, Processing filters (or better cam).

    Yeees, 3d modelling with "webcam-pad" :)

    It can be used as a 3d scanner, but object colors and nonperfect light sources make a lot of noise... so I working on scanner with laser pointer (but not real-time...)

    I think it's possible to make full python script for connect webcam direct to Blender...many many ideas, Blender is perfect tool!

  7. Nooow that's really cool...

    I think blender won't be open source anymore if there's more powerful techniques like this oO

  8. It looks like some sort of freaky horror movie material. Of course it would be hard to use such a displacement system for anything useful because the objects are stuck in an otherwise flat surface. However, you could still do some interesting things, like a face looking out of a wall.

    I wonder if it is able to record the mesh deformation as an animation?

  9. I'm wondering if you could also simultaneously use the color image data from the camera to add the color layer on top of the displacement map, via nodes or other method. Kind of like layering skin, with diffuse, bump , specular and color maps. Then you'd have a real image, cool effect.

  10. I searched for posibility of recording deformable mesh...but I can't find suitable python function... maybe in 2.5 will be.

    There is problem with real-time refreshing texture.

    Maybe standard webcam for capturing colorfull texture, and IR webcam with IR diodes for grayscale map...I will try

  11. I searched for posibility of recording deformable mesh...but I can't find suitable python function... maybe in 2.5 will be.

    There is problem with real-time refreshing texture.

    Maybe standard webcam for capturing colorfull texture, and IR webcam with IR diodes for grayscale map...I will try
    OH! You're my new favorite blogger fyi

  12. I think it's possible to make full python script for connect webcam direct to Blender…many many ideas, Blender is perfect tool!

    Yes Sebastian, just use OpenCV+PIL and you're done :) That's the same thing I did when doing the object tracking in the BGE :)

  13. Looks Fun, great idea, just getting into Processing myself.

    A similar idea, video with Red, Green and Blue lights positioned around the subjects face to get a NOR map video? :-)

  14. Hi Sebastian, this project is really useful, since it will allow to "track" relative vertex positions for facial animations, which is a real pain for me, but I see this tool of yours will help me a GREAT deal. Now, I am trying to install it in an acer netbook, and though I already installed the winvdig, it stops running and says that my camera had a problem trying to link the video output to the video renderer. Could you suggest a workaround for this?

    Thanks a lot for enhancing our blender experience with your cutting-edge inventions!! Please, keep them coming ;)

  15. Really really great. However, your instructions on the site aren't very friendly. How do you setup com0com with the webcam? I just never used com0com=P

  16. You should just instal Com0com and rename deafult CNCA0 and CNCB0 with COM10 and COM11. Processing send data to COM10 and Com0com send it from COM10 to COM11. Blender reads from COM11.

    I wrote article about using wirtual serial ports, but just in Polish.

  17. holycrap thats an awesome combination!!!
    I wonder if one could randomize the displacement while it is created to achieve different effects with this,
    like stirring your hands infront of the cam to achieve a vortex waterdrop like effect or just some random chaos:)

    Great work, baffling result!

  18. Awesome work!!! this method opens many possibilities, especially creating 3D scans without having to buy extremely expensive equipment.

Leave A Reply

To add a profile picture to your message, register your email address with Gravatar.com. To protect your email address, create an account on BlenderNation and log in when posting a message.