Turn your webcam into a 3D scanner

Bart · November 28, 2009

Several people reported this: Cambridge University Qi Pan is developing software that will reconstruct textured 3D models from webcam video. A Linux and later a Windows downloadable demo should be available in a couple of months. The results are already quite amazing – check out the video inside.

Links

Author Bart

I have a LONG history with Blender - I wrote some of the earliest Blender tutorials, worked for Not a Number and helped run the crowdfunding campaign that open sourced Blender (the first one on the internet!). I founded BlenderNation in 2006 and have been editing it every single day since then ;-) I also run the Blender Artists forum and I'm Community Lead at n8n.

49 responses

Iconoclast

28 November 2009 at 12:48

This looks awesome, keep up the good works.

Robin

28 November 2009 at 12:54

Wow

Vince

28 November 2009 at 12:56

I`m just speachless…. wow!

tmr232

28 November 2009 at 13:26

This is really cool!
I wonder how would something like that can handle holes, though. But I guess this will be done in the future…

It reminds me of this – http://www.robots.ox.ac.uk/~gk/youtube.html .

Sergio Frias

28 November 2009 at 13:35

THIS IS AMAZING!!!! ,just imagine how easy it will be to digitalize your face/head or the entire body…i think it will be better that DAVID 3d scanner,wich require to have a laser .
hope it will be free as well ;)

Nik

28 November 2009 at 14:00

VERY Impressive! The applications for this are endless, especially for markerless augmented reality systems! I take my hat off to the person who made it.

Jonathan

28 November 2009 at 14:08

…I need… a webcam… XD

tntermini

28 November 2009 at 14:48

that’s revolutionary!

lile

28 November 2009 at 15:37

amazing !!

the fun is that google ads are for ‘affordable 3d scanners’ on this page :)

can’t wait to see the followings of this story

ByronK

28 November 2009 at 16:57

That is truley amazing!

nemesis

28 November 2009 at 17:20

Now, this is something I need.

Thanks for the article.

kenny_lex

28 November 2009 at 17:21

Woow. I did ask in a Blender forum if it was possible to convert a point cloud to a 3D mesh, and here is the answer. With better calculations algorithms it will soon be possible to film a real house (or landscape) and get a rough 3D mesh of it.

Cool with the reel time functionality, this make the 3D future interesting.

MeshWeaver

28 November 2009 at 17:32

one word: AMAZING!!!

ooh, can’t wait until the demo!!!!!!!!!!!!

and it even adds the TEXTURES!!! wow

madturtle

28 November 2009 at 18:02

Cool !!!!
….in the movie tron the need a installation large then a building – now you need a web-cam and an impressive
piece of software !! ;)

_L_

28 November 2009 at 18:41

Wow :)

Eibriel

28 November 2009 at 19:20

Wow! :O

pildanovak

28 November 2009 at 19:47

actually look at papers from Polyfeys from the 90’s. Similar algorithms are around for years, just that nobody took the effort to make them realtime & opensource.
great news.

smokin juan

28 November 2009 at 20:00

You can sign up for notification of the proforma demo release here: http://mi.eng.cam.ac.uk/~qp202/
In the meantime, you can generate point clouds with voodoo camera tracker: http://www.digilab.uni-hannover.de/docs/manual.html
Good luck making a mesh from the output, but it could be a usefull reference tool.

Jordan Harris

28 November 2009 at 20:31

Holy cow…

That’s pretty insane. I mean, the results aren’t perfect or anything, but it’s a huge step.

~ J

txrx

28 November 2009 at 20:53

That’s awesome, great work and the way of the future etc! :D

Santiago

28 November 2009 at 21:18

Thats just amazing, i actually have been working with blender for a short time, not very good at modelindo, but im great with textures and riggings/animation. This would solve my modling problems, and i could just make real life models and then get them into the pc with this software.

Hopefully it would be free… because if it is comercial, im sure it would cost a lot of money.

Blendiac

28 November 2009 at 21:47

How does it know not to include his hand?!

ggFEL

28 November 2009 at 22:06

How does this affect the jobs of modelers in the near future?

Of course is amazing, it just worries me how automatized is everything. The mesh is not perfect but a Re-mesh algorithm can solve that.

I hope is not something to worry about.

Cyrr

28 November 2009 at 22:09

Segmentation Blendiac. It takes the point cloud and tries to find points that create a face. The face has to be plane. Then it tries to find faces that have the same edge and with these edges it is possible to add the next face. I assume the algorithm uses some coplanarity issues.
We develop at our company the same stuff. We take LIDAR data from airial planes flying over cities and trying to reconstruct complete cities.
the problem is that houses are not as simple as shown in this demo. Many cities have very complex roof shapes and of course you have gables, chimneys, antennas and all that stuff.
Therefore we only reconstruct LOD2 models (Level of Detail). These are generalized buildings.
As pildanovak already mentioned – the algorithms are old.
We work together with the University of Vienna
http://www.ipf.tuwien.ac.at/cd-labor/31.0.html

txrx

28 November 2009 at 22:14

I’m wondering how complicated the mesh can get using this method (and how it didn’t pick out his hand, I suspect that there’s some form of straight edge tolerance on this/mixed into the motion tracking)

Where abouts were the webcam and laser scripts I saw a while back? I’ve gotta google that again since I’ve got a single beam laser (for aligning my astronomy telescopes) but need to make some form of relflective mirror on a small motor to throw the laser up and down onto the turntable and the subject matter.

Wasa

28 November 2009 at 22:17

While it is pretty neat, it seems it still has a way to go developement wise.
The model used is very simple, yet the mesh seems quite messy, and a
bit inaccurate when it comes to concave surfaces.
I’m sure this is something that will see some improvement in the future,
but concerns me that they are showing it off either this early in
development, or that it’s still this poor at handling it at this late in
development.

japrogramer

28 November 2009 at 22:39

now imagine having a computer controlled 3-axis turntable in a dark room with laser to light the model wherever the computer detects inconsistencies and to further map the object with greater precision … also multiple cameras oriented towards the model from multiple angles ,
and all available for order-in a small kit
that would result in a market for this product and with a market there’s sure to be some competition – with this competition the product will become better and smaller (more implementable in any location )
with a broader market we could all soon have this at our disposal ,in our arsenal of open source tools ,
new comers to the 3d industry would be able to create stunning work with little effort
and the world will become a better open source world , with less arguments that lead to global disputes
resulting in a more peaceful world – creating an environment where human beings can grow to accept each others differences

for brighter future,
keep up the great work !!!

namekuseijin

28 November 2009 at 22:43

Great, but I’m much better with the usual polygonal editing than with any manual crafts… :)

Woutervd

28 November 2009 at 23:43

What, no mac! I might want to look at linux once again in the near future…

rubicon

29 November 2009 at 00:05

Quite impressive in that small scale.
I’m not too much in the textures, but a robust reconstruction of a bigger model (a milling machine for example)
in a not-so-well-lighted environment (a factory building) with changing lighting. Where you can’t move the model, but have to move the camera…… and you get out a (relative) good mesh, that would kick all 3D-Laserscanners out off business. Be careful, they will offer you lot’s of money ;)

4museman

29 November 2009 at 00:18

WOW! This looks really interesting!

But I also wonder, that it doesn’t track the hand even in initialisation stage, when point cloud is created by tracking moving portions of the image. No single point placed on the hand. Ok, the magic is probably the “…motion estimation followed by bundle adjustment” part, which I dont fully understand. :)

I think it would be very funny to take the camera and move it through the room. I’d love to see the result of this! :-D :-D

Can’t wait to check this software out!

Mpu.Bater

29 November 2009 at 02:58

Great stuff!
I hope it won’t make me jobless in the future ;p

Charon

29 November 2009 at 03:28

Not prefect but pretty amazing! Make things affordable and fast!!

wysiwyg

29 November 2009 at 03:41

wow! will this be open source or at least freeware?

ralmon

29 November 2009 at 05:18

Wow awesome. Had to be developed further though to be usable, but still, it rocks for experiments.

ibkanat

29 November 2009 at 05:59

Meshlab(open source) can create meshes from point clouds. You can try it out using radio heads free point cloud data off google. This is a interesting blog about similar tech http://binarymillenium.com/
This was impressivre to me as well http://www.youtube.com/watch?v=TuHJUS2olyc

amonymous

29 November 2009 at 23:10

It does include his hand (and the background) in the texture after 1:30, but then removes most of it

Tim, LA

30 November 2009 at 10:10

Sorry guys, but u could use this only for scanning boxes and cans!!
This because feature tracking und probabilistic algorithms are used!!
For all the other objects take the David Scanner or more professional (expensive!!) systems!

gvrixtel

30 November 2009 at 11:35

Wonder if in the near future video files can be turned into 3d environments by using tools based on this software. This would enable you to get more accurate models of buildings and people long gone or even give you the opertunity to have a look around inside movies.

Nikolas Karampelas

30 November 2009 at 12:57

Amazing, but will there be a MacOS version? :)

memals

30 November 2009 at 14:41

Tim, LA: its pretty obvious that at this point the software is limited to very basic shapes, but all things can be created from basic shapes/faces. Wait for USB 3.0 to bring some serious resolution into the mix and then the fun can begin

Tim, LA

30 November 2009 at 17:13

@memals: I think you better take this one!
http://www.youtube.com/watch?v=XSrW-wAWZe4

sqgsgds

30 November 2009 at 19:09

Nobody thinking about how this could transform robots abilities to see and interact with the world around them.

djallalnamri

30 November 2009 at 21:21

no comment…if this really can be done!

epat

1 December 2009 at 14:39

That’s pretty cool. I came across a program a number of years ago that could do this but you had to take seperate images and manually load them into it. When I saw it I thought, ‘wouldn’t it be nice to have a program that could do this from a video instead’, and here it is… :)

I hope they decide to open source it! That would be really nice.

Anders Otte

1 December 2009 at 16:03

This is a very nice progress on 3D model recognition.
it reminds me of the presentation by Thomas Vetter and Volker Blanz in Siggraph 1999:
A Morphable Model for the Synthesis of 3D Faces
Can’t we work towards a similar 3D recognition from a single 2D image combined with a reference model database? It’s probably an extremely enormous task !!!
Some links:
Video on ChaosComputerClub (PLEASE LOOK AT THIS VIDEO):
chaosradio.ccc.de/ctv088.html
Some Volker Blanz pages
http://www.mpi-inf.mpg.de/~blanz/
http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/b/Blanz:Volker.html
Some Thomas Vetter pages:
http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/v/Vetter:Thomas.html
en.scientificcommons.org/thomas_vetter

Best regards, Anders Otte