First Glimpse: Tomato Tracking Branch

Bart · June 11, 2011

A first impression of the GSoC camera tracking by Sergey Sharybin is now available. The video shows how you add trackers and how Blender will do it’s best to follow them through the footage. Awesome stuff!

Link:

The Very First Tomato (right-click and save-as)

Author Bart

I have a LONG history with Blender - I wrote some of the earliest Blender tutorials, worked for Not a Number and helped run the crowdfunding campaign that open sourced Blender (the first one on the internet!). I founded BlenderNation in 2006 and have been editing it every single day since then ;-) I also run the Blender Artists forum and I'm Community Lead at n8n.

47 responses

Fish

11 June 2011 at 16:06

confirmed, it runs :-)
awesome…

looking forward to a basic camera data export script thingie..whatever :-)

http://vimeo.com/24959865

SavageCode

11 June 2011 at 16:10

Looks like it’s going to be ready for Mango! :D

Ziauddin MK

11 June 2011 at 16:21

alien language, but still good to see this!!! =]

LswaN

11 June 2011 at 16:48

Yes, this does indeed look amazing. Definitely looking forward to an export script or native control of the camera, so that we can put this feature to work!

Rio

11 June 2011 at 17:14

This is looking very promising. However there is still one thing crucial thing missing that is required for succesfull matchmoving and that is the ability to enter the camera’s film back as it will dramatically affect the camera’s field of view.

Take the movie he used to demonstrate for example. From the file naming convention i know it is shot on one of the canon DSLR’s however i don’t know which one, it could be the 5DMKII or the 7D. Big deal you’d think, a camera is a camera. Well it’s not. The sensors in these camera’s differ in size. the 5DMKII is a Full Frame sensor while the 7D has an APS-C sized sensor which is smaller.

If he’d used the same say 50 mm lens on both camera’s the field of view would be radically different. Compared to the 5DMKII the 7D looks like the lens used is much longer, about 80mm (50mm x 1.6) and thus your solve would be wrong.

I hope this will be taken into account as it is something I have been missing a long time in blender and it keeps me from using it as a layout and prep program. This and also the lack of proper animated COLLADA import and export aswell as FBX import.

Just my 2 cents, Keep it up!

BlenderBoy

11 June 2011 at 18:48

Rio is right. Although possible to track w/o film back info. the track won’t match exactly. This should be considered during further dev. Nevertheless, this is a welcomed addition to Blender… good job devs. I personally am a fan of SynthEyes; Russ Andersson’s algorithm is much better than MatchMover (in my experience) and it’s pretty inexpensive compared to other solutions out there.

Kinshuk

11 June 2011 at 18:57

Didn’t expect it so soon, love you Tomato! :D

D@nny

11 June 2011 at 19:03

Looks very interesting. But for now it needs a whole lot of tweaking, but ik looks very promising!
Keep up that great work, guys!

Potado

11 June 2011 at 19:04

This looks so awesome! Now we don’t need any of those third party programs to track for us.

Will it be possible for you to export your tracking data to other programs like after effects also?

Magiciandude

11 June 2011 at 19:36

Wow! Looks fantastic! Keep up the great work! Very impressive!

Chucker

11 June 2011 at 20:00

@Potado, well there is a Export Camera and empties exporter available for Blender 2.5

This is great, is there nothing Blender will not be able to do?

Cedal

11 June 2011 at 20:38

whaa awesome! Blender will be in the future THE program for everything!
NO need for AE in tha future! :P :P

Konstantins

11 June 2011 at 22:00

Can’t wait for tracking in Blender :)

ibkanat

11 June 2011 at 22:02

Dont worry the guys working on libmv and blender know what they are doing they will get it right(camera settings(flim back, etc)).

Igna Valer

11 June 2011 at 23:36

I have been waiting this tool for a long time. Thank u!
This is the best news of this year for me.

ejnersan

12 June 2011 at 00:16

@Rio, BlenderBoy

I have been updating and pushing for inclusion of Matt Ebbs sensor-patch for some time now:
http://projects.blender.org/tracker/index.php?func=detail&aid=24427&group_id=9&atid=127

No core developer has responded, and it seems even Sergey Sharybin is hesitant about it… I will keep updating the patch and can only hope the devs realizes the importance of setting the correct filmback/sensor size.

Jasper

12 June 2011 at 00:27

This is great stuff !!

Keir Mierle

12 June 2011 at 01:30

I’m the BDFL from libmv, which is the library Sergey is plugging to do the image processing. I wanted to emphasize to everyone that the first release will not be as fast or robust as SynthEyes or the other commercial tools, and will require manual calibration. We will continue to improve things, but the first release will be a basic tool that will work in the simple cases.

A solid matchmoving pipeline is a big and complicated piece of software that will take some refinement before we get some of the better stuff. So please don’t be too harsh when you first try it out in a month or so; realize that we are going to release early with something basic rather than wait indefinitely until it is perfect.

Thanks for everyone’s patience with this! It’s been a long time coming.

dono

12 June 2011 at 01:39

\o/

Paul VG

12 June 2011 at 01:44

Brilliant news!! Thank you developers

Keir Mierle

12 June 2011 at 01:46

BlenderNation ate my last post after I hit submit. What a waste of time. Here’s approximately what I wrote:

I am the lead developer of libmv, which is the library Sergey has plugged into Blender to do image analysis for matchmoving, in addition to camera solving. I just wanted to post on here to emphasize that the first release will be quite basic when compared to what you get with the big commercial packages like SynthEyes. In particular, there will be no rolling shutter correction, no autocalbration (you will have to print out a checkerboard and calibrate your camera), no autotracking (you will have to manually place and correct the trackers), and you will have to manually specify the 2 frames from which to start the 3D reconstruction.

Starting out with the basics will let us get something to you guys faster, but please keep in mind that building a robust automatic matchmoving pipeline is a large and complicated task that will take time to get right. The first release will not be good enough to replace the commercial tools.

Dane

12 June 2011 at 06:39

Alright :-) camera tracking, now all we need is a real open source video editor (light works has much promise, it is coming along beautifully in that direction) and someone could produce a live action video with some intense CG using entirely open source tools :-)

of course as mentioned above it wont be as robust as some commercial tools, but you don’t need that to get the job done, its just nice :-)

Pawel

12 June 2011 at 06:45

@ Ziauddin MK
Although it’s hardly intelligible, it seems he’s actually speaking English.

Tom

12 June 2011 at 06:50

I hardly understood anyhting… what is this good for?

comeinandburn

12 June 2011 at 07:14

Go Sergey ! Thanks for the update… this is going to be a godsend for doing VFX/3D integration using Opensource.

Whimsy Collective

12 June 2011 at 08:26

Great stuff! Now what happened with Libmv?

Dweezler

12 June 2011 at 13:35

@Tom:

http://en.wikipedia.org/wiki/Match_moving

FloridaJo

12 June 2011 at 14:02

Sebatian, you know all the nice places in town.

rorkimaru

12 June 2011 at 14:17

@Dane Openshot is pretty good as a video editor. It doesn’t have everything but between Blender and it you can get a lot done.

Glad to see camera tracking is coming. I’ve a few projects I’ve kept on ice because camera tracking wasn’t feasible so hurray for the future ^_^

Rio

12 June 2011 at 14:23

@ejnersan: This looks very good, imho i think it should be integrated in the blender camera system ASAP as the lenses in blender don’t make sense at all right now. This ‘simple’ fix should give blender a considerable boost for use as a previs, layout, matchmove tool.

@Keir Mierle: The basics is all i need :) Advanced features are nice (see pftrack 2011 nodal workflow) but if we can’t control our solves on a fundamental level the program is worthless to us. What I expect to see in all matchmoving programs is this (this is my own list btw i’m just a single matchmover):

Tracker related:

– Placing tracks manually
– adjusting track pattern and search regions
– offsetting the tracker pattern from the tracking point
– manually set if the tracker has to take tracking pattern rotation, scale or skew in account
– setting a pattern failure threshold.
– which color channel to track
– selecting different tracking algorithms if available
(better speed, better accuracy, optical flow (i know is maybe a bit advanced))
– centering the viewer on the tracker to be able to judge its quality.
– the ability to enable/disable the tracker for different time regions
– the ability to track 1 frame at a time. in the demonstration the tracking went faster than real time although it can be a nice time saver, it makes it very hard to judge a track and stopping it in time

Solver related:

– Setting the initial frames manually
– Setting solve frame range
– including/excluding tracks from the solve (should already be possible in the tracking stage but doing it here to will prevent tracking information loss)
– setting camera filmback and focal length as well as letting the program figure it out the best it can
– placing constraints on groups of trackers (planar and distance)
– filtering out bad matching trackers by threshold and adjusting the solve accordingly

I’m not sure which parts of this applies to blender and which to libmv.

Sorry if I’m being redundant with this list. I’m sure you’ve got your own list and people working on it but I’d really like this to work so we can all expand our tool set. It’s great what is being accomplished here. Please keep it up!

Rio

Simon

12 June 2011 at 14:26

@ejnersan:
you are doing a great job! I can’t understand why the devs are so hesitant on this. It is extremely important when working with film/photo material. When they start working on mango they’ll soon realize that.

dave62

12 June 2011 at 18:02

@ejnersan good to know that someone keeps focus on this nice patch. i was very happy when i saw it some month ago at matts blog but i wondered when it get into trunk..
even without any tracking stuff in blender it would be a great addition to proper combine cgi with real footage.

keep up good work! thx

yudhir

12 June 2011 at 18:33

Take help from ‘Voodoo’ guys guys!

MSZ

12 June 2011 at 19:20

Again… can’t wait for that feature to be fully implemented. That is very promising tool.

loopduplicate

12 June 2011 at 19:50

Sweet! Fun!

johnny

12 June 2011 at 21:59

great! cant wait for ingratiation with 3d models, way better than other external trackers.

Nik

13 June 2011 at 00:22

Wow hey, could marker tracking like this be used for facial motion capture? Like if it just creates a bunch of animated empties for each face marker… That would also help in cases where the camera is stationary, but there are moving objects in the scene.

Whimsy Collective

13 June 2011 at 15:32

I wonder if it will support planar tracking and automatic tracking Libmv or PATM. Also, Adobe research labs at some Universities had something similar Libmv/Voodoo. By the way, have you seen Mocha for After Effects?

http://www.youtube.com/watch?v=Y9HMn6bd-v8

Whimsy Collective

13 June 2011 at 15:32

I wonder if it will support planar tracking and automatic tracking Libmv or PTAM (or PATM). Also, Adobe research labs at some Universities had something similar Libmv/Voodoo. By the way, have you seen Mocha for After Effects?

Matt

13 June 2011 at 15:41

For all of you who want a good video editor there is cinelerra-cv. It’s comparable to premiere(not after fx)

satom

13 June 2011 at 16:29

great! cant wait to try it…. seems getting simpler using internal features soon… thanks developer

SavageCode

13 June 2011 at 16:39

On video editors, VideoLAN is developing one, which I assume should be very good and solid, just like VLC, let’s hope the project can go on until official release.

Christiaan (AniCator)

14 June 2011 at 14:09

But this is only 2D tracking right? Will there be support for 3D camera tracking/mapping?

eonmach

15 June 2011 at 06:58

Looks awesome, a 2D matching engine sounds like a big idea, but IMHO I think that a 3D matching utility is an entirely different beast.

Christiaan (AniCator)

15 June 2011 at 10:23

I always thought they called 2D tracking Motion Tracking and 3D tracking Camera Tracking but I must be wrong then.

Whimsy Collective

29 June 2011 at 04:57

Rotoscoping using planar tracking as Mocha does it, would be cool!

zahidryk

30 April 2013 at 08:21

awesome can you share footage plz.Thx

47 responses

Leave a reply Cancel